Post Job Free

Resume

Sign in

Cloud, ETL Engineering

Location:
Cary, NC, 27519
Posted:
April 01, 2024

Contact this candidate

Resume:

ARUN ANBARASU

ad4ppa@r.postjobfree.com

908-***-****

ACCOMPLISHED

Cloud Services (AWS) • Cloud Data Engineering • Data Warehouse • Data Lake • Data Modeling • MLOps • Data Analytics • Feature Engineering • ETL• SQL • Data Quality • Data Integration and Governance•

Over 17 years of experience.

TECHNOLOGIES, CERTIFICATIONS, & PROFESSIONAL DEVELOPMENT Technologies:

Snowflake, Python, Unix, Spark, AWS Ecosystem (S3, SQS/SNS, IAM, Lambda, Glue, Sagemaker, Step functions, EventBridge, CloudFormation etc..), Snowpark, Shell Scripting, Oracle, PL/SQL, SQL, PostgreSQL, ETL, Ab-Initio, AWS RDS, Linux, Autosys, Hadoop, Hive, Java 1.8, Kafka 1.0, Control-M

• Certifications:

- Oracle SQL Developer.

- AWS Certified Solutions Architect - Associate (SAA-C02)

- PCEP, Certified Entry-Level Python Programmer

- SNOWPRO Core Certification

• Professional Development:

- SP800, Getting Started with Apache Spark Data Frames

- SP820, ETL Part 1: Data Extraction (AWS Databricks)

- Snowflake - Hands on Essentials

PROFESSIONAL SUMMARY:

- A strong experience on various phases of SDLC application development and implementation for both On-prem and Cloud applications.

- Design, develop, and deploy applications using AWS services such as EC2, S3, Lambda, and SNS/SQS.

- Design cloud workload data pipelines and model orchestration using Sagemaker and AWS Services.

- Worked on Data modeling techniques – Relational model, Dimensional data model and DB design.

- Developed automated deployment pipelines for creation fo AWS resources using AWS CloudFormation and Jenkins Core.

- Feature Engineering – Generate model ready data for scoring ML predictions on various business use cases using Pyspark, AWS Services, Oracle, and Snowflake.

- Developed a Cloud based, configurable post model rule engine for business rule processing using Python, AWS Glue, Snowflake services.

- Expertise on Data warehouse and DataLake ETL development and data processing using Pyspark, Snowflake, Oracle, Cloud services and Abinitio ETL tool.

- Experience on Physical and Logical Data modeling and creating ERD diagram for Oracle and Snowlake based applications.

- Experience with designing and architecting analytics and reporting solutions using Oracle and Snowflake.

- Migrated Oracle ETL based application to modernized and optimized Spark based ETL batch application leveraging Cloud capabilities.

- Built stateless ELT applications using AWS Ecosystems, Snowflake, and messaging services.

- Built application specific Jenkins-Core pipeline for CI/CD process to AWS Cloud and EDL.

- Design and developed oracle PL/SQL batch process with ETL tool (Ab-Abinitio) for data validation, curation, and injection process from multiple sources.

- Adheres to data access, data governance and integration policy in all phases of development.

- Automated application’s BAU process through Shell Script, Autosys & Abinitio Control Center.

- Worked on Agile and Scaled Agile Methodology and principles for application development & implementation.

EDUCATION

University of Madras 2004

Bachelor of Technology, Information Technology

WORK EXPERIENCE

Fidelity Investments Present

Principal Full Stack Engineer

PWI - Unified Customer Segmentation

● Generated model ready data assets for customers segmentation for machine learning models across PI and WI (EDL’s) datasets and implementing using predefined cloud data pipelines.

● Built python based generic business rule engine for processing business rules using AWS services

(Glue, Lambda, Step functions) for seamless process of post model scores.

● Designed the cloud workloads integrating with model implementation orchestration to comply with fidelity standards for customer segmentation process.

● Designed the relational and dimensional data modeling for target segmentation storage.

● Created Snowflake’s end-to-end data model design, model review with enterprise and implementation to production.

● Implemented SNOWPARK process leveraging Snowflake compute capability for data retrieval,, complex joins, data transformation and manipulation purpose.

● Developed a complex Procedure for transformation of model input data from multiple tables and implemented data quality checks.

● Built application specific Jenkins-Core pipeline for CI/CD deployment process along with uDeploy integration to migrate resources to AWS infrastructure.

● Automated uDeploy component changes and addition, modification to uDeploy application through Jenkins pipeline process.

● Automated cloud segmentation tagging process to help with data scientist and business users for research and analysis purpose.

● Closely worked with product owners, Stakeholders and architects as required to influence technical improvements to the platform.

● Worked on Agile and Scaled Agile Methodology and principles for application development & implementation.

UCS Customer Segmentation Dashboard (COHORT)

• Built Customer’s COHORT data collection process leveraging UCS for brokerage personalization and marketing use case.

• Designed the target data model per enterprise standards using dimensional modeling, reviewed with data modelers for implementation to production.

• COHORT fact is a one stop resource for having all customers Assets, Demographics, Net Money Flows and Cohorts (Confidence Prediction, Strategic Customer and Unified Customer Segment) metrics for BI reporting.

● E2E process is developed using Snowflake -EDL and automated through Control-M job.

● Experience with designing and architecting analytics and reporting solutions Discover Financial Services

Lead System Engineer

● Design, Develop, and optimized AWS Cloud ELT solutions for various MartTech and Ads Campaign processes and building generic data ingestion pipeline.

● Onboarded new marketing application’s data feed(s) in AWS for cloud Data processing and build data store in Snowflake for downstream consumption.

● Built Google’s ADS REST API data pipeline in AWS Cloud for Ads Campaign process.

● Created DynamoDB table structures for services requiring low-latency processing and analysis.

● Migrated legacy Oracle - ETL application into AWS platform using prebuilt cloud pipelines and using Snowflake as datastore.

● With REST API, onboarded marketing feeds into cloud DWH for campaign process.

● Optimize complex and poor performing SQL statements to enhance their performance overall.

● Redesigned/Restructured legacy ELT application into stateless decoupled cloud applications using AWS Cloud and Snowflake pipelines.

● Built a curated and integrated data warehouse system from multiple sources for Marketing and Analytical space using AWS and Snowflake.

● Developed QLIK replication process for seamless data copy from legacy Oracle data store to Snowflake.

● Managed and automated legacy Campaign system (Unica) to improve business experience on multiple running campaign workflows.

Synchrony

Lead Data Engineer

● Designed and developed PySpark based ETL ingestion framework for ingesting complex business data feeds.

● Created AWS S3 document store for all curated data sources for cloud-based consumption and Analysis.

● Expert in designing complex ETL applications using Spark framework (Python), Dataframes, Spark SQL, partitioning techniques, broadcast variables, etc.

● Built a generic Spark based data quality checks and customized data transformation process when acquiring source data.

● Created Data standardization layer leveraging Pyspark framework capabilities to bring data in common formats to support data consumers for their analysis and to use data in a consistent manner

(collection, ECE teams).

● Created Data Curation layer using Spark and Big data technologies and preserve data in an effective manner for generation of management reporting, Data Analysis and Data Science drive.

● Transformed and migrated legacy Oracle-ETL (Abinitio) system into Spark backed Big Data system.

● Developed Spark based common routines/packages needed for data validations and transformation needs.

● Automated the application’s BAU process through Shell Script, Autosys & Abinitio Control Center.

● Followed the Agile and Scaled Agile Methodology for application development & implementation. Citigroup

Architect/Tech Lead

● Led the Abinitio ETL design, development, and implementation of DWH application for critical business requirements.

● Built Abinitio ETL pipeline for data acquisition for different sources of financial systems.

● Designed and developed query scheduling engine and data validation tools for large volume of data tests.

● Used bulk processing for better performance and easy retrieval of data, being a part of performance tuning of the procedures and packages.

● Assisted with performance improvement suggestions on data warehouse for large scale of data processing.

● Using Ref cursors and collections for accessing complex data resulted from the joining of a large number of tables in PL/SQL blocks and have experience with exception handling.

• Developed and implemented the complex PL/SQL Procedures, Functions and Triggers for data validation and enrichment purpose.

• Provided high level Support and maintenance of ongoing DWH applications. IBM, Chennai

Senior Software Engineer

● Developed and maintained complex Oracle PL/SQL objects like Package, Procedure and Triggers.

● Provide Backend DB Fixes to any production issues.

● Make code changes for enhancements designed by the Team Lead using PL/SQL.

● Developed PL/SQL scripts for data manipulation and business rules validation.

● Perform Unit Testing of the Database components coded.

● Support the Test Cycle with any bug fixes in the code developed.

● Tuning the SQL Query and PL/SQL Routines which is taking more execution time.

● Support and Maintenance of business applications. CSC, Bangalore

System Analyst

● Code Development and migration to higher environments and productionize.

● Logical and physical data model design and review of team.

● Performance Optimization on complex SQL queries, PL/SQL procedures and Functions.

● Prepare data migration plans including migration risk, milestones, quality and business sign-off detail.

● Conducting unit testing and debugging to ensure the functionality and accuracy of PL/SQL code.

● Documenting technical specifications, data models, and processes.

● Perform source system data analysis and preparation source to target data mapping.

● Prepared the detailed design document and Unit Test Plan based on the Functional Requirements.



Contact this candidate