Data Engineer

PROFESSIONAL SUMMARY:

Data Engineer with *+ years of experience in data engineering, ETL development, cloud analytics, and business intelligence across finance, healthcare, and enterprise domains.

Proficient in Azure, AWS and GCP services, including Azure Data Factory (ADF), Azure Synapse Analytics, Azure SQL, AWS Glue, AWS Redshift, AWS S3, AWS SageMaker, Amazon Athena and Lambda, for developing scalable and high-performance data solutions.

Hands-on experience in designing and optimizing cloud-based data warehouses including Snowflake enabling high-performance analytics and scalable data solutions

Strong hands-on experience with Python (Pandas, PySpark, Boto3), SQL, T-SQL, Hadoop

enabling efficient data transformations, batch processing, and real-time analytics.

Designed and optimized cloud-based data warehouses in Azure Synapse, AWS Redshift, and Data Lake, improving query performance and scalability.

Developed and deployed machine learning models for predictive analytics, fraud detection, and demand forecasting using Azure Machine Learning (AML), AWS SageMaker, XGBoost, LightGBM, and Scikit-learn.

Integrates Databricks into scalable ETL workflows to drive efficiency and enable real-time insights.

Experienced in shell scripting for automation of ETL workflows, monitoring jobs, and simplifying data processing tasks in UNIX/Linux environments.

Implemented data governance, compliance, and security best practices using Azure Purview, AWS Lake Formation, IAM, and Role-Based Access Control (RBAC), ensuring GDPR, HIPAA, and SOX compliance.

Expert in API integrations and data exchange solutions using Azure API Management, AWS API Gateway, REST APIs, ensuring seamless communication between enterprise systems.

Automated CI/CD workflows for data solutions using Terraform, CloudFormation, Azure DevOps, AWS Code Pipeline, GitHub Actions, and Jenkins, improving deployment efficiency and reducing manual interventions.

Experience in A/B testing, time-series forecasting, and statistical analysis, improving data- driven decision-making across multiple industries.

Collaborated with cross-functional teams, including finance, marketing, DevOps, and business intelligence teams, ensuring alignment of data solutions with business goals.

TECHNICAL SKILLS:

Cloud & Data Engineering: Azure Services (Azure Data Factory (ADF), Azure Synapse Analytics, Azure SQL, Azure Blob Storage, Azure Functions), Hadoop, Data Warehousing & ETL (Azure Data Lake, Delta Lake, SQL Server, Snowflake, Apache Iceberg, Databricks, Informatica, Data Lake Storage), AWS Services: AWS Glue, AWS Redshift, AWS S3, AWS Lambda, AWS RDS, EC2, AWS DynamoDB, AWS EventBridge, AWS CloudWatch, GCP ((BigQuery, Cloud Storage, DataProc, Cloud Composer, Dataflow, OLAP),

Programming & Scripting: Languages (Python (Pandas, Pytest, PyUnit, Polars, NumPy, PySpark, PyTorch), SQL, T-SQL, Shell Scripting), Hive, Automation & Infrastructure-as-Code (IaC) (Terraform, Bicep, Azure DevOps, CI/CD Pipelines), AWS Code Pipeline, Oracle.

Data Analytics & Business Intelligence: Tableau, Looker, Power BI, AWS QuickSight, Azure ML, AWS SageMaker, DAX, MDX, Predictive Modeling (XGBoost, LightGBM, Scikit-learn), Time Series Forecasting (Prophet, ARIMA), A/B Testing

DevOps & API Management: GitHub Actions, Azure DevOps, AWS Code Pipeline, Jenkins, Bitbucket, REST APIs, Azure API Management, AWS API Gateway. Data Governance (Azure Purview, AWS Glue Data Catalog, RBAC, IA

Compliance: HIPAA, GDPR, SOX

PROFESSIONAL EXPERIENCE:

Client: Huntington National Bank June 2023- Present

Senior Data Engineer

Responsibilities:

Developed custom ETL frameworks in Azure Data Factory and AWS Glue, improving pipeline efficiency and reducing data processing time by 35%.

Designed and maintained Snowflake data pipelines and optimized queries for faster analytics, enabling business teams to reduce reporting latency by 30%.

Implemented anomaly detection models using Azure Machine Learning (AML), AWS SageMaker, and Power BI, improving fraud detection accuracy by 20%.

Designed serverless architecture using Azure Functions and AWS Lambda, enhancing real-time data processing capabilities.

Developed high-performance SQL stored procedures in AWS Redshift and Azure SQL, reducing query execution time by 40%.

Leveraged Google Cloud Platform services such as BigQuery and Cloud Storage to complement our multi-cloud data strategy, enabling seamless cross-cloud analytics and enhanced scalability of our data processing pipelines

Developed and scheduled UNIX shell scripts to automate data validation, logging, and ETL pipeline monitoring, reducing manual intervention and improving reliability of batch processes.

Leveraged Databricks to optimize Spark-based data transformations, enhancing pipeline efficiency and enabling real-time analytics.

Created Power BI and QuickSight paginated reports for executive-level decision-making, improving visibility into key financial metrics.

Implemented metadata-driven dynamic pipeline execution in ADF and AWS Glue, enabling

configurable and reusable workflows.

Environment: Azure Cloud, AWS Cloud, Azure Data Factory (ADF), Hadoop, AWS Glue, Azure Synapse Analytics, Databricks, Delta Lake, AWS Redshift, AWS Lambda, GCP, Azure SQL, Snowflake, Python, PySpark, Power BI, AWS QuickSight, Azure Machine Learning (AML), ML Flow, AWS SageMaker, AWS EventBridge, Azure Event Hubs, Terraform, AWS CloudFormation, GitHub Actions, Oracle.

Client: Wipro August 2020 – December 2022

Senior Data Engineer

Responsibilities:

Migrated legacy on-prem SQL databases to Azure Synapse and AWS Redshift, improving query performance and reducing operational costs by 40%.

Developed data lineage tracking solutions in Azure Purview and AWS Glue Data Catalog, ensuring compliance with GDPR and HIPAA regulations.

Built CI/CD pipelines for data deployments using Terraform, AWS CodePipeline, and Azure DevOps, reducing deployment failures by 45%.

Built distributed caching layers for frequently accessed datasets using AWS DynamoDB and Azure Cosmos DB, reducing Power BI and QuickSight report load times significantly.

Designed and implemented an enterprise-wide data catalog using Azure Data Lake and AWS Lake Formation, improving data discoverability.

Utilized Databricks to develop and deploy scalable ETL pipelines, streamlining data processing and fostering collaborative analytics.

Developed event-driven processing using AWS Lambda and Azure Functions, reducing manual intervention for data ingestion task

Environment: Azure Cloud, AWS Cloud, Azure Data Factory (ADF), AWS Glue, Azure Synapse Analytics, Databricks, GCP, AWS Redshift, Delta Lake, Oracle, Hadoop, AWS S3, AWS Lambda, Azure SQL, Azure Data Lake, AWS DynamoDB, Power BI, AWS QuickSight, Terraform, AWS CloudFormation, Azure DevOps, AWS CodePipeline, AWS Lake Formation, Jenkins, Azure Purview.

Client: Wipro July 2019 – August 2020

Data Engineer

Responsibilities:

Designed ETL workflows for structured and semi-structured data sources using ADF, AWS Glue,

and Azure SQL, improving data integration by 30%.

Developed real-time Power BI and QuickSight dashboards with DAX scripting, providing actionable insights for business stakeholders.

Migrated legacy SQL workloads to Snowflake, implementing partitioning and clustering strategies that improved query efficiency and reduced compute costs

Automated data validation and cleansing processes using Python and AWS Lambda, enhancing data accuracy and reducing errors by 25%.

Integrated Databricks for interactive Spark jobs, significantly reducing batch processing times and improving overall data workflow performance.

Implemented a monitoring system using AWS CloudWatch and Azure Monitor for real-time tracking of ETL pipeline failures, reducing downtime by 40%.

Designed automated data reconciliation workflows in ADF, AWS Glue, and SQL, improving financial data accuracy.

Developed batch processing jobs using PySpark on AWS EMR, enabling faster data transformations for large datasets.

Explored and deployed GCP solutions like BigQuery and Cloud Dataflow to support agile ETL processes, thereby enabling real-time analytics and improved insights for business intelligence dashboards.

Environment: Azure Cloud, Hadoop, Snowflake, Azure Data Factory (ADF), Oracle, Azure SQL, Power BI, Python, SQL, T-SQL, Azure Functions, Databricks, GCP

EDUCATION:

Master’s in data science - University of West Florida

CERTIFICATIONS:

AWS Certified Machine Learning – Specialty

Contact this candidate