Post Job Free
Sign in

Azure Data Enginner

Location:
Texas City, TX
Salary:
70
Posted:
November 16, 2026

Contact this candidate

Resume:

Jaya Gopal Chodagiri

+1-972-***-**** ************@*****.***

Dallas, Texas – 75056

Professional summary:

Senior Data Engineer with 7+ years of experience designing, building, and optimizing cloud-native data platforms and scalable ETL/ELT pipelines across Azure and AWS ecosystems.

Expertise in big data processing frameworks including Apache Spark, PySpark, Databricks, and Airflow for large-scale data transformation and orchestration.

Strong experience in developing high-performance data pipelines using Azure Data Factory (ADF), AWS Glue, and Snowflake, handling millions of records daily.

Strong expertise in Microsoft Azure ecosystem and Microsoft Fabric, including Fabric Lakehouse, Data Factory (Fabric), Dataflows Gen2, and OneLake for unified data architecture.

Hands-on experience in designing modern lakehouse architectures using Azure Databricks and Microsoft Fabric, enabling scalable analytics and real-time data processing.

Experience leveraging Microsoft Fabric for end-to-end data engineering, data integration, and Power BI-driven analytics within a unified SaaS platform.

Proficient in data modeling, data warehousing, and lakehouse architectures using ADLS Gen2, Azure Synapse, Snowflake, and Redshift.

Hands-on expertise in CI/CD pipelines, DevOps practices, and Infrastructure as Code (IaC) using Azure DevOps, Jenkins, Git, and Docker.

Skilled in implementing data security, governance, and compliance frameworks, including RBAC, data masking, encryption, and HIPAA/PII standards.

Experience in building real-time and batch data processing solutions integrating Kafka streams, APIs, and cloud-native services.

Strong background in performance tuning, query optimization, and distributed computing to enhance pipeline efficiency and reduce processing time.

Expertise in enabling AI/ML workflows by preparing feature-rich datasets and supporting model training pipelines.

Proven ability to collaborate with data scientists, analysts, and business stakeholders to deliver scalable and business-driven data solutions.

Experience in developing interactive dashboards and data visualization solutions using Power BI and Tableau.

Adept at working in Agile/Scrum environments, managing end-to-end data engineering lifecycle from ingestion to analytics.

Strong experience in cloud migration, modernization, and legacy system transformation initiatives.

Skilled in building automated workflows and orchestration frameworks to improve operational efficiency and reduce manual intervention.

Self-driven and results-oriented professional with a focus on delivering high-quality, scalable, and secure data solutions in enterprise environments.

Technical Skills:

Programming & Query Languages: Python, SQL, Scala, PL/SQL, Shell Scripting

Cloud & Data Platforms: Microsoft Azure (ADF, ADLS Gen2, Synapse, Azure SQL, Logic Apps, Functions), Microsoft Fabric (Lakehouse, Data Factory, Dataflows Gen2, OneLake, Power BI), AWS (S3, EC2, Glue, Lambda, RDS), Snowflake

Data Engineering & Big Data: Apache Spark, PySpark, Databricks, Apache Airflow, ETL/ELT Pipelines, Data Lakes, Data Warehousing

Databases & Storage: Azure SQL Database, AWS RDS, MongoDB, Cassandra, DynamoDB

Visualization & Analytics: Power BI, Tableau

DevOps & CI/CD: Azure DevOps, Jenkins, Git, GitHub, Docker, Infrastructure as Code

Data Security & Governance: RBAC, Data Masking, Encryption, HIPAA/PHI/PII Compliance, Azure Key Vault, Azure AD

Methodologies & Tools: Agile/Scrum, Eclipse, PyCharm, Maven

Work Experience:

Client: UnitedHealth Group Dallas, TX Jun 2024 - Present

Role: Senior Azure & Microsoft Fabric Data Engineer

Leveraged Azure Cloud, Snowflake, and Databricks to design and automate end-to-end ETL pipelines in ADF, processing ~4M medical claims daily with enhanced reliability.

Optimized ETL processes using PySpark jobs in Azure Databricks for data cleansing, validation, and large-scale transformations, reducing runtime by 60%.

Designed and implemented Microsoft Fabric Lakehouse architecture to unify data ingestion, transformation, and analytics using OneLake and Dataflows Gen2.

Leveraged Fabric Data Factory and pipelines to orchestrate end-to-end data workflows, improving data integration efficiency and reducing latency.

Integrated Microsoft Fabric with Power BI to deliver real-time, business-ready datasets and enterprise reporting solutions.

Migrated data warehouse from Snowflake to Databricks, rewriting SQL logic into Spark-based transformations to boost scalability and cost efficiency.

Implemented robust data security measures including row-level security (RLS), data masking, and RBAC using Azure Key Vault and AAD to safeguard PHI/PII data.

Developed a job orchestration framework with Logic Apps and Azure Functions to streamline workflows and improve operational efficiency.

Collaborated with analytics teams on AI/ML enablement by preparing ML-ready datasets and feature pipelines in PySpark, supporting predictive modeling.

Integrated curated data and ML outputs into Power BI dashboards to provide real-time and predictive intelligence for business stakeholders.

Managed deployments via Azure DevOps and GitHub using infrastructure as code templates to ensure streamlined CI/CD processes and maintain version control in an Agile environment.

Environment: Microsoft Fabric (Lakehouse, Data Factory, Dataflows Gen2, OneLake, Power BI), Azure Data Factory (ADF), Azure Databricks, PySpark, Snowflake, ADLS Gen2, Azure SQL, Azure Logic Apps, Azure Functions, Azure DevOps, GitHub, Docker, Azure Key Vault, Azure Active Directory (AAD), CI/CD, Agile/Scrum

Client: Verizon Dallas, TX Feb 2023 - May 2024

Role: Senior Azure Data Engineer

Engineered and deployed data pipelines using Azure Data Factory and Databricks for customer data processing, enhancing data integration speed and quality.

Collaborated with cross-functional teams, including engineers and data scientists, to build a Customer Data Platform (CDP) managing over 1.5M daily interactions, resulting in an 18% increase in user engagement.

Developed Spark jobs on Databricks to cleanse, standardize, and transform high-volume datasets from SQL Server and Kafka streams, securing data accuracy.

Supported modern data platform evolution towards Azure-native and Fabric-ready architecture for scalable analytics.

Delivered real-time analytics via Power BI, reducing query times by 40% and enabling immediate, data-driven decision-making.

Automated over 50 workflows using Python and Logic Apps, significantly reducing manual effort by 25% and improving pipeline reliability.

Implemented CI/CD pipelines in Azure DevOps and containerization with Docker to ensure smooth deployments and stable production releases.

Supported AI/ML initiatives by building and optimizing training datasets, thereby improving model accuracy and reducing retraining cycles.

Integrated real-time data streams into recommendation pipelines to power AI-driven personalization and targeted marketing strategies.

Enhanced business intelligence by integrating ML model outcomes into Power BI dashboards, shifting analytics from descriptive to predictive.

Environment: Azure Data Factory (ADF), Azure Databricks, PySpark, Apache Spark, Kafka, Azure SQL, Power BI, Python, Azure DevOps, Docker, Logic Apps, CI/CD Pipelines, Git, Agile/Scrum

Client: Cleveland State University Cleveland, OH Aug 2021 - Dec 2022

Role: Graduate Assistant

Supervised laboratory sessions, workshops, and academic activities, providing consistent guidance to students.

Designed and built interactive Power BI dashboards to visualize key academic and operational metrics, driving data-informed decisions.

Provided technical support to faculty and students by resolving issues from support tickets, ensuring smooth operation of academic systems.

Assisted in the transition and development of the college website during the CSU merger, ensuring seamless integration of resources and information.

Environment: Power BI, SQL, Excel, Data Visualization Tools, Academic Systems, Windows Environment

Client: Veritas Soft Solutions, Hyderabad, India. May 2017 - Jul 2021

Role: Data Engineer

Designed, implemented, and maintained production-grade data pipelines on AWS using services such as S3, EC2, Glue, and Lambda, ensuring efficient data integration and processing.

Leveraged Apache Airflow and Python to orchestrate and optimize data operations and processing workflows, enhancing scalability and efficiency.

Implemented data validation and cleansing routines with Python libraries including Pandas and PySpark, improving data quality and consistency across large datasets.

Developed SQL Server tables, views, stored procedures, and functions to support downstream analytics and reporting requirements.

Conducted performance tuning and implemented monitoring solutions for data pipelines, achieving improved query performance and resource utilization.

Integrated on-premises legacy databases with cloud services to support both real-time and batch processing, boosting system reliability by 20%.

Guided technical teams on data engineering best practices and methodologies to ensure aligned, secure, and efficient data solutions.

Led an initiative to automate AWS EC2 resource management, optimizing costs by 15% for critical projects.

Refactored Python code to optimize data pipeline performance, reducing processing time and resource demands in cloud environments.

Conducted data modeling and schema design for operational databases, resulting in a 30% improvement in query performance.

Developed solutions for loading data into a Snowflake data lake environment from AWS EMR, enabling business users and data scientists to leverage Tableau/OBIEE for advanced analytics.

Created AWS Glue jobs and managed catalog table creation with crawlers and manual configurations to facilitate efficient ETL processes.

Implemented a CI/CD pipeline using GitHub and AWS, supporting continuous enhancements and reliable production deployments.

Built and automated the creation of S3 data lake infrastructure with AWS Lambda functions and API Gateways, streamlining subsequent ETL processes to a Cloud warehouse (Redshift) for advanced analytics.

Developed Snow pipes for continuous data injection using AWS S3 event handlers, ensuring timely and seamless data flow.

Environment: AWS (S3, EC2, Glue, Lambda, RDS), Apache Airflow, PySpark, Python, SQL Server, Snowflake, Snowpipe, Redshift, API Gateway, MongoDB, Cassandra, DynamoDB, GitHub, CI/CD Pipelines, Linux, Agile



Contact this candidate