Cloud Data Engineer - ETL/ELT, Azure-AWS

Dhanush Chalvadi

Data Engineer

Corpus Christi, TX 361-***-**** *******.********@*******.*** LinkedIn SUMMARY

Data Engineer with 4+ years of experience designing and optimizing cloud-based data platforms, ETL/ELT pipelines, and enterprise data warehouses across Azure and AWS environments. Expertise in Python, SQL, PySpark, Databricks, Snowflake, Spark, and Kafka. Proven track record of building scalable data solutions that support analytics, reporting, machine learning, and AI-driven initiatives. Experienced in Generative AI, Azure OpenAI, and RAG-based data architectures.

TECHNICAL SKILLS

Programming: Python, SQL, PySpark, Scala, Shell Scripting Big Data Technologies: Apache Spark, Hadoop, Hive, Kafka, Databricks Cloud Platforms: Azure Data Factory, Azure Data Lake, Azure Synapse, AWS S3, AWS Glue, AWS Redshift, AWS Lambda

Data Warehousing: Snowflake, Azure Synapse, Redshift, SQL Server Orchestration & DevOps: Apache Airflow, Jenkins, Git, CI/CD, Docker Databases: SQL Server, PostgreSQL, Oracle, MySQL, MongoDB AI & Machine Learning: Generative AI, Azure OpenAI, Machine Learning Data Pipelines, Vector Databases, RAG Pipelines, Feature Engineering, ML Data Preparation BI & Analytics: Power BI, Tableau, Advanced Excel

Methodologies: Agile, Scrum, Data Governance, Data Quality Management, Data Modeling EXPERIENCE

Data Engineer McKinsey & Company Sep 2024 – Present

• Designed and developed scalable ETL/ELT pipelines using Python, PySpark, Databricks, Azure Data Factory, and Azure Synapse to process large volumes of structured and unstructured enterprise data.

• Built and maintained Lakehouse and Data Warehouse solutions using Azure Data Lake, Snowflake, Delta Lake, and dimensional modeling techniques, reducing query execution times by 40%.

• Led the development of scalable cloud-based data solutions using Python, SQL, and Azure technologies, enhancing data accessibility, governance, and performance for cross-functional business teams.

• Developed batch and near real-time data processing frameworks using Apache Spark and Kafka, supporting analytics workloads and processing over 5TB+ of data daily.

• Led cloud migration initiatives by modernizing legacy data platforms and moving critical workloads to Azure, improving scalability and reducing operational overhead by 35%.

• Implemented data quality frameworks, validation rules, and automated monitoring processes that improved data reliability and maintained 99.8% data accuracy.

• Partnered with business stakeholders, analysts, and data scientists to translate complex requirements into scalable and business-focused data solutions.

• Automated deployment, monitoring, and orchestration processes using Airflow, Jenkins, Git, Docker, and CI/CD pipelines to improve platform stability and delivery efficiency.

• Developed AI-enabled data solutions using Azure OpenAI, vector databases, and RAG architectures to support intelligent search, knowledge retrieval, and AI-driven analytics. Data Engineer Avenir Technologies Jan 2020 – Dec 2022

• Developed and maintained ETL/ELT pipelines using Python, SQL, Spark, Azure Data Factory, and AWS Glue to integrate data from multiple enterprise systems.

• Designed dimensional models, fact tables, and star schemas to support business intelligence, financial reporting, and analytical workloads.

• Built scalable data ingestion and transformation workflows using Apache Spark and cloud-native services for enterprise reporting and analytics.

• Optimized SQL queries, Spark jobs, and warehouse processes through performance tuning techniques, reducing processing times by 30%.

• Implemented automated data validation, reconciliation, and monitoring solutions that reduced production data issues by 45%.

• Collaborated with business users, analysts, and architects to gather requirements and deliver scalable data engineering solutions.

• Supported Snowflake, Azure Synapse, and Amazon Redshift environments while ensuring compliance with data governance and security standards.

• Automated recurring data workflows and scheduling processes, saving over 20+ hours per month through improved operational efficiency.

EDUCATION

Master of Science in Computer Science Texas A&M University–Corpus Christi

Contact this candidate