Data Engineer Azure

HARSHAVARDHAN KALAGARLA

AZURE DATA ENGINEER LinkedIn 443-***-**** **************@*****.***

PROFESSIONAL SUMMARY

• Azure Data Engineer with 4+ years of experience in cloud data engineering, data warehousing, and big data technologies.

• Expertise in ETL development using Azure Data Factory, Azure Synapse Analytics, Azure Databricks, and Apache Spark.

• Skilled in real-time data processing with Azure Stream Analytics, Kafka, and event-driven architectures.

• Proficient in Python, SQL, and experienced with containerization (Docker, Kubernetes) for scalable data solutions.

• Strong experience in Azure DevOps, CI/CD automation, and data security using Azure Key Vault, RBAC, and governance frameworks.

• Hands-on expertise in data visualization Power BI and optimizing data pipelines for performance and cost efficiency.

WORK HISTORY

Azure Data Engineer, 03/2023 - Current

Humana – Washington (Remote)

• Designed and optimized ETL pipelines using Azure Data Factory, boosting data processing efficiency by 40% and ensuring seamless data flow across systems.

• Developed real-time data streaming applications to process daily events using Azure Stream Analytics and Apache Spark on Databricks, significantly enhancing data ingestion speed and reliability.

• Orchestrated complex workflows with Apache Airflow, reducing manual intervention time and enhancing operational efficiency through 80% automation.

• Implemented scalable ETL solutions using containerization with Docker and AKS, cutting deployment time by 60% and enabling dynamic scaling to meet varying data loads.

• Built interactive and dynamic dashboards using Power BI, improving data accessibility and empowering stakeholders with actionable insights

Azure Data Engineer, 03/2020 - 07/2022

Access Healthcare – India

• Designed and optimized ETL pipelines using Azure Data Factory and built high-performance Spark applications with PySpark and Spark-SQL on Azure Databricks for efficient data transformations and scalability.

• Implemented real-time data processing solutions with Azure Stream Analytics, Event Hub, and Service Bus Queue, enabling fast data ingestion and smooth transformation workflows.

• Developed automated workflows using Azure Functions and Logic Apps, reducing manual tasks and improving operational efficiency.

• Improved Databricks cluster performance by optimizing resource usage and implementing cost-effective scaling, ensuring faster job execution and reduced costs.

• Automated CI/CD pipelines with Azure DevOps, streamlining code integration, testing, and deployment while organizing data storage with Azure SQL Database and Data Lake. Tech Skills

• ETL/Middleware Tools: Azure Data Factory, Azure Databricks, Apache Airflow, SSIS, Talend

• Big Data Technologies: Hadoop, Apache Spark

• Azure Services: Azure Kubernetes Service (AKS), Azure Data Factory, Azure Databricks, Azure SQL DW, ADLS Gen2, Azure Synapse Analytics, Azure Functions, Azure Logic Apps, Azure Purview, PolyBase

• Real-time Data Processing: Kafka, Azure Stream Analytics

• Containerization & Orchestration: Docker, Kubernetes

• Traditional Databases: MySQL, Oracle, SQL Server, MongoDB, Cassandra

• Datawarehouse: Redshift, Snowflake

• Programming Languages: Python, Pyspark, Java, MS-SQL

• Data Visualization: Power BI, Tableau

• CI/CD and DevOps Tools: Azure DevOps, GitHub

• Development Tools: TOAD, SQL Developer, SSMS, Visual Studio, Azure Data Studio

• Security & Compliance: Azure Key Vault, RBAC, Network Isolation

• Monitoring & Logging: Azure Monitor, Log Analytics, Application Insights Certifications:

• Azure Data Engineer Associate – DP203

• AWS Certified Data Engineer – Associate certification Education

Master’s in Data science

University of Maryland Baltimore County, Maryland Aug 2022 – Jun 2024 (CGPA- 3.44/4.00) Bachelor of Computer Science

Bharath Institute of Higher Education and Research – India Aug 2017 – Jun 2021(CGPA 8.69/10.00)

Contact this candidate