Data Engineer Azure

Shreya Andela

Mobile: +1-945-***-**** email: **************@*****.***

PROFESSIONAL SUMMARY

Azure Data Engineer with 4+ years delivering scalable, secure and high-performance data platforms in finance and healthcare. Expert in ETL/ELT pipelines, data governance, real-time streaming architectures, big data processing, cloud migration, data warehouse design and data modeling using Azure Data Factory, Databricks, Synapse Analytics, Snowflake, Cosmos DB and Kafka. Proven track record of cutting analytics delivery time by 40%, reducing cloud costs by 20% and achieving 100% audit compliance (HIPAA, PCI-DSS, SOX). Adept at automation, workflow automation, monitoring and governance to drive actionable insights and business growth.

TECHNICAL SKILLS

Cloud & Data Platforms: Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage, Snowflake, Cosmos DB, Azure Stream Analytics, Databricks

ETL, ELT & Data Engineering: SSIS, PolyBase, Delta Lake, Data Lakes, Data Marts, Data Warehouse, Star/Snowflake Schema, Data Modeling, HL7, FHIR, EHR Integration, Batch/Real-Time Processing, Talend

Big Data & Streaming: Apache Kafka, Spark (PySpark, Structured Streaming), Scala

Programming & Scripting: Python, SQL, Scala, T-SQL, PowerShell, Azure CLI

DevOps & Automation: Azure DevOps, Jenkins, CI/CD, Airflow, Terraform, ARM Templates, Logic Apps, ADF Triggers, Automation, Git workflows and CI/CD pipeline management

Databases: SQL Server, Oracle, Teradata, Tableau

Security & Compliance: RBAC, Key Vault, Encryption-at-rest/in-transit, HIPAA, PCI-DSS, SOX

Monitoring & BI: Azure Monitor, Log Analytics, Power BI

Data Processes: Data mapping, cleansing, and editing; Extract, Transform, and Load (ETL) using Python and Visual Studio; Data pipelines, Data automation, Data integration, KPIs, Reports and dashboards, Data analytics and visualization

PROFESSIONAL EXPERIENCE

Client: Citi Group, Dallas, TX

Role: Azure Data Engineer

Duration: Aug 2024 – Present

Spearheaded the design and deployment of enterprise-scale Azure data pipelines, integrating ADF, Databricks (PySpark, Scala) and Azure Stream Analytics to process billions of financial transactions daily, enabling real-time fraud detection and reducing compliance reporting delays by 35%.

Built and maintained a Synapse-based data warehouse leveraging PolyBase + ADLS to handle multi-terabyte datasets, improving query performance by 45% for over 200 analysts and business users.

Engineered Delta Lake and Cosmos DB architectures to store and query ACID-compliant, low-latency datasets supporting mission-critical financial risk models.

Automated infrastructure provisioning and deployment using Terraform & ARM templates, reducing environment setup time from 3 days to 4 hours and improving deployment consistency across environments.

Implemented Airflow and Git workflows for CI/CD pipeline management, streamlining deployment and orchestration of complex data workflows.

Performed data mapping, cleansing, and editing for high-volume transactional data to improve accuracy of downstream analytics.

Developed ETL workflows using Python and Visual Studio for transforming and validating incoming datasets from multiple financial systems.

Integrated Talend for specific transformation and data quality processes to enhance automation and ensure compliance with banking regulations.

Designed and published interactive Power BI and Tableau dashboards connected to Synapse, delivering data analytics and visualization capabilities for executives, improving decision-making speed by 30%.

Defined and monitored KPIs for fraud detection efficiency, data processing time, and SLA adherence, driving operational improvements.

Migrated 50TB+ of critical on-prem Oracle/Teradata datasets to Azure, executing a secure cloud migration strategy with zero downtime during cutover.

Implemented workflow automation with ADF triggers (Event, Schedule, Tumbling) and Logic Apps to support time-sensitive financial reporting deadlines.

Built enterprise-grade data integration and automation workflows, reducing manual intervention by 80% in nightly data loads.

Led big data ingestion initiatives, optimizing ingestion, transformation, and monitoring for 50TB+ financial datasets, ensuring optimal pipeline health and SLA adherence.

Collaborated with compliance officers, data architects, and cross-functional engineering teams to prepare audit-ready technical documentation meeting data governance and regulatory standards.

Client: Unisys, India

Role: Data Engineer

Duration: July 2020 – June 2023

Developed complex ETL and ELT workflows in SSIS, ADF, Snowflake, and Talend to integrate multi-source healthcare datasets, including clinical records, insurance claims and patient demographics for analytics and operational reporting.

Created ETL scripts in Python and Visual Studio to process and validate incoming datasets, ensuring compatibility with healthcare industry standards.

Applied data mapping, cleansing, and editing techniques to ensure integrity and compliance with HIPAA and clinical data quality rules.

Optimized SQL queries, stored procedures and transformation logic to reduce pipeline runtime by 40%, enabling faster population health analytics for hospitals and insurers.

Led the migration of legacy ETL processes from on-prem SQL Server to a cloud-based ADF + Snowflake architecture, enhancing scalability, interoperability and HIPAA compliance.

Built real-time ingestion pipelines using Kafka + Spark Structured Streaming and Scala to process HL7, FHIR and EHR events, reducing ingestion latency to <5 seconds and improving care response times.

Implemented Airflow for workflow orchestration to manage multi-source healthcare data processing pipelines.

Automated infrastructure provisioning using Terraform to standardize environment creation and deployment for healthcare analytics systems.

Designed and implemented secure healthcare data lakes, marts and enterprise data warehouses, providing centralized access to petabytes of patient and operational data for predictive analytics initiatives.

Applied RBAC, encryption-at-rest/in-transit and Azure Key Vault to enforce security protocols, passing all HIPAA and compliance audits with zero findings.

Created monitoring and alerting solutions in Azure Monitor & Log Analytics to track pipeline health, detect failures early and maintain SLA adherence for critical clinical reports.

Created reports, dashboards, and KPIs in Tableau and Power BI for clinical and operational analytics.

Applied Git workflows and CI/CD pipeline management best practices to ensure consistent, controlled releases of data engineering solutions.

Collaborated with data analysts, clinicians, and compliance officers to translate regulatory and clinical requirements into robust, scalable data engineering solutions.

EDUCATION

University of North Texas, USA

Master of Sciences (MS) in Computer Science

Puducherry Technological University, India

Bachelor of Technology (B. Tech), in Computer Science and Engineering

CERTIFICATIONS

• Microsoft Certified: Azure Data Engineer Associate (DP-203)

• SnowPro Core Certification – Snowflake

• Databricks Certified Data Engineer Associate

Contact this candidate