Azure Data Engineer - ETL/ELT, Databricks, ADF, Delta Lake, Power BI

PROFESSIONAL SUMMARY

Motivated and technically skilled Azure Data Engineer with nearly 4 years of experience in delivering scalable, secure, and high-performance ETL/ELT solutions across diverse cloud and hybrid environments. Strong background in Databricks, Azure Data Factory, Delta Lake, Synapse, Fabric, and Power BI, with deep expertise in data modeling, Python-based automation, and metadata-driven architecture. Proven record of building and optimizing large-scale data pipelines, ensuring data governance (IAM/RBAC), and driving measurable improvements in data quality and delivery speed. Known for translating complex business requirements into actionable data strategies, collaborating cross-functionally with stakeholders, and leading automation-focused initiatives.

Experienced BI and Data Engineer with 4 years of expertise in designing and implementing data warehousing, data engineering, BI and Data Analytical solutions.

Skilled in developing ETL pipelines using ADF, Databricks, Synapse, SSIS, and Alteryx ensuring efficient data integration, transformation, and seamless data flow across platforms.

Proficient in data visualization and reporting using Power BI, Tableau, SSRS, and Looker, enabling stakeholders to interpret complex datasets and make data-driven decisions.

Strong analytical and technical skills in SQL, Python, and DBT to facilitate data modeling, transformation, and infrastructure as code, driving automation and optimizing data processes.

Demonstrated expertise in data engineering and analytics, translating raw data into actionable insights to drive strategic business outcomes and growth through effective data storytelling.

Proven track record in collaborative project environments with Git, DevOps practices and agile methodologies, using JIRA for task management and achieving project milestones. TECHNICAL SKILLS

Cloud Platforms: Azure, AWS

Data Engineering Tools: Azure Data Factory, Databricks, Synapse, Airflow, SSIS, Delta Lake

Databases: Azure SQL DB, Cosmos DB, Redshift, SAP BW

Languages: Python, SQL, PySpark

Reporting & BI: Power BI, Tableau, SSRS, SSAS, SAP BO

Orchestration & Automation: Azure DevOps, Git, Alteryx

Data Governance: Unity Catalog, Azure Purview, IAM, RBAC PROFESSIONAL EXPERIENCE

Client: TechnoServe, May 2024 to Aug 2025

Azure BI & Data Engineer

Collaborated with product owners and analysts to gather end-user requirements and KPIs, enhancing delivery accuracy and reducing post-deployment fixes by 35% through proper documentation, data mapping, detailed requirement walkthroughs, and consistent Agile sprint ceremonies and feedback sessions.

Designed and deployed a greenfield data engineering project using Databricks, Unity Catalog, Microsoft Fabric, and Azure Synapse, creating a robust lakehouse architecture with audit trails, schema evolution support, and scalable architecture for future enhancements and seamless cross-domain data integrations.

Employed PySpark in Databricks for parallel ETL processing on ADLS-based datasets, reducing pipeline execution time by over 60% while maintaining data integrity, accuracy, and seamless cloud integration.

Automated data validations, regression testing, and alerts using Python and ADF, significantly improving pipeline reliability, reducing manual QA efforts by 70%, and accelerating data delivery timelines consistently.

Engineered metadata-driven pipelines using Delta Lake, Azure Purview, and Python, supporting dynamic data ingestion, auditing, lineage capture, and documentation across multiple domains for consistent data governance, ensuring data accuracy, regulatory compliance, and overall operational efficiency.

Delivered Power BI dashboards linked to Delta Lake layers, allowing business teams to track customer lifecycle metrics, conversion rates, and real-time visuals with a 40% increase in user adoption and engagement through enhanced interactivity and intuitive design features. Vamshi Krishna

Data Analyst & Engineer

EMAIL ID: *****************@*****.*** PHONE NUMBER: +447*********

Architected a unified data lake and warehouse model with ADLS and Synapse, balancing real-time and historical data access, improving query performance by 50%, and reducing latency across workloads.

Implemented fine-grained access control policies using IAM and Unity Catalog, securing 100+ assets while ensuring business teams had role-appropriate access and centralized governance and auditing.

Developed Python scripts to manage Fabric job triggers, monitoring, and failure recovery, enhancing operational reliability, error traceability, and robust audit logging for compliance needs.

Integrated CRM/Dataverse data into the Azure ecosystem, enriching customer segmentation efforts, improving marketing insights turnaround by 3x, and enabling highly targeted campaign reporting.

Developed DevOps pipelines for deploying ADF, notebooks, and security configurations, minimizing manual tasks, improving CI/CD workflows, and achieving 95% consistency across deployment environments.

Documented and conducted internal training on metadata design, pipeline templating, and RBAC configuration, reducing ramp-up time for new team members by 50% and boosting cross-team efficiency. Tech Stack: Azure Data Factory, Databricks, Synapse, Fabric, ADLS, Delta Lake, Dataverse, Azure SQL DB, Unity Catalog, CRM, Python, PySpark, Power BI, DevOps.

Client: National Bank of Australia / Accenture, Nov 2021 to Mar 2023 Data Engineer

Participated in stakeholder interviews to define cross-functional reporting needs across CRM and marketing domains, ensuring requirement clarity and a 25% reduction in revisions, which improved collaboration and minimized project delays significantly through effective communication and documentation.

Built a comprehensive data platform from scratch using Databricks, Synapse, Unity Catalog, and ADF, improving data accessibility and real-time KPI availability, accelerating data-driven decision-making across multiple business units, and enabling scalable analytics solutions for future growth.

Developed PySpark pipelines to transform high-volume Cosmos DB and CRM data, reducing data lag from 6 hours to 45 minutes for daily refresh, enabling near real-time analytics for marketing and sales teams.

Created Python-based utilities for test automation and orchestration, achieving 80%-unit test coverage and early detection of data quality issues, significantly decreasing production defects and improving pipeline reliability through automated validation and monitoring processes.

Designed a metadata-based ingestion framework that uses Delta Lake, Purview, and Python for ingestion configuration, schema validation, and exception handling, enhancing scalability and maintainability of enterprise-wide data pipelines with improved monitoring and dynamic error handling.

Built centralized Power BI dashboards on top of curated Delta tables for business KPIs and financial forecasting, decreasing manual reporting time by 60%, and providing executives with actionable, timely insights that enhanced strategic planning and operational efficiency significantly.

Migrated batch pipelines to Airflow-managed DAGs, improving observability and reducing missed SLAs with alerting and retry mechanisms, increasing overall pipeline robustness and fault tolerance.

Designed a dual-zone Delta Lake architecture with staging and curated layers, enabling rollback and version tracking while lowering query time by 30%, improving data freshness and audit capabilities.

Enforced data governance protocols by implementing RBAC roles in Unity Catalog, limiting access by business unit and sensitivity level, ensuring compliance with data privacy regulations across multiple regions.

Delivered onboarding documentation, version-controlled notebooks, and reusable YAML-based pipeline templates to reduce development time across future projects, fostering best practices, improving collaboration, and accelerating new team member productivity significantly. Tech Stack: Azure ADF, Databricks, Synapse, CosmosDB, ADLS, Delta Lake, Azure SQL DB, Unity Catalog, Python, PySpark, Power BI, DevOps, Airflow, S3, RedShift, CRM Client: Microsoft / Accenture, July 2020 to Oct 2021 MSBI Developer

Engaged with functional consultants and business users to gather complex reporting requirements, reducing dependency on legacy BusinessObjects (BO) reports and increasing satisfaction scores by 30%, significantly improving cross-team collaboration and user adoption.

Implemented a Databricks + Unity Catalog transformation layer, replacing SAP-centric processing with modular, traceable PySpark logic, enhancing maintainability and scalability of data pipelines.

Rewrote legacy ETL logic from SSIS into PySpark notebooks, achieving an average 65% performance improvement on multi-million-row datasets, enabling faster data refreshes and improved SLA adherence.

Developed Python test suites integrated into CI pipelines, improving regression coverage and minimizing post-deployment bugs in data flows, reducing manual testing efforts and accelerating release cycles.

Designed a metadata ingestion framework supporting SAP BW, SSAS, and Azure SQL sources, accelerating onboarding of new data sources by 50%, and standardizing data ingestion and validation processes.

Created executive-ready Power BI dashboards for sales, finance, and operations, leveraging SAP data and reducing time to insight by 40%, empowering leadership with timely, actionable analytics.

Set up Delta Lake zones (raw, bronze, silver, gold) with validation and quality scoring layers, improving trust in downstream reporting datasets and facilitating compliance with data governance policies.

Led the design of a hybrid data lake and warehouse model with SSAS + Synapse on top of ADLS-backed Delta tables, optimizing query performance and supporting both real-time and historical analytics.

Configured IAM and RBAC policies in Unity Catalog and Synapse, enforcing data masking and audit logging to align with ISO standards, strengthening data security and compliance posture.

Delivered weekly notebooks and KPI metrics to leadership with integrated Git version control and audit tagging, improving transparency, traceability, and collaborative decision-making processes.

Integrated SAP BO exports into Delta Lake workflows, enabling dashboard harmonization across Power BI and Tableau for regional teams, facilitating consistent reporting and better business insights.

Created deployment pipelines in DevOps to automate ARM deployments and parameterized Data Factory integration, reducing manual intervention and accelerating delivery cadence. Tech Stack: Azure Data Factory, Databricks, SSIS, SSAS, SAP BW, SAP BO, ADLS, Delta Lake, Azure SQL DB, Python, PySpark, SQL, Power BI, Git

EDUCATION

Bachelor of Technology (B.Tech) – Computer Science Engineering JNTUH Graduation:2022

MSc (Masters) Management with Data Analytics and Professional Development and Planning BPP UNIVERSITY Graduation: 2024

CERTIFICATION

AZ-900: Microsoft Azure Fundamentals

DP-203: Data Engineering on Microsoft Azure

PL-300: Microsoft Power BI Data Analyst

Contact this candidate