Lead, Sr. Data Engineer, Azure Data Engineer, DP-203 Certified

MEGHANA G. SHARMA

Lead, Sr. Data Engineer Azure Data Engineer DP-203 Certified

913-***-**** ~ Open for anywhere in US ~ CTC Only

Summary of Qualifications:

oResults-driven IT Professional with 12+ years of progressive experience in designing, building, and supporting enterprise-scale data platforms across telecommunications, financial services, retail banking, and government domains.

oStrong expertise in Azure cloud–native data engineering, including end-to-end pipeline orchestration using Azure Data Factory, distributed data processing with Azure Databricks (PySpark), and lakehouse implementations leveraging Azure Data Lake Storage and Delta Lake.

oProven ability to modernize legacy data platforms by migrating on-premises ETL workflows to scalable, secure, and high-performance Azure architectures, improving reliability, maintainability, and analytics readiness.

oExtensive experience in data modeling and transformation, including dimensional modeling (fact/dimension tables), Slowly Changing Dimensions (SCD Type 1 & 2), and implementation of data quality, validation, and reconciliation frameworks.

oStrong background in performance optimization, with hands-on experience tuning Spark jobs, SQL queries, indexing strategies, and batch workflows to support high-volume data processing and reporting workloads.

oDemonstrated success implementing CI/CD and DevOps practices using Azure DevOps, Git, and automation frameworks to enable controlled, repeatable deployments across Development, QA, and Production environments.

oDeep foundation in enterprise database engineering, including Oracle and Azure SQL platforms, with expertise in schema design, stored procedures, batch processing, and transactional data systems supporting mission-critical applications.

oExperience delivering solutions in highly regulated environments, including banking and government systems, with strong emphasis on data accuracy, auditability, security, and compliance.

oProven technical leadership and collaboration skills, having led sprint planning, code reviews, mentoring, and cross-functional coordination with architects, DBAs, QA teams, and business stakeholders in Agile delivery models.

oAdept at translating complex business requirements into scalable, production-ready data solutions, supporting analytics, reporting, and data-driven decision making across the enterprise.

Awards & Honors, Certifications & Education:

oInfosys Best Employee of the Month – July 2016

oInfosys Award of ‘Rewards and Recognition’ in Green Horn Category

oInfosys INSTA Award

oSpot Award at EVRY

oCognizant P&R Awards in Bronze Category

oAZ-303 Microsoft Azure Architect Technologies

oAZ-304 Microsoft Azure Architect Design

oDP-203 Microsoft Azure Data Engineer

oBachelor’s Degree in Information Science (Graduated in 2012)

Siddhartha Institute of Technology (Affiliated to VTU) - Karnataka, India

Professional Experience:

Client: Capital One – Dallas, TX Mar 2025 – Present

Role: Sr. Data Engineer, Azure Engineer

This project focused on building and enhancing a secure, scalable Azure-based data engineering platform to support enterprise analytics, reporting, and data-driven decision-making initiatives. The scope involved ingesting, transforming, and curating large volumes of structured financial data from multiple enterprise systems into cloud-native analytical data stores. The project emphasized modern data engineering practices, including cloud-native orchestration, distributed data processing, performance optimization, and data quality enforcement using Azure Data Factory, Azure Databricks, and Azure Data Lake Storage. The engagement supported analytics use cases such as customer segmentation, pricing analysis, and operational reporting, while ensuring secure, reliable, and production-ready data pipelines aligned with Capital One’s enterprise data standards.

oDesigned and developed end-to-end data ingestion and transformation pipelines on Azure, supporting analytics and reporting workloads across multiple business domains.

oBuilt and orchestrated Azure Data Factory (ADF v2) pipelines to ingest data from enterprise source systems into Azure Data Lake Storage (ADLS Gen2) and downstream analytical layers.

oImplemented distributed data transformations using Azure Databricks (PySpark), processing high-volume datasets with optimized Spark jobs for performance and scalability.

oApplied data validation, reconciliation, and quality checks using Python and SQL to ensure data accuracy, consistency, and reliability across pipelines.

oMigrated and optimized legacy data processing workflows into cloud-native Azure data engineering solutions, improving maintainability and reducing pipeline execution times.

oDeveloped and optimized dimensional data models (fact and dimension tables) to support analytical queries and BI consumption.

oImplemented incremental data processing and change handling patterns to efficiently manage data refreshes and reduce processing overhead.

oTuned data pipelines and Spark jobs using partitioning strategies, caching, and performance optimization techniques to improve throughput and execution efficiency.

oLeveraged SQL and Python to perform complex transformations, joins, and aggregations across large datasets stored in Azure data platforms.

oImplemented version control and release management practices using Git and Azure DevOps, supporting controlled deployments and collaborative development.

oCollaborated with business analysts, data consumers, and platform teams to gather requirements, prioritize workloads, and deliver data solutions aligned with business needs.

oActively participated in Agile delivery processes, including sprint planning, backlog refinement, and release coordination.

Environment: Azure Data Factory (ADF v2), Azure Databricks (PySpark), Azure Data Lake Storage Gen2, Azure SQL Database, SQL, Python, Azure DevOps, Git, Power BI, Agile/Scrum

Employer: Cognizant – Bengaluru, India Feb 2022 – Feb 2024

Role: Team Lead Data Engineer, Azure Engineer

Project: Merch Insight

The Merch Insight project focused on building a centralized data platform to support merchandise planning, ordering, and inventory optimization across multiple regions, seasons, and store clusters. The platform enabled business planners to create and manage long-term merchandise plans by leveraging historical sales data, seasonal trends, and store-level attributes. A key aspect of the solution was store clustering, where stores were grouped based on geography, sales behavior, and demand patterns to improve allocation efficiency and reduce inventory risks. The project involved designing and implementing scalable Azure-based data pipelines to ingest, transform, and consolidate data from multiple operational systems into an analytical data store, supporting planning workflows, approval processes, and reporting needs. The solution provided improved visibility into inventory, demand forecasts, and approval statuses, enabling stakeholders to make informed, data-driven decisions and improve overall merchandise planning and execution.

oLed requirements gathering and sprint planning activities, collaborating with business stakeholders to translate merchandise planning and inventory requirements into technical deliverables.

oManaged task decomposition, prioritization, and estimation of user stories, ensuring on-time delivery across multiple sprints in an Agile environment.

oContributed to the design of Azure-based data architecture, supporting scalable ingestion, transformation, and storage of enterprise data.

oDesigned, developed, and maintained data integration pipelines using Azure Data Factory, enabling reliable extraction and transformation of data from multiple source systems.

oImplemented data transformation and processing logic using Azure Synapse Analytics and PySpark notebooks, supporting analytical and reporting workloads.

oIntegrated disparate data sources into a centralized Azure data warehouse, enabling consistent and trusted datasets for downstream consumption.

oDeveloped and optimized solutions using Azure SQL Database, including performance tuning, indexing strategies, and query optimization to support high-volume transactional and analytical workloads.

oCreated and maintained stored procedures and database objects, along with unit test cases, to ensure data accuracy, performance, and reliability.

oImplemented Azure Logic Apps to automate data workflows, notifications, and process triggers, improving operational efficiency.

oPerformed query performance tuning for application-generated SQL, working closely with application (.NET) teams to resolve performance bottlenecks.

oLed code reviews and technical mentoring for team members, providing guidance on best practices, performance optimization, and issue resolution.

oManaged and configured key Azure resources, including Azure Storage, Azure Data Lake Storage, Azure Key Vault, and access controls to ensure secure data handling.

oCollaborated with cross-functional teams across development, QA, and business units to support releases, issue resolution, and continuous improvements.

Environment: Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, Azure SQL Database, Azure Logic Apps, Azure Key Vault, SQL Server, T-SQL, PySpark, Python, Jupyter Notebook, Azure Virtual Machines, Azure Data Studio, SSMS, TFS, Jenkins, Docker, Visual Studio, Agile/Scrum

Client: Oslo Banks – Norway, Oslo (Onsite) Dec 2017 – Feb 2022

Employer: EVRY India Pvt. Ltd – Bengaluru, India

Role: Lead Data Engineer

The Retail Banking Services project for Oslo-based banks focused on building and enhancing a centralized data and transaction platform to support core retail banking operations in compliance with regional regulatory standards. The platform enabled secure management of customer accounts, retail banking products, and high-volume financial transactions across multiple channels. Key functional areas included account lifecycle management, transaction processing, interest calculation, statement generation, and loan and credit monitoring. The project emphasized data accuracy, performance, and reliability, with strong controls around validation, auditability, and risk management. As part of the engagement, the team designed and optimized enterprise-grade database solutions and supported early-stage cloud adoption initiatives, ensuring scalable, secure, and compliant data handling for mission-critical banking workloads.

oServed as Lead Data Engineer, overseeing database design, development, and performance optimization for core retail banking data platforms.

oCollaborated closely with clients, product owners, and business stakeholders to understand regulatory, functional, and reporting requirements, translating them into optimized data and database designs.

oDesigned and implemented conceptual, logical, and physical data models to support customer accounts, transactions, loans, and retail banking products.

oDeveloped and optimized database packages, stored procedures, and functions to support transaction processing, interest calculations, statement generation, and batch workloads.

oImplemented database indexing and partitioning strategies to improve query performance and ensure scalability for high-volume transactional data.

oPerformed query performance tuning on SQL generated by Java-based applications (Hibernate / jOOQ), resolving bottlenecks and improving response times.

oBuilt and executed unit test cases for database objects to ensure correctness, reliability, and regression safety.

oSupported batch processing workflows for periodic account updates, interest computations, and reporting processes.

oConducted code reviews and provided technical guidance to team members, ensuring adherence to best practices in database development and performance optimization.

oLed knowledge transfer and mentoring sessions for team members from diverse technical backgrounds, improving overall team capability and delivery quality.

oUtilized CI/CD and version control tools to manage database changes, deployments, and releases in a controlled and auditable manner.

oContributed to the setup and management of Azure resources in support of application hosting and supporting services, aligning with early cloud adoption initiatives during the project timeline.

Environment: Oracle Database 12c, Oracle SQL Developer, SQL/PLSQL, Database Design & Modeling, Indexing & Partitioning, Flyway, Jenkins, Docker, Azure (basic resource usage), JIRA, Visual Studio, Tortoise SVN

Client: Income Tax Department - Govt. of India Jan 2015 – Dec 2017

Employer: Infosys – Bengaluru, India

Role: Sr. Database Engineer

The Income Tax Returns (ITR) Processing System was a large-scale, mission-critical tax administration platform developed for the Government of India to automate the end-to-end processing of electronically filed income tax returns across assessment years. The system handled high-volume taxpayer data and executed complex, rule-driven validations in accordance with the Income Tax Act and annual Finance Act amendments. The scope included initial data validations, tax re-computation, interest and penalty calculations, refund or demand determination, and generation of statutory intimations and notices. The platform also supported grievance handling, rectification workflows, audit trails, and SLA-driven issue resolution, ensuring secure, accurate, and transparent tax processing during peak filing periods with strict compliance and availability requirements.

oAnalyzed business requirements, statutory rules, and technical specifications to design and implement database solutions aligned with government tax regulations.

oDesigned, developed, and maintained Oracle database schemas and objects, including tables, views, materialized views, indexes, constraints, triggers, sequences, and synonyms to support large-scale tax processing workloads.

oDeveloped and optimized PL/SQL packages, procedures, and functions to implement rule-based validations, tax calculations, interest computation, refunds, and demand determination logic.

oImplemented implicit and explicit cursors, cursor loops, and reference cursors to efficiently process large datasets and complex transactional workflows.

oPerformed unit testing, debugging, and validation of database code to ensure accuracy, stability, and compliance with statutory requirements.

oSupported system integration testing (SIT) and assisted with deployment of database changes into production environments.

oOptimized database performance through indexing strategies, query tuning, and execution plan analysis, ensuring reliable performance during high-volume peak filing cycles.

oImplemented and managed Oracle Job Scheduler processes to support batch executions, periodic validations, and scheduled processing activities.

oConducted peer code reviews to enforce coding standards, performance best practices, and data integrity.

oCollaborated with business users and stakeholders to clarify requirements, assess impact of changes, and obtain approvals through detailed test results and documentation.

oProduced and maintained technical design documents and project artifacts, ensuring traceability of changes and adherence to project standards.

oSupported production management teams by investigating and resolving database-related incidents and data issues within defined SLAs.

oDelivered knowledge transfer sessions to new and existing team members, ensuring continuity and operational readiness.

Environment: Oracle Database, Oracle SQL, PL/SQL, Oracle SQL Developer, Oracle Job Scheduler, Database Design & Modeling, Indexing & Performance Tuning, Tortoise SVN, Notepad++, Enterprise Production Support

Client: Deutsche Bank – New York, NY June 2013 – Jan 2015

Employer: TCS – Bengaluru, India

Project: Eagle Pace

Role: Software Engineer

Eagle PACE is a centralized Reference Data Management (RDM) platform used by the Data Services group within Deutsche Asset Management to support trading, accounting, and fund management systems. The platform acted as a Golden Data Repository, ingesting securities reference data from multiple external vendor sources across different subject areas. Incoming data was validated, standardized, and enriched using predefined business rules to create a trusted golden copy of security master data. The system supported creation of new securities, ongoing data refreshes from vendor feeds, and outbound data distribution to downstream trading and accounting applications. The platform played a critical role in ensuring data accuracy, consistency, and timeliness across enterprise investment systems.

oContributed to the design and development of reference data processing modules supporting security master data ingestion, validation, and enrichment workflows.

oDeveloped and enhanced Oracle PL/SQL packages, procedures, and functions to implement business rules for data validation, transformation, and golden record creation.

oSupported ingestion of reference data from multiple vendor feeds, applying rule-based checks to ensure data accuracy, completeness, and consistency before downstream distribution.

oImplemented and optimized database objects including tables, indexes, views, and constraints to support high-volume reference data processing.

oProactively tuned stored procedures and SQL queries to improve system performance and reduce batch processing times.

oDesigned and maintained outbound data feeds supplying validated reference data to trading, accounting, and fund management systems.

oAssisted in new security creation workflows, enabling timely availability of reference data for downstream trading and accounting applications.

oWorked closely with client stakeholders and business users to understand data requirements, clarify rules, and support implementation and change requests.

oSupported batch scheduling and monitoring using Control-M to ensure timely execution of reference data loads and refresh processes.

oHandled multiple functional modules simultaneously, providing operational support and resolving data and processing issues.

oParticipated in code reviews and knowledge sharing, ensuring adherence to development standards and best practices.

Environment: Oracle Database, Oracle SQL, PL/SQL, Oracle SQL Developer, Control-M Job Scheduler, Database Design & Performance Tuning, Tortoise SVN, Notepad++, Enterprise Banking Systems

Employer: TCS – Bengaluru, India Dec 2012 – May 2013

Role: Software Engineer

Project: Hospital Management

The Hospital Management System project involved the development and enhancement of an enterprise application designed to manage core administrative and operational functions of a hospital. The application was structured into multiple functional modules, including building management, department management, ward and bed allocation, and billing. The system aimed to streamline hospital operations by centralizing data, improving resource utilization, and supporting day-to-day administrative workflows. As part of this early-career engagement, the role provided hands-on exposure to application development, database interactions, and testing activities within a structured enterprise delivery environment.

oContributed to the development of a new sub-module for Building Management, supporting configuration and management of hospital infrastructure data.

oAssisted in enhancing existing application modules to accommodate evolving business and functional requirements.

oDeveloped and maintained database components using Oracle SQL and PL/SQL to support application functionality and data persistence.

oParticipated in the preparation of business scenarios, test cases, and user manuals, supporting application validation and user adoption.

oGained hands-on experience in functional testing, user acceptance testing (UAT), and defect analysis, ensuring application quality and stability.

oCollaborated with senior developers and functional teams to understand requirements, troubleshoot issues, and support application enhancements.

Environment: Oracle Database, Oracle SQL, PL/SQL, Oracle SQL Developer, C++, Visual Studio, Functional & UAT Testing, Enterprise Application Development

Technical competencies:

Cloud Platforms & Data Services: Microsoft Azure, Azure Data Factory (ADF v2), Azure Databricks, Azure Data Lake Storage (ADLS Gen2), Azure Synapse Analytics, Azure SQL Database, Azure Blob Storage, Azure Key Vault, Azure Virtual Machines, Azure Logic Apps

Data Engineering & Big Data: End-to-End Data Pipeline Design, ELT / ETL Architecture, Distributed Data Processing, Batch Data Processing, Incremental Data Loads, Change Data Handling, Data Quality & Validation Frameworks, Lakehouse Architecture, Delta Lake (ACID Transactions, Upserts, Schema Enforcement), Dimensional Data Modeling (Fact & Dimension Tables), Slowly Changing Dimensions (SCD Type 1 & Type 2)

Programming & Query Languages: Python, PySpark, SQL, T-SQL, PL/SQL

Databases & Warehousing: Oracle Database (12c), Azure SQL Database, SQL Server, Snowflake (Data Loading & Query Optimization – where applicable), Database Schema Design, Indexing & Partitioning, Stored Procedures, Functions, Packages, Query Optimization & Performance Tuning

Orchestration & Automation: Azure Data Factory Pipelines & Triggers, Control-M Job Scheduler, Oracle Job Scheduler, Batch Processing Frameworks

CI/CD, DevOps & Version Control: Azure DevOps (CI/CD Pipelines), GIT, Jenkins, Flyway, Docker, TFS, Tortoise SVN

Monitoring, Security & Governance: Pipeline Monitoring & Logging, Failure Handling & Alerts, Secure Secrets Management (Azure Key Vault), Role-Based Access Control (RBAC), Data Validation & Audit Controls

Reporting & BI: Power BI, Analytical Query Optimization, BI Data Preparation & Consumption

Development Tools: Azure Data Studio, Oracle SQL Developer, SQL Server Management Studio (SSMS), Jupyter Notebook, Visual Studio, Notepad++

Methodologies & Collaboration: Agile / Scrum, Sprint Planning & Backlog Grooming, Requirement Analysis & Stakeholder Collaboration, Code Reviews & Technical Mentoring, Production Support & Incident Resolution

References: Provided upon request…

Contact this candidate