Data Engineer Azure
Resume:
Uma
Azure Data Engineer
Email: *********@*****.***
Professional Summary
Data Engineer with 5+years of experience in data engineering, database development, and cloud-based data warehousing solutions. Proven expertise in SQL Server development, data migration, and ETL pipeline orchestration using Azure Data Factory, Databricks, and Snowflake. Experienced in writing, optimizing, and refactoring complex stored procedures to ensure performance, scalability, and data integrity across large datasets.
Adept at analyzing and tuning SQL queries for efficiency and supporting critical database migration initiatives.
Strong background in validating data pipelines post-migration to ensure accuracy and compliance with business requirements.
Highly skilled in collaborating with architects, developers, and QA teams to deliver robust solutions within Agile environments.
I am proficient at data visualization using Power BI and strong in application development.
Good Experience on the Implementation of row-level security by defining various constraints for each defined ROLE.
Experts create different visualizations using Slicers, Lines, Pies, Histograms, Maps, Scatter, Bullets, Heat Maps, Tree maps, etc.
Involved in troubleshooting of performance issues associated with Power BI reports.
maintaining and supporting operational reports, dashboards, and scorecards using Microsoft Power BI
Having good Experience in creating dashboards, volume reports, operating summaries, presentations, and graphs.
Good Experience on writing DAX in MS Power BI Desktop.
Experience working with Azure Logic APP Integration tool.
Experience working with Data warehouses like Teradata, Oracle, SAP
Strong experience in Design, Dev
development, Data Migration, Testing, Support and Maintenance using big data technologies like HDFS, MapReduce, Hive, Python, Sqoop, Airflow, HBase, Spark and Python.
Implemented multiple big data projects on cloud using AWS components like S3, DynamoDB, Glue, Athena, Data Pipeline, EMR, EC2, Lambda, CloudWatch and Redshift.
Experience in analyzing data using HiveQL and MapReduce Programs.
Experienced in ingesting data into HDFS from various Relational databases like MYSQL, Oracle, DB2, Teradata, and Postgres using Sqoop
Experienced importing real-time streaming logs and aggregating the data to HDFS using Kafka and Flume.
Experience on the Implementation of Azure log analytics, providing Platform as a service for SD-WAN firewall logs.
Experience in building the data pipeline by leveraging the Azure Data Factory.
Selecting appropriate low-cost, driven AWS/Azure services to design and deploy an application based on given requirements.
Expertise working with databases like Azure SQL DB, Azure SQL DW
Solid programming experience on working with Python and Scala.
Experience working in a cross-functional AGILE Scrum team.
Technical Skills:
Languages: T-SQL, PL/pgSQL, Python, Scala, Shell Scripting, DAX
Big Data & Cloud Technologies: Hadoop (HDFS, Hive, Sqoop, Flume, Spark, MapReduce), Azure Data Factory, Databricks, Snowflake, AWS (S3, Glue, Athena, EMR, Lambda, CloudWatch), Azure (SQL DB, SQL DW, Logic Apps)
ETL & Data Pipelines: Azure Data Factory, Apache Airflow, AWS Data Pipeline, Sqoop
Databases: SQL Server, PostgreSQL, Sybase, Oracle, MySQL, Teradata, SAP
BI & Visualization Tools: Power BI, Tableau
DevOps & CI/CD: Git, GitHub, Jenkins, Azure DevOps
Performance & Monitoring: Query Store, SSDT, SQL Agent, Execution Plans, SET STATISTICS IO, Column store
Testing & QA: Automated SQL Testing, Data Validation Scripts, Error Logging Frameworks
Security: Row-Level Security, Transaction Safety, Exception Handling
Tools & IDEs: SSMS, pgAdmin, Confluence, JIRA, Visual Studio Code
Methodologies: Agile (Scrum)
Professional Experience
AT&T Dallas, Texas Jun 2024- Till Date
Azure Data Engineer
Designed and managed scalable ETL pipelines utilizing Azure Data Factory.
Automated data ingestion and transformation processes using Azure Databricks interactive and job clusters.
Applied data lake partitioning strategies to enhance performance and minimize storage expenses.
Developed reusable Azure Data Factory pipeline templates with parameterized settings.
Transitioned on-premises ETL workflows to cloud-native pipelines combining Azure Data Factory and Databricks.
Rewrote legacy Sybase and SQL Server procedures into optimized PostgreSQL modules.
Optimized SQL queries through indexing, execution plan analysis, and profiling tools.
Created audit logging and error-handling frameworks to ensure compliance.
Merged multiple procedures into modular, parameter-driven designs.
Standardized schema-qualified procedures and enforced consistent naming conventions.
Implemented row-level security and role-based access controls within SQL databases.
Developed automated data quality validation checks with alerting mechanisms.
Integrated data lineage and metadata management using Microsoft Purview.
Utilized Spark on Databricks to process datasets spanning multiple terabytes.
Built real-time streaming data pipelines with Kafka and Flume on Azure.
Migrated batch jobs from Hadoop/HDFS to Azure cloud-native platforms.
Incorporated Azure Data Factory and Databricks into CI/CD pipelines using Azure DevOps.
Delivered data solutions following Agile and Scrum sprint methodologies.
Created architecture diagrams and maintained pipeline documentation in Confluence.
Mentored junior engineers on stored procedures, orchestration techniques, and Git version control.
Tools: Azure Data Factory, Azure Databricks (Spark, Python, SQL), Apache Kafka, Apache Flume, Hadoop/HDFS, PostgreSQL, SQL Server, Sybase, Git, Confluence, Agile/Scrum
Charter Communications, Washington, DC Aug 2021- May 2024
Azure Data Engineer
RESPONSIBILITIES:
Developed internal audit logs within stored procedures to trace data anomalies.
Rewrote Sybase stored procedures using PostgreSQL’s PL/pgSQL syntax and standards.
Added exception handling blocks and transaction safety to legacy stored procedures.
Applied indexing strategies to support stored procedure query efficiency.
Created procedure-level profiling scripts to identify bottlenecks during batch runs.
Validated procedure outputs using controlled data sets and diff-check utilities.
Applied dependency analysis to remove unused or orphaned stored procedures.
Tuned procedures using SET STATISTICS IO and execution plans for SQL Server.
Analyzed object-level migration gaps using tools like SQL Server Data Tools (SSDT).
Documented migration playbooks for stored procedures and ETL routines.
Evaluated legacy Sybase business logic for modernization opportunities in PostgreSQL.
Performed dry-run test cycles on new PostgreSQL environments post-migration.
Developed shell/Python scripts to automate batch validations across environments.
Managed SQL compatibility issues using conditional logic within stored procedures.
Enforced use of schema-qualified object names to avoid ambiguity.
Consolidated multiple small procedures into parameter-driven single modules.
Removed hardcoded literals and replaced them with configuration-driven parameters.
Applied SAR Gable filter conditions to enhance procedure scan efficiency.
Reviewed procedure code against internal SQL coding standards checklist.
Monitored wait statistics and I/O usage to isolate bottlenecks in stored procedures.
Integrated caching layers for lookup-heavy procedures to reduce round-trip.
Applied throttling strategies for procedures hitting concurrency limits.
Reduced memory usage by rewriting inefficient string manipulations in procedures.
Maintained Jira stories for every major procedure refactor or performance fix.
Provided mentorship on version control strategies for stored procedure changes.
Worked with DevOps to integrate procedure deployments into CI/CD pipelines.
Participated in war-room sessions to troubleshoot post-migration procedure issues.
Created visual architecture diagrams mapping dependencies across stored procedures and tables.
Implemented row-level security using stored procedures and user roles.
Built audit tables to capture before/after state changes from stored procedures.
Ran query simulations across dev, QA, and prod to validate procedure consistency.
Tools: Sybase, PostgreSQL, SQL Server, PL/pgSQL, T-SQL, Python, Shell Scripting, SQL Server Data Tools SSDT, Execution Plans, SET STATISTICS IO, Query Profiling, CI/CD Jenkins/Git, Jira, DevOps Tools, Row-Level Security,
Data Engineer
Teleperformance, India Jan 2020 - July 2021
RESPONSIBILITIES:
Created checksum comparison reports for source and target datasets to verify row-level fidelity.
Conducted dry-run validations of ETL pipelines post-procedure conversion.
Designed reference mapping tables to support logic transformation across database platforms.
Implemented multi-threaded scripts for validating high-volume procedure outputs.
Coordinated UAT sign-off for all migrated stored procedures.
Created automated test scripts to validate procedure logic across multiple schemas.
Standardized error logging patterns within procedures for centralized monitoring.
Validated numeric precision and datatype consistency post-migration for critical procedures.
Migrated logic-heavy reports into stored procedures to centralize business rules.
Designed rollback logic in procedures for transaction-safe batch updates.
Participated in change advisory board (CAB) reviews for stored procedure deployments.
Created Confluence pages summarizing stored procedure changes and rationale.
Presented post-migration stored procedure performance metrics to stakeholders.
Designed a partitioning strategy for data lakes to optimize cost and performance.
Built reusable ADF pipeline templates for standardized ingestion and transformation.
Implemented data quality checks with alerts for nulls, duplicates, and schema drift.
Provided root cause analysis for pipeline failures and proposed long-term fixes.
Conducted side-by-side performance comparisons between Sybase and PostgreSQL versions of stored procedures.
Implemented procedure-level retry logic for transient database connection issues.
Used partitioned tables to speed up large table access inside procedures.
Scheduled off-peak execution of resource-heavy procedures to optimize resource usage.
Tuned procedures to minimize tempDB spills using column store indexes and batch mode.
Migrated dynamic SQL code to parameterized procedures to enhance security and maintainability.
Refactored procedures to use temp tables instead of table variables for better scalability.
Rewrote legacy error-prone procedures with transaction-safe control blocks.
Built helper functions to reduce code redundancy in frequently used logic blocks.
Replaced legacy RAISERROR calls with standardized exception logging procedures.
Tools: SQL Server, PostgreSQL, Sybase, T-SQL, PL/pgSQL, Python, Azure Data Factory, Git, Jenkins, Confluence, Query Store, Partitioning, Error Logging, Performance Tuning
EDUCATION:
Bachelor of Computer Science Engineering
St. Martin's Engineering College, India.
Masters in Computer Science
University Of Dayton, OH.
Contact this candidate