Data Engineer Quality

Sai Durga Rao Padamati

*************@*****.*** +1-839-***-**** United states

Summary:

Data Engineer with 3+ years of experience in designing, building, and optimizing scalable data pipelines and data warehouse solutions. Proficient in ETL development, data modelling, and big data ecosystems including Hadoop, Spark, and Kafka, with hands-on expertise in cloud platforms such as AWS and Azure. Skilled in SQL, Python, Hive, Databricks, and real-time streaming solutions, with a strong focus on performance optimization, data quality, and workflow automation. Adept at collaborating with cross-functional teams to deliver data-driven solutions that support analytics, reporting, and business intelligence. Technical skills:

Programming: Python, SQL, Scala

Cloud & Big data: AWS (S3, Redshift, EMR, Glue), Azure (Data Factory, Databricks, Synapse), Hadoop, Spark, Hive, Kafka

Databases: SQL Server, Oracle, MySQL, PostgreSQL

Data Warehousing: Snowflake, Redshift, Azure Synapse

ETL Tools: Informatica, Talend, Azure Data Factory

Visualization: Power BI, Tableau

Operating systems: Linux, Windows, UNIX

Development Methods: Agile/Scrum, Waterfall

Version Control: Git, GitHub

Other: Airflow, Docker, Kubernetes, Jenkins, CI/CD pipelines

Soft skills: problem solving, communication, collaboration Work Experience:

Client: Discover financials, IL Feb 2025 to present Role: Data Engineer

Designed and developed ETL pipelines using Azure Data Factory and Databricks, processing 5TB+ daily data.

Built real-time streaming pipelines with Kafka and Spark Structured Streaming for processing customer clickstream data.

Migrated on-premise SQL Server data warehouse to Azure Synapse Analytics, reducing query time by 40%.

Developed data quality frameworks ensuring 99.9% data accuracy for downstream analytics.

Automated reporting workflows in Tableau and Power BI, improving reporting efficiency by 30%.

Implemented data quality checks using PySpark and SQL, achieving 98% data accuracy. Client: American Express, India June 2020 to July 2023 Role: Data Engineer

Designed and deployed batch ETL jobs in AWS Glue & Spark, processing structured and semi-structured data.

Implemented data partitioning and bucketing strategies in Hive, improving query performance by 25%.

Built data ingestion pipelines from multiple sources (APIs, flat files, RDBMS) into AWS S3 and Redshift.

Partnered with data scientists to deliver feature engineering pipelines for machine learning models.

Conducted performance tuning of SQL queries and optimized data models for faster analytics.

Automated workflows using Apache Airflow, reducing manual intervention by 40%. Educational details:

Masters in Computer Science from Roosevelt University, Chicago, USA.

Bachelors in computer Science from Vishnu Institute of Technology, Bhimavaram, India. Certifications:

Microsoft Certified: Fabric Data Engineer Associate

Contact this candidate