Data Engineer Processing

Dinesh Reddy Mulaka

Data Engineer

US, 469-***-****, ******.**********@*****.***

Professional summary

Data Engineer with over 5 years of experience in designing and implementing advanced data pipeline solutions, specializing in ETL, big data, and cloud technologies. Proficient in Python, Spark, and AWS, driving significant improvements in data processing efficiency and cost optimization. Passionate about leveraging innovative data engineering techniques to deliver transformative insights and support strategic decision-making.

Employment history

Data Engineer, Oct 2023 - Present

Electronic Arts, TX

•Designed and implemented scalable ETL pipelines using Azure Data Factory, T-SQL, Spark SQL, and U-SQL, improving data ingestion efficiency by 40% across Azure Data Lake, Azure SQL, and Databricks.

•Optimized Spark applications in Scala and Pyspark, reducing data transformation time by 30% for large-scale datasets.

•Migrated legacy MapReduce programs to Spark, enhancing performance by 60% and cutting infrastructure costs.

•Configured and managed Apache Kafka clusters for real-time data streaming, ensuring high availability and fault tolerance.

•Automated ETL workflows with Apache Airflow, integrating with AWS S3 and Snowflake, resulting in 40% faster data processing.

•Designed and implemented dimensional data models (Star, Snowflake schemas) to optimize data storage and reporting.

•Developed interactive Tableau and Power BI dashboards, enabling real-time decision-making for stakeholders.

•Executed complex SQL and Hive queries for data validation and transformation, ensuring 99.9% data accuracy.

•Managed version control and CI/CD pipelines using Git, streamlining collaborative development and deployment.

•Worked in Agile/Scrum teams, participating in sprint planning, retrospectives, and cross-functional meetings.

•Created and optimized high-concurrency Spark clusters in Azure Databricks, improving query performance by 50%.

•Developed stored procedures, views, and triggers using SQL and PL/SQL, improving database performance by 35%.

•Integrated Hive queries with Tableau dashboards, enhancing data accessibility and visualization across the enterprise.

•Collaborated with business users to define technical requirements and deliver data-driven insights.

Data Engineer, Oct 2019 - Jun 2022

Tech Mahindra Ltd., India

•Developed Spark applications on Amazon EMR using Scala and Spark SQL/Streaming for high-performance data processing and validation.

•Leveraged AWS Glue and EMR for Spark SQL data processing, storing outputs in Amazon S3 and querying datasets using Athena.

•Built AWS Lambda functions integrated with Pyspark to perform real-time data aggregation and validation workflows.

•Implemented data encryption and hashing techniques using AWS KMS to meet client-specific data security standards.

•Utilized Apache Airflow (MWAA on AWS) for orchestrating, scheduling, and monitoring ETL pipelines.

•Designed DAGs to automate ETL workflows, optimizing pipeline performance and ensuring scalability.

•Developed interactive reports and dashboards in Amazon Quick Sight, integrating data from S3, Redshift, and RDS for real-time insights and comparisons between legacy and current systems.

•Collaborated with business stakeholders to gather requirements and design data products supporting analytical and reporting needs.

•Optimized Amazon Redshift queries and schema designs to improve performance and reduce operational costs.

•Built normalized (3NF) data models for ODS/OLTP systems and implemented dimensional modeling with star and snowflake schemas for efficient reporting.

•Engineered real-time data streaming pipelines from Amazon Kinesis Data Streams to S3 and into HDFS.

•Developed and executed AWS migration strategies using DMS, Glue, and Data Lake services to transition legacy systems to cloud- native architectures.

•Integrated data ingestion workflows from REST APIs via AWS API Gateway and streamed data into Kinesis for real-time processing.

•Maintained version control and collaborative development workflows using AWS Code Commit.

•Wrote complex HiveQL scripts to transform and prepare data within Redshift for advanced analytics.

•Created stored procedures in Redshift and RDS to support data transformation as part of end-to-end ETL processes.

Education

Masters in Computer Science, Aug 2022 - Dec 2023

Campbellsville University, Kentucky, US

Bachelors in Mechanical Engineer, 2016 - 2020

Koneru Lakshmaiah Educational Foundation, Andhra

Courses

Azure Data Scientist Associate

Microsoft

Advanced SQL

HackerRank

Python Basic

HackerRank

Skills

Python, SQL, Scala, MATLAB, Spark, Kafka, Airflow, AWS, Azure, GCP, Docker, Jenkins, MongoDB, Cassandra, Snowflake, Tableau, Power BI, Git, Linux, ETL, Hadoop.

Links

LinkedIn: www.linkedin.com.

Contact this candidate