Data Engineer Power Bi

Rakesh Reddy

Data Engineer

573-***-**** ️ ******************@*****.***

Open to relocation LinkedIn

Professional Summary

Experienced Data Engineer with 5+ years of expertise in building and optimizing large-scale data pipelines, data lakes, and real-time analytics solutions across Azure, AWS, and GCP cloud platforms.

Proficient in ETL/ELT development using tools such as Azure Data Factory, AWS Glue, Databricks, and Apache Spark, with hands-on experience in both batch and streaming workloads.

Strong programming background in Python, SQL, and PySpark, capable of writing reusable code modules and complex data transformations.

Built real-time streaming solutions using Apache Kafka, AWS Kinesis, and Spark Structured Streaming for fraud detection and IoT analytics.

Designed and managed data lakehouse and data warehouse architectures using Snowflake, Azure Synapse, and Amazon Redshift.

Experienced in Docker and Kubernetes for containerization and orchestration of Spark/Python applications in production environments.

Implemented robust CI/CD pipelines using Azure DevOps, GitHub Actions, and Jenkins, supporting continuous deployment and rollback strategies.

Automated infrastructure provisioning using Terraform and AWS CloudFormation, reducing setup time and enabling reproducible environments.

Practiced secure coding and access control techniques including IAM, Azure Key Vault, Secrets Manager, and row-level security in Snowflake.

Developed business dashboards and reporting solutions using Power BI, Tableau, and Amazon QuickSight, delivering critical insights to stakeholders.

Adept in Agile and Scrum methodologies, regularly participating in sprint planning, demos, and retrospectives.

Passionate about performance tuning, cost optimization, automation, and enabling advanced analytics for scalable business impact.

Technical Skills

Cloud Platforms: Azure (ADF, ADLS, Databricks, Synapse), AWS (Glue, S3, Redshift, EMR, Lambda, IAM), GCP (BigQuery, Cloud Storage)

Big Data & Stream Processing: Apache Spark, PySpark, Databricks, Apache Kafka, Kinesis, Apache Airflow, Hadoop

Programming & Scripting: Python, SQL, Shell Scripting, JSON, YAML, Git ETL/ELT & Orchestration: Azure Data Factory, AWS Glue, Informatica, Apache Airflow, AWS Step Functions, Tidal

Data Warehousing & Lakehouse: Snowflake, Azure Synapse, Amazon Redshift, Delta Lake, Hudi DevOps & Containerization: Docker, Kubernetes, Azure DevOps, GitHub Actions, Jenkins, Terraform, CloudFormation

Monitoring & Security: Azure Monitor, AWS CloudWatch, Datadog, Azure Key Vault, AWS Secrets Manager, IAM

Data Visualization & BI: Power BI, Tableau, Amazon QuickSight Collaboration & Agile Tools: Jira, Confluence, Postman, Swagger, Agile/Scrum Education

Master of Science in Computer Science

Southeast Missouri State University – Cape Girardeau, MO Professional Experience

Client: Fiserv – Alpharetta, GA

Data Engineer Jan 2023 – Present

Environment: Azure, AWS, Snowflake, Databricks, Apache Airflow, Kafka, Docker, Kubernetes, Power BI

Designed and implemented robust ETL/ELT pipelines using ADF, Databricks, and AWS Glue for ingesting and processing data from varied sources.

Developed real-time analytics pipelines using Apache Kafka and Spark Structured Streaming for fraud detection and transactional monitoring.

Engineered ELT solutions in Snowflake using stored procedures, CTEs, and analytical functions for optimized reporting and data enrichment.

Managed workflow orchestration through Apache Airflow with custom-built DAGs and SLA alerting.

Automated deployment pipelines via Azure DevOps and GitHub Actions, ensuring continuous integration and production readiness.

Containerized data processing jobs with Docker and orchestrated using Kubernetes for autoscaling and fault tolerance.

Integrated Azure Key Vault and AWS Secrets Manager for secure credentials and API keys handling.

Developed executive dashboards in Power BI and Tableau for daily KPIs, SLA breaches, and operational insights.

Enabled fine-grained access control using Snowflake role-based access and secure views for compliance.

Collaborated with data science teams to deliver feature-ready datasets for machine learning models.

Monitored system and pipeline health via Azure Monitor, CloudWatch, and custom alerts.

Optimized query and pipeline performance using partitioning, caching, and cluster tuning. Client: PwC – Hyderabad, India

Data Engineer Jun 2019 – Jul 2021

Environment: Azure, AWS, Databricks, Snowflake, Kafka, Power BI, Python, SQL

Developed scalable ETL workflows using Databricks (PySpark) and ADF for SAP ERP and other legacy data systems.

Built and maintained streaming data ingestion pipelines using Apache Kafka and Spark Streaming.

Designed multi-zone data lakes on ADLS Gen2 and managed ingestion into Snowflake using data pipelines.

Built business-specific data marts and applied RBAC in Snowflake for secure access across departments.

Created Power BI dashboards with dynamic row-level security and real-time refresh.

Contributed to Agile development cycles including planning, estimation, and review of features.

Created unit test scripts and validation frameworks to ensure data quality in production.

Implemented DevOps automation with Git, Jenkins, and Terraform for infrastructure provisioning. Client: TCS – Hyderabad, India

Data Engineer Jun 2018 – May 2019

Environment: Azure, Snowflake, Informatica, Synapse, SQL, Python, Power BI

Designed and implemented ETL workflows using Informatica PowerCenter and Azure Data Factory

(ADF) to integrate healthcare claims, patient records, and provider data into centralized data lakes.

Developed advanced SQL logic and Python scripts for data cleansing, enrichment, and anomaly detection across millions of records.

Engineered data pipelines that supported data ingestion from flat files, APIs, and legacy RDBMS into Azure Synapse Analytics and Snowflake.

Built dimensional data models and optimized analytical queries for faster report generation using Synapse SQL Pools.

Developed interactive dashboards in Power BI for healthcare KPIs including patient wait times, claim approval rates, and provider performance.

Implemented data quality frameworks to validate data integrity using Python, stored procedures, and Informatica checks.

Collaborated with data analysts and BI developers to ensure clean, ready-to-use datasets were available in Snowflake for business insights.

Supported cloud migration initiatives by re-engineering legacy ETL jobs and scripts to run on cloud- native platforms.

Configured monitoring alerts using Azure Monitor and Log Analytics to track pipeline failures and job performance.

Participated in Agile delivery cycles including sprint planning, user story grooming, and retrospectives to ensure timely project delivery.

Documented data workflows, dependencies, and metadata lineage for data governance and audit readiness.

Collaborated with DevOps team to integrate deployment of data pipelines into CI/CD pipelines using Azure DevOps and Git.

Contact this candidate