Data Engineer Power Bi

SUPRIYA REDDY

PROFESSIONAL SUMMARY:

Results-driven Data Engineer with over 4+ years of experience designing, developing, and optimizing large-scale data pipelines in cloud and hybrid environments. Proficient in Python, SQL, Spark, and modern data platforms including Azure and AWS. Skilled in building scalable ETL/ELT processes, real-time streaming solutions with Kafka, and advanced dashboards using Tableau and Power BI. Proven ability to enhance data quality, automate workflows, and deliver high-performance data solutions in Agile settings.

TECHNICAL SKILLS:

oProgramming Languages: Python, Java, R, SQL, Scala

oCloud Computing: AWS, Azure, Security (RBAC), Compliance (GDPR, HIPAA, FISMA)

oContainerization: Docker, Kubernetes

oConfiguration Management Tools: Ansible, Chef, Puppet, Terraform

oVisualization Tools: Tableau, Tableau Desktop,Power BI, Matplotlib, Seaborn, Splunk

oSDLC/Testing Methodologies: Agile, Waterfall, Scrum, TDD, Kanban

oCI/CD Tools: Jenkins, Azure DevOps, GitHub Actions, GitLab pipelines

oData Processing &Pipelines: ETL/Teleprocesses, Databricks, Talend, AWS Glue, Azure Synapse, Informatica

oData Modeling & Database Management: Database Design, Data Warehousing, Redshift, SQL Server,Data Lakes, Logical & Physical Data Modeling

oStreaming Platforms: Apache Kafka, Apache Flink, AWS Kinesis, Azure Event Hubs

oMachine Learning & Data Analytics: Pipelines, Data Analysis, Reporting Dashboards, Amazon Sage Maker

oProgramming Concepts: Event-driven and asynchronous programming, multi-threading

oTesting Automation: Unit testing frameworks, Python-based automated scripts, experience with QA techniques.

EDUCATION:

oMaster’s in computer science-New England College

PROFESSIONAL EXPERIENCE:

Citi Group, Warren, NJ Nov 2023–Present Data Engineer

Responsibilities:

Led the development of scalable data pipelines for SACWIS, enhancing data integration and decision-making.

Designed and implemented ETL pipelines using Azure Data Factory, Informatica, and Apache Airflow, improving data quality and reducing delivery times by 40%.

Migrated legacy on-premises Oracle database to Snowflake, optimizing query performance and enhancing scalability.

Built real-time data streaming solutions using Kafka and Spark Streaming for improved event-driven processing.

Designed, developed, and maintained reports and multi - level dashboards in Tableau.

Built interactive dashboards and stories using Tableau Desktop for accuracy in report generation applying advanced tableau functionality: parameters, actions and tooltip changes.

Developed Python and SQL-based data processing scripts to automate transformation logic and handle large-scale data processing across various sources.

Worked on designing and development of Tableau visualization solutions and Created Business requirement documents and plans for creating dashboards.

Developed automated monitoring dashboards in Power BI, providing real-time insights into workforce and social service programs.

Defined best practices for Tableau report development and effectively used data blending feature in tableau.

Automated CI/CD pipelines using Terraform and Azure DevOps, reducing deployment times by35%.

Engineered bigdata processing workflows using Databricks and Delta Lake to enable scalable and fault-tolerant data storage.

Integrated natural language processing (NLP)for text analytics on case management data, improving fraud detection.

Collaborated with cross-functional teams to streamline data lake architecture and enhance reporting capabilities.

Developed and integrated REST API connections to support dynamic data ingestion from external systems.

Optimized complex SQL queries and stored procedures to accelerate data extraction and transformation

processes.

Provided training and mentorship to junior data engineers, improving team performance and knowledge sharing.

Conducted data quality assessments, ensuring integrity and accuracy across large datasets.

Environment: Microsoft Azure, Azure Data Lake, Snowflake, Azure Data Factory, Informatica, Apache Airflow, Databricks, Apache Spark, Delta Lake, Apache Kafka, Spark Streaming, Oracle, SQL Server, PostgreSQL, Tableau, Tableau Desktop, Power BI, Terraform, Azure DevOps, Python, SQL, Prometheus, Grafana, Kubernetes, Docker.

YUPP, HYD, India June 2019–May 2022

Data Engineer Responsibilities:

Contributed to the estimation and development of modules for an electronic prescription application, enhancing operational efficiency and cross-team collaboration.

Designed and developed scalable ETL pipelines, improving data ingestion efficiency by 35%.

Built real-time streaming solutions, reducing event processing latency by 50%.

Migrated data from on-prem Oracle databases, optimizing query performance and reducing storage costs.

Implemented cloud-based containerized solutions using Docker and Amazon Elastic Kubernetes Service (EKS), improving the flexibility and scalability of data engineering pipelines.

Introduced automation and DevOps practices, leveraging AWS Code Pipeline, Terraform, and Kubernetes for Continuous Integration/Continuous Deployment (CI/CD), reducing manual intervention and increasing deployment frequency by 40%.

Implemented data partitioning and indexing techniques to optimize query execution in Snowflake.

Developed complex SQL queries, stored procedures, and views for real-time fraud detection analytics.

Integrated Tableau dashboards for business reporting, providing real-time insights to stakeholders.

Leveraged Delta Lake on Databricks for structured and semi-structured data processing.

Ensured data governance and security compliance using AWSIAM and encryption mechanisms.

Conducted performance tuning of Spark jobs to improve processing times by 45%.

Collaborated with cross-functional teams, aligning data solutions with banking regulations.

Provided mentorship to junior engineers, improving team productivity and knowledge sharing.

Created database schemas and data models for longitudinal data analysis and research initiatives.

Proficient in SQL optimization and creating stored procedures for data extraction and transformation.

Created Tableau and SSRS dashboards, enabling data-driven decision-making.

Mentored junior analysts on data engineering best practices, SQL optimization, and Python scripting, boosting team productivity.

Planned and managed cloud-based data warehouses (Snowflake, Redshift) for optimized query performance and analytical reporting.

Environment: AWS (EKS, Redshift, IAM, Code Pipeline), Snowflake, Databricks, Delta Lake, Apache Spark, Apache Kafka, Oracle, SQL Server, PostgreSQL, Tableau, SSRS, Docker, Kubernetes, Terraform, Python, SQL

Contact this candidate