Data Engineer Quality

MANJU VISVANATHAN

*****.*********@*****.*** 832-***-**** Houston, TX 77063

Summary

Data Engineer with expertise in PySpark and AWS services, focused on modernizing ETL processes to improve performance and data quality. Demonstrated ability in troubleshooting and incident management, ensuring high-performance data pipelines. Collaborated with cross-functional teams to deliver robust data solutions. Skills

• Programming languages: Python, Scala

• Data processing frameworks: PySpark, ETL, and Hadoop

• Database management: Snowflake, SQL

• Development tools: Visual Studio, IntelliJ, Eclipse

• Cloud services: AWS Glue, EMR, S3, EC2, Lambda, DynamoDB, ECS, and EKS Experience

Data Engineer - 3 Years’ Experience Cognizant Capital One Houston, TX, USA 02/2022 – 04/2025

• Analyzed existing legacy SQL scripts to pinpoint available modernized sources in the Streaming Data Platform, designed, and developed ETL processes using the PySpark framework to load enterprise data into the data warehouse.

• Collaborated with data scientists and analysts to understand data needs and implement appropriate data models to translate business needs into technical specifications for data solutions within our bank retail products.

• Conducted real-time analytics on streaming data using Snowflake and Spark to derive insights and support decision- making processes for debit card transactions.

• Designed, constructed, and maintained scalable ETL data pipelines for data ingestion, cleaning, and processing using Python and SQL.

• ETL workflows are scheduled and managed through DAGs using Apache Airflow.

• AWS Glue is used to automatically run ETL jobs when new data arrives in Amazon S3 buckets.

• Provided round-the-clock support for ETL jobs while gaining expertise in incident troubleshooting and management.

• Created stored procedures for automating periodic tasks in SQL, and optimized SQL queries and database schemas for performance improvements in data retrieval operations.

• Automated data quality checks are performed by creating data-wise rules in New Relic dashboards to ensure the integrity and reliability of datasets.

• Extensive knowledge and experience using Python data frame libraries, like Pandas and Polars.

• Managed version control and deployment of data applications using Git, Docker, and Jenkins.

• Provided technical mentorship to junior data engineers, guiding them on best practices and project execution.

• Documented data architecture designs and changes, ensuring knowledge transfer and system maintainability.

• Participated in agile development processes, contributing to sprint planning, stand-ups, and reviews to ensure timely delivery of data projects.

• Conducted rigorous testing and validation of data pipelines to ensure the accuracy and completeness of data using enterprise-wide model platforms.

• Configured and maintained cloud-based big data infrastructure on platforms like AWS to enhance data storage and computation capabilities.

Facets Business Analyst - 2 Years’ Experience Accenture - India 06/2010 - 04/2012

• Analyzed stored procedures in Microsoft SQL Server, XML, and XSD to gather requirements.

• Documented application architecture, functionality, requirements, use cases, and flow charts.

• Designed comprehensive test plans, scenarios, scripts, and results matrices for system validation.

• Collaborated with development team to architect custom domain models and technical implementation solutions.

• Conducted detailed analysis supporting system test case creation for online testing in Claim module.

• Facilitated implementation and enhancement of critical modules including Providers, Claims, and Membership. Facets Developer - 2 Years’ Experience Cognizant - India 08/2008 - 05/2010

• Developed user interfaces and reporting solutions on SQL Server 2005 for Facets.

• Streamlined reporting processes by executing automated stored procedures within Facets Reporting server.

• Created innovative interfaces while addressing challenges in current applications to improve performance.

• Recommended strategic approaches based on analysis of business needs.

• Produced detailed design documentation, including functional and technical specifications for ad hoc requests. Education

University of Houston Houston, TX, USA 12/2018 Master of Science: Computer Science GPA: 3.92

Sri Venkateswara College of Engineering Chennai, Tamil Nadu, India 05/2008 Bachelor of Engineering: Computer Science

Contact this candidate