Data Engineer Business Intelligence

ANIRUDH SUNDARESAN

**************@*****.*** 857-***-**** Sunnyvale, CA

SUMMARY

• Experienced Data Engineer with 3+ years of expertise in designing, building, and optimizing large-scale, high-performance data systems for e-commerce and business intelligence applications.

• Strong background in data modeling, data warehousing, ETL processes, and analytics, with a focus on reducing latency and ensuring data platform health.

• Proficient in Python, Java, and Scala, with deep expertise in databases (SQL/NoSQL), big data technologies (Hadoop, Apache Spark, Kafka, Presto), and cloud platforms (AWS/Azure).

• Skilled in optimizing database performance, streamlining data flows, and automating data processing using AWS services such as Lambda, Redshift, Athena, Glue, S3, and Airflow.

• Skills: Python, JavaScript, Apache Spark, SQL, Cloud Computing, AWS (Glue, S3, Airflow, Lambda, Redshift), VBA Programming, Quicksight, Tableau PROFESSIONAL EXPERIENCE

Data Engineer II, Amazon Web Services – Seattle, WA Oct 2024- Present

• Championed industry best practices in data governance, security, and version control. Fostered a data-driven culture by conducting knowledge-sharing sessions and advocating for the consistent application of best practices in data infrastructure, ensuring alignment between technical and business teams to drive impactful analytics outcomes.

• Built a scalable automation tool for archiving MySQL tables based on partition columns and filters, resulting in a 30% reduction in CPU utilization on RDS instances. This optimization ensured the efficient flow of data across systems while maintaining system performance and data accessibility.

• Revamped Weekly Business Review reporting by implementing engine-level performance tracking and deploying four SQL-based metrics (Overall Summary, Top Gainers, Top Decliners, Revenue Accounts). This initiative significantly improved executive visibility into InfluxDB and Live Analytics services, enabling senior leadership to make data-driven decisions with confidence.

• Led the data onboarding initiative for Database Migration Service (DMS), overseeing ETL processes and schema modifications across multiple databases. This initiative included version tracking and proactive migration monitoring, enhancing analytics capabilities and improving operational visibility for stakeholders.

Data Engineer, Amazon – Seattle, WA Jul 2023 - Sep 2024

• Designed, built, and optimized 150 data pipelines through AWS CDK using SQL and Python to extract, transform, and load (ETL) data into the data lake. These pipelines enabled efficient data accessibility for Business Data Analysts, enhancing operational analytics and supporting large-scale data processing with cloud-based solutions to ensure scalability and flexibility.

• Developed and implemented a comprehensive data engineering framework using AWS Lambda, Airflow, Glue, and S3. This framework not only automated data processing but also ensured high data quality through rigorous checks during ingestion and transformation. The system included real- time alerts via Slack and email for shift events, improving data integrity and saving $900k annually in labor costs.

• Led the migration of two critical ETL pipelines from an internal scheduler to AWS Airflow (MWAA), architecting DAG-based workflow automation with Python and AWS Fargate. This transition enhanced the scalability and reliability of the workflows, improving monitoring and fault tolerance, and facilitating the processing of large datasets with optimized performance across cloud platforms.

• Worked closely with data scientists and analysts to understand evolving business requirements, ensuring the transformation of raw data into actionable insights. Delivered 10 interactive dashboards in Quicksight, providing real-time insights to business users and supporting data-driven decision-making across various teams.

Data Warehouse Support Engineer, Amazon – Seattle, WA Jun 2021 - Jun 2023

• Migrated legacy data pipelines from MySQL to a cloud-based infrastructure using AWS tools such as Glue, Airflow and Redshift improving the performance by 60%.

• Maintained the MYSQL-based data platform supporting labor planning applications by optimizing high-IO data flows, achieving an average query performance improvement of 74% and reducing runtime by two hours per flow.

• Enabled the team to generate reports seamlessly by moving the Excel based reports from a physical desktop to Quicksight Dashboards enabling real-time reporting reducing the latency by 2 hours and the error rates to 0%. Operations Analyst, Wayfair LLC (Internship) – Boston, MA Jul 2019 - Dec 2019

• Automated a process using Excel macros, VBA programming and SQL scripting saving my division over 50 working hours per month.

• Carried out ad hoc requests to derive KPIs by building SQL queries to identify sales metrics such as avg conversion rate and customer turnover rate.

• Built an algorithm on Python to identify sales trends, top selling products, and evaluate product dimensions resulting in increased revenue generation from new businesses by 40%.

EDUCATION

Master of Science in Industrial Engineering, Northeastern University, Boston, MA Sep 2018 - Dec 2020 Bachelor of Engineering in Mechanical Engineering, R N S Institute of Technology, Bangalore, India Aug 2014 - Jun 2018 ACADEMIC PROJECTS

Analysis of Opioid Drug Utilization – Microsoft SQL Server, PostgreSQL, MS Visual Studio, Talend, Tableau Spring 2020

• Automated ETL processes, by building a data pipeline on MS Visual Studio making it easier to wrangle data and reducing time by ~40%.

• Designed and developed ETL packages using SQL Server Integration Services (SSIS) to load data from SQL server, flat files to SQL Server database through Bulk Insert and BCP for Enterprise Data Warehouse.

• Created T-SQL scripts and complex stored procedures for data validation and implemented error handling in SSIS using row redirects and check points.

• Developed data mappings between source systems and warehouse components and performed data profiling using Talend.

• Added Slowly Changing Dimensions (SCD) to Drug list table to accommodate addition, deletion and update of drug information.

Contact this candidate