Data Engineer Senior

Harshitha Kotichintala

Senior Data Engineer

848-***-**** ️ *************@*****.*** LinkedIn Professional Summary

Senior Data Engineer with 5.3 years of experience in designing scalable ETL pipelines, handling structured & unstructured data, and implementing batch & streaming processing using Spark, PySpark, Databricks, Snowflake, and AWS. I am skilled in data validation, transformation integrity, fault tolerance, and monitoring. Technical Skills

Big Data & Processing: Apache Spark (PySpark, Spark SQL), Databricks, Delta Lake, Hadoop

Programming & Frameworks: Python, SQL, Shell Scripting, Django

Cloud & Data Warehousing: AWS (Glue, S3, Lambda, Athena, EMR, Redshift), Snowflake

ETL & Pipelines: Incremental loading, CDC, Batch & Streaming (Kafka, Structured Streaming), Airflow, StreamSets

Professional Experience

Senior Data Engineer Johnson & Johnson, India (Jul 2023 - Sep 2023)

Led migration of 50TB+ data to Snowflake using Databricks and Spark, optimizing processing speeds.

Implemented CDC for efficient incremental loading and real-time processing via Kafka & Snowflake Streams.

Automated ETL workflows using Apache Airflow, improving operational efficiency. Tech Stack: Snowflake, Databricks, Apache Spark, Kafka, Airflow, Python Senior Data Engineer Larsen & Toubro (NatWest Bank) (Aug 2022 - Jun 2023)

Migrated hybrid data sources to Snowflake using Spark, AWS Glue, and Databricks.

Replaced Informatica ETL with Spark & Python, boosting performance.

Optimized Snowflake and S3 for faster query execution and automated ETL monitoring via Airflow.

Tech Stack: Snowflake, Spark, AWS Glue, Databricks, Kafka, Airflow, Python Data Science Engineer Finisar II-VI India (Jun 2018 - Jul 2022)

Automated data ingestion from SQL Server & Oracle to Hive using StreamSets.

Built a Django app to monitor 120+ StreamSets pipelines dynamically.

Developed ML-based security threat detection models and visualized data in Neo4j.

Tech Stack: Hadoop, StreamSets, Snowflake, Python, Django, ML (Basics), Tableau

Education

M.S. Business Analytics, Sacred Heart University, CT (2024 - 2025)

MBA, NIT Warangal (2016 - 2018)

B.Tech, Kakatiya University (2012 - 2016)

Achievements

Built a Django app during lockdown to monitor 200+ pipelines using StreamSets' REST API, streamlining operations and eliminating redundant email notifications.

Contact this candidate