Data Engineer Big

Sushmitha Reddy Pollapalli

@: ***************@*****.*** Ph: +1-516-***-****

http://linkedin.com/in/sushmitha-p-79928a167

SUMMARY

• Accomplished Big Data Engineer with over 5 years of hands-on experience in designing, implementing, and managing complex data architectures and analytics solutions.

• Proficient in a wide range of Big Data technologies, including Hadoop, Spark, Kafka, SQL, Cassandra, Hive, Sqoop, NiFi, and cloud platforms like AWS, Azure, and GCP.

• Strong expertise in data integration using tools like Sqoop and NiFi, and real-time data processing with Kafka and Spark Streaming.

• Skilled in programming languages Scala and Python, with PySpark experience for integrating Python and Spark.

• Deep understanding of NoSQL databases, particularly Cassandra, for handling high-availability data volumes and scalability.

• Demonstrated ability to architect and implement scalable data pipelines for efficient data ingestion, transformation, and analysis.

• Proven track record of delivering data-driven solutions that have enabled organizations to make informed decisions and gain competitive advantages.

PROFESSIONAL EXPERIENCE

TECHPRO SOLUTIONS INC Feb 2024 – Current

Role: Data Engineer (GCP) NJ, USA

Responsibilities:

• Developed and maintained data pipelines using GCP services like Dataflow and Cloud Functions.

• Managed data storage solutions leveraging Google Cloud Storage, Big Query, and Cloud SQL.

• Designed and optimized ETL processes with Google Cloud Data Fusion.

• Resolved data issues using Google Cloud Monitoring and Logging.

• Collaborated on data modeling and integration for analytics and reporting. DELOITTE May 2022 – Aug 2022

Role: Data Engineer (AWS) Hyd, India

Responsibilities:

• Architected and managed data storage solutions using Amazon S3, Redshift, and RDS.

• Constructed and optimized ETL pipelines with AWS Glue and Data Pipeline.

• Engineered data models and schemas using AWS Glue Data Catalog and Amazon Redshift.

• Developed and maintained data quality processes to ensure data accuracy and integrity.

• Worked closely with cross-functional teams to understand data requirements and deliver actionable insights.

ICICI LTD Oct 2020 – May 2022

Role: Data Engineer (AWS) Blr, India

Promoted to

Role: Big Data Developer

Responsibilities:

• Architected and governed data storage solutions on AWS (S3, Redshift, RDS).

• Engineered and optimized ETL pipelines using AWS Glue and Data Pipeline.

• Automated and orchestrated data infrastructure with CloudFormation and CodePipeline.

• Developed and maintained complex Spark applications using Scala, PySpark, and Spark SQL.

• Implemented and optimized real-time streaming data processing with Kafka and Spark Streaming. ACCEND SYSTEMS PRIVATE LIMITED May 2018 – Oct 2020 Hadoop Developer Blr, India

Promoted to

ETL Developer

Responsibilities:

• Orchestrated data ingestion and extraction into HDFS and Hive using Sqoop.

• Constructed Hive tables, executed data loading, and authored Hive queries.

• Optimized Hive queries for cost reduction.

• Processed large datasets using Hadoop MapReduce jobs.

• Managed Hive tables and partitions.

TECH STACK

Big Data and Analytics: Hadoop (HDFS, MapReduce, YARN), Spark (RDD, DataFrames, Spark SQL), Kafka, Cassandra, NiFi.

Cloud Services: AWS (S3, EMR, EC2, Athena, RDS, Kinesis), Azure (Blobs, ADL, HDInsight, VM, SQL), GCP (Storage, Dataproc, Compute Engine, Teraform).

Data Integration and Warehousing: Sqoop, Hive, Python, Scala, PySpark, SQL (MySQL, PostgreSQL), Apache NiFi.

ACADEMIC PROJECTS

University Of Missouri, KC Aug 2022- Dec 2023

1. Live Twitter Data Analysis on Cloud: Developed a real-time Twitter data analysis application using AWS cloud services, PySpark, and Plotly to extract, process, and visualize data. 2. Spotify Data Analysis: Conducted in-depth analysis of Spotify data trends using PySpark and Azure cloud services, generating reports and dashboards.

3. Traffic Simulation and Prediction: Implemented a traffic prediction model using Triple Exponential Smoothing and visualized traffic patterns on a Chicago map, incorporating real-world street data for optimal route planning.

EDUCATION

University Of Missouri, KC Aug 2022- Dec 2023

MS Computer Science (Data Science) MO, USA

Indus Business Academy Jun 2018 – Dec 2020

PG Diploma in Management (Marketing & Finance) Blr, India Osmania University Jun 2015- May 2018

Bachelor of Commerce (Computers) Hyd, India

Contact this candidate