Data Engineer Big

ROHIT SHRIVASTAVA

Shivpuram, Sisodia colony Guna (M.P)

913******* *****.***************@*****.***

https://www.linkedin.com/in/rohit-shrivastava-40b784258 Nov 2024 - Jan 2025

March 2023 - July 2023

October 2022 - Feb 2023

June 2021 - October 2022

Objective

Aspiring to leverage my 2.9 years of experience as a Data Engineer to design and optimize scalable data pipelines using Python, Hadoop, Hive, and Spark. Seeking an opportunity to contribute my expertise in Azure (ADLS Gen2, ADF, Synapse, Azure SQL) and AWS (S3, Glue, Athena, Quicksight, Redshift, Databricks) to build efficient ETL solutions, improve data quality, and enhance business intelligence capabilities. Experience

Collabera Digital (Contract) Client (ZS Associates) Data Engineer

Muvi Entertainment pvt Ltd

Bigdata Engineer

47 Billion (Indore)

Big data developer

Excotron Solutions

Hadoop developer

Projects

Project 1: Border and Cie.

Domain worked on : Banking Domain

Technologies used : Hadoop, Hive, SQL, HBase, Sqoop, Cloudera. Roles and Responsibilities:

Take responsibility for Hadoop development and implementation Writing the script files for processing data and loading to HDFS. Loading files to HDFS and writing Hive queries to process data Involved in partitioning of Hive tables. Creating Hive tables to store the processed data in a tables. Setup Hive with MySQL as a remote Meta store. Moved all the log/text files generated by various products into HDFS location. Project 2 : Data transformation to Data lake.

Domain worked on : Retail domain

Technologies used : Spark, Hive, SQL, HBase

Roles and Responsibilities :

Successfully completed POC on pipeline.

Responsible to merge data coming from different sources and loaded into HDFS. Supported code/design analysis, strategy development and project planning. Used Spark- SQL to process the data and to run the Spark engine. Combined data from MySQL and file source and applied various transformations. Creation of PySpark jobs.

2013

2016

2021

Project 3 : Create 360 view for US Healthcare Industry Overview: Developed a comprehensive 360 view of patient and healthcare data. Technologies: SQL, Spark, PySpark, AWS S3, AWS Glue, Athena Roles and Responsibilities:

Integrated data from multiple healthcare sources into a centralized data lake Processed and transformed large-scale healthcare data using Spark and Hive Created data pipelines to ensure accurate and real-time insights Implemented data validation and governance policies for compliance Optimized data retrieval using partitioning and indexing Skills

Programming Languages: Python Scala

Big Data & Distributed Systems: Hadoop Hive Sqoop HBase PySpark Databases & Data Warehousing: MySQL Redshift RDS Azure SQL Data Warehousing ETL Cloud Technologies: AWS: S3 Glue Athena Quicksight RDS Redshift Databricks. Azure: ADLS Gen2 ADF Synapse Azure SQL

Data Engineering & Analytics: Data Pipelines Data Processing Data Integration Data Validation Performance Optimization

Education

Davv University

BCA

56 %

Jiwaji University

Bachelor of Law

63%

Jiwaji University

MSc (computer science)

71%

Personal Information

Name : Rohit Shrivastava

Marital status: Married

Address: Shivpuram Sisodiya Colony, Guna(M.P)

Pin : 473001

Contact this candidate