Post Job Free

Resume

Sign in

Spark, Kafka, Hbase, Scala,Hadoop, Hive, Pig, Sqoop, Oozie,Cassandra

Location:
Mumbai, Maharashtra, India
Posted:
December 26, 2019

Contact this candidate

Resume:

E-mail: ada5hg@r.postjobfree.com

Phone: +91-702*******

SUMMARY

4+ Years of experience in BigData Technologies. Hands on experience with Big Data Hadoop core components and Eco Systems (HDFS, Hive, Pig, Spark, Sqoop, Flume, Oozie, Kafka and NiFi).

An ambitious professional aiming to acquire a responsible position in the industry that utilizes my leadership

and learning potential and facilitates my growth as well as sharpen my sight into constantly evolving technology.

Technical Skills

Hadoop Ecosystems

HDFS, Spark, Hive, Pig, Sqoop, Flume, Tez, Oozie.

Database

MySQL, Cassandra, Hbase

Tools

Sqoop, Hive, Flume, Nifi, Kafka, Elastic Search.

Languages

Scala, Pig Latin, HiveQL.

IDE’s

Eclipse, DBeaver.

Operating Systems

Windows, Ubuntu, Cent-OS.

EXPERIENCE & PROJECTS

Saama Technologies Ltd.

Role: Hadoop/Spark Developer

Project 1: ControlTower Sept 2018 – Present

Blue Dart Express or Blue Dart is an Indian Logistics Company providing courier delivery services headquartered in Chennai, Tamil Nadu. It has a subsidiary cargo airline, Blue Dart Aviation that operates in South Asian countries.

Contribution:

• Oracle GG pushed the data to the kafka topic.

• Developed apache spark-streaming code to read data from Kafka topic.

• Implemented apache spark code to read multiple tables from the real-time records and filter the data

based on requirement.

• Used spark for data analysis and store final computation result to HBase tables.

• Read HBase tables and creating indexes in Elastic search.

• Used Crontab to schedule the spark job.

Skill Used: Cloudera, Scala, Kafka, Spark-Streaming, HBase, Spark, Elastic Search, Unix.

KPIT Technologies Ltd.

Role: Hadoop/Spark Developer

Project 1: OnTrack-VTS Oct 2017 – June 2018

The goal of this project is to implement a set of sensors into a commercial vehicle suspension to derive the payload weight being carried by the vehicle. OnTrack VTS UI application allows users to visualize the data collected by the DCC (Data Collection Component) from VTS boxes deployed on the vehicle. The basic objective of the UI is to display vehicle location and the payload weight that is carrying at any given point of time.

Contribution:

• Developed shell script to load the json data to the Kafka topic.

• Developed apache spark-streaming code to read data from Kafka topic.

• Wrote apache spark code to read json formatted data from Cassandra table.

• Used spark-sql for data analysis and store final computation result to Cassandra tables.

• Used Crontab scheduler to schedule the spark job.

Skill Used: Hortonworks Distribution (HDP2.3), Scala, Kafka, Spark-Streaming, Cassandra, Spark, Oozie.

KPIT Technologies Ltd.

Role: Hadoop/Spark Developer

Project 2: Intelliwind March 2017 – Sept 2017

The KPIT IntelliWind is a new breed of wind farm monitoring and action platform that visualizes, analyzes, predicts and diagnoses your wind turbine performance. The product presents a cross device, cloud based product catering to users playing various roles within the organization. The dashboard will help the users visualize, analyze and act turbine power generation and general health, all while giving the users the opportunity to perform predictive and preventive analysis.

Contribution:

• Developed Apache NiFi flow to read data from Hadoop edge node and put it to Kafka topic.

• Developed apache spark-streaming code to read data from Kafka topic and push data into Cassandra.

• Created Cassandra table for data computation.

• Wrote Spark code to read data from Cassandra tables.

• Used spark-sql, spark-dataset to perform data computation and store final computation result

to Cassandra tables.

• Schedule spark job through Oozie workflow engine.

Skill Used: Hortonworks Distribution (HDP2.3), Scala, Apache NiFi, Kafka, Spark-Streaming, Cassandra, Spark, Oozie.

Capgemini India Pvt Ltd.

Role: Hadoop Developer

Project 3: Enterprise Data Services (EDS) March 2015 – Feb 2017

The EDS jobs have been designed to receive feed from source systems, process the data, load the appropriate warehouse tables and also send feed to destination system with minimal manual intervention .The purpose of the this project is collecting, analyzing and reporting aggregate data. This solution was to process and analyze the structured and semi structured data. This solution also involved processing and filtering over semi-structured data. This solution was executed on Hortonworks environment using Pig and Hive as processing engines.

• Understanding the Machine Attributes and its data.

• Cleanse data and give it a proper structure in Pig.

• Load the data from pig into Hive table for analysis using HCatalog.

• Developed the Sqoop command in order to make the interaction between Hive and Mysql database.

• Tested raw data and executed performance scripts.

Skill Used: Hadoop Server, HDFS, Hive, Pig, Sqoop, Hcatalog, MySql, Unix.

QUALIFICATIONS

• B.E from Mumbai University with 69.23% marks.

• Diploma, Maharashtra Board with 84.74% marks.

• SSC, Maharashtra board with 84.46% marks.

CERTIFICATIONS

• Hortonworks Certified Hadoop Developer (HDPCD) Jan 2016.

• http://bcert.me/sxeobvza

Professional Organizations

• Saama Technologies Ltd (Sept 2018 – Present).

• KPIT Technologies Ltd (March 2017 – June 2018).

• Capgemini India Pvt Ltd (Feb 2015 – March 2017).

Personal Information

Date of Birth 7th Dec 1991

Nationality Indian

Pan Card No BEYPG5973B.

Passport No L8101002.

Current Address E-603, Prime Plus Building, Near ICICI bank, Pimple Saudagar, Pune-411027.

Permanent Address RM No-02, Mauli Krupa Society, DahnukarWadi, Kandivali (w), Mumbai-400067.

Declaration

I hereby declare that all particulars mention above are true and correct to the best of my knowledge and beliefs.



Contact this candidate