Hadoop Developer

SHANKAR RAVULA

+91-810******* E-Mail: *********@*****.***

Professional Summary:

2+ years of overall IT experience in Application Development in Sql and Big Data Hadoop.

1.2 years of exclusive experience in Hadoop and its components like HDFS, Map Reduce, Apache Pig, Hive, Sqoop, HBase.

Extensive Experience in Setting Hadoop Cluster

Good working knowledge with Hive and Apache Pig

Involved in writing the Pig scripts to reduce the job execution time

Very well experienced in designing and developing both server side and client side applications.

Excellent communication, interpersonal, analytical skills, and strong ability to perform as part of team.

Exceptional ability to learn new concepts.

Hard working and enthusiastic.

Knowledge on MongoDB.

Professional Experience:

Currently Working as a Software Engineer in Infosys Ltd, Hyderabad, India since November’ 2013.

Qualifications:

B.Tech

In Jawaharlal Nehru Technological University Hyderabad

Technical Skills:

Languages

MapReduce, Pig, Sqoop, Pig, Hive, Hbase.

Framework

Hadoop.

Version Control / Tracking Tool

SVN.

Database

SQL (DDL, DML, DCL).

Operating Systems

Windows7, Windows XP, Windows8.

Project Details:

PROJECT #1:

Project Name : Target – Web Intelligence

Client : Target Minneapolis, Minnesota, USA.

Environment : Hadoop, Apache Pig, Hive, SQOOP, MySQL.

Duration : Nov 2014 to till Date

Role : Hadoop Developer

Description:

This Project is all about the rehousting of their (Target) current existing project into Hadoop platform. Previously Target was using mysql DB for storing their competitor’s retailer’s information.[The Crawled web data]. Early Target use to have only 4 competitor retailers namely Amazon.com, walmart.com etc….

But as and when the competitor retailers are increasing the data generated out of their web crawling is also increased massively and which cannot be accomodable in a mysql kind of data box with the same reason Target wants to move it Hadoop, where exactly we can handle massive amount of data by means of its cluster nodes and also to satisfy the scaling needs of the Target business operation.

Roles and Responsibilities:

Moved all crawl data flat files generated from various retailers to HDFS for further processing.

Written the Apache PIG scripts to process the HDFS data.

Created Hive tables to store the processed results in a tabular format.

Developed the sqoop scripts in order to make the interaction between Pig and MySQL Database.

Involved in gathering the requirements, designing, development and testing

Writing the script files for processing data and loading to HDFS

Writing CLI commands using HDFS.

Developed the UNIX shell scripts for creating the reports from Hive data.

Completely involved in the requirement analysis phase.

Analyzing the requirement to setup a cluster

Created two different users (hduser for performing hdfs operations and map red user for performing map reduce operations only)

Ensured NFS is configured for Name Node

Setting Password less hadoop

Setting up cron job to delete hadoop logs/local old job files/cluster temp files

Setup Hive with MySQL as a Remote Metastore

Moved all log/text files generated by various products into HDFS location

Written Map Reduce code that will take input as log files and parse the logs and structure them in tabular format to facilitate effective querying on the log data

Created External Hive Table on top of parsed data.

PROJECT #2:

Title : Project Management Information System (PMIS)

Client : DMDE, Secunderabad

Duration : Dec 2013 to Nov 2014

Team size : 3

Environment : Oracle 9i, Windows, Reports.

Role : SQL Developer

Project Description:

Project Management Information system is a Database System which is used for maintaining the entire Database regarding Naval Projects developed by DMDE. This Project Processing Lifecycle of Products from gathering of Technical Requirements to Final Production which includes Master data, financial data, Purchase Orders, Quality Assurance report etc. The centralized data is maintained in the oracle database. The end users will be able to take the Business Decisions based on our different Forecast reports. Finally the data is maintained in different staging servers. The data warehouse systems will be maintained internally for Business analysis and ongoing customization.

Responsibilities:

Requirement gathering and worked according to the CR.

Worked under the senior level guidance.

Data validation/Reconciliation report generation.

Code Development as per the client requirements.

Involved in the development backend code, altered tables to add new columns, Constraints, Sequences and Indexes as per business requirements.

Perform DML, DDL Operations as per the Business requirement.

Creating views and prepares the Business Reports.

Resolved production issues by modifying backend code as and when required.

Used different joins, sub queries and nested query in SQL query.

Involved in creation of sequences for automatic generation of Product ID.

Created Database Objects like tables, Views, sequences, Synonyms, Stored Procedures, functions, Packages, Cursors, Ref Cursor and Triggers.

Testing of code functionality using testing environment.

Contact this candidate