Hitesh

Contact Details: 484-***-****.

Hadoop Developer.

Software Professional with 8+ years of experience in Hadoop and Web application development in Java/J2EE technologies and excellent management and communication skill.

•Knowledge of Large Data Clusters with ability to work in administration of large data clusters in big data environments using Hadoop technologies.

•Well acquainted with the complete software development life cycle: Design, Development, Testing, Deployment and Documentation.

•Excellent project implementation skills, a team-player with good communication, problem-solving and analytical skills and ability to quickly adapt new environments and technologies.

TECHNICAL SUMMARY

•Eight (8+) years of IT Experience in various domains such as Financial Services and Manufacturing.

•Experience in Hadoop Technologies and its components like Hortonworks, HUE, HDFS, Hive, Impala, HBase, Pig, Spark, Scala, Spark-Sql, Kafka,Oozie, Sqoop, Mapreduce.

•Experience in developing Spark programs using Scala, RDD and SparkSQL for processing and analytics of data.

•Experience in AWS technologies - Amazon EC2,Amazon S3,IAM,Auto Scaling.

•Experience in Amazon EC2 infrastructure for computational tasks and Simple Storage Service (S3) as Storage mechanism.

•Design recommend best approach suited for data movement from different sources to HDFS using Apache/Confluent Kafka.

•Implementation of data ingestion large volumes of data to Kafka.

•Implementation and adding the performance tuning parameters for Kafka Producers,Broker,Rest Proxy and Consumers for optimum performance.

•Wrote Kafka producers to stream the data from external rest APIs to Kafka topics.

•Implementation in Kafka brokers, zookeepers, Kafka connect, schema registry, KSQL, Rest proxy and Kafka Control center.

•Experience in creating producers, consumers and consumer groups for handling data.

•Experience working with setting up Confluent Connectors & Replicators to replicate data from one data center to another.

•Experience in Talend ETL Big data tool.

•Experience in understanding the data and designing/Implementing the enterprise platforms like Hadoop Data lake.

•Experience in big data application phases like data ingestion,data analytics and data visualization.

•Experience in designing and developing data lakes using Real time data ingestion into AWS S3.

•Used Avro,Parquet and ORC data formats to store data in HDFS.

•Experience in usage of Hadoop distribution with Cloudera.

•Experience in creating external Hive tables,partitioning,bucketing,loading and aggregating data using HIVE.

•Expertise as a developer in Java J2EE technologies and languages that include Core Java, J2EE, JSP, Servlet, Struts framework, Spring, Hibernate, SQL, REST Web-Services, SOAP and Object Oriented methodology.

•Worked on Hadoop Framework and its ecosystem like MapReduce Programming, HDFS, Hive, Pig, HBase, Spark-Sql,Sqoop, Oozie.

•Hands on Experience in Maven, Git and Eclipse.

•Extensive experience in design, development and implementation of software applications in distributed environment like JSP, Servlet.

•Experience in Object-Relational mapper, Hibernate, for data access layer.

•Expertise with Deployment in UNIX Environment and UNIX Commands.

•Experience in handling Client Interactions and ability to quickly adapt to new environments and learn new technologies.

•Good understanding of Software Development Life Cycle (SDLC) and Project Life Cycle.

•Excellent communication, good leadership and problem solving skills.

Technical Skills Summary

Operating Systems:

Windows 2000,Unix, Windows XP

Programming Languages:

C, C++, Java, Scala,J2EE, SQL, HTML.

Technologies:

Hadoop,Spark,Java Servlets, JSP, JDBC, Log4J.

Framework:

Hortonworks,Struts, Hibernate, Spring, JUnit, Pig, Hive.

Databases:

Oracle 9i SQL,Greenplum,WinSQL.

Web/Application Servers:

Apache-Tomcat, Cygwin.

IDEs:

Talend, Redwood, Perforce, JIRA, QuickBuild, Autosys, Eclipse, EPMO (Project management Tool).

Web Designing:

HTML, CSS, XML.

EDUCATIONAL QUALIFICATIONS

•2011 – Bachelor of Engineering (Computers ) – University of Mumbai, India

ADDITIONAL CERTIFICATIONS & SKILLS

OCJP Certified Java Programmer.

Professional Experience:

Company: Infosys Technologies Ltd -USA

Client: Southern California Edison, Rosemead –CA

Duration: Nov 2017 to Present

Role: Technology Lead - Hadoop.

Southern California Edison, the largest subsidiary of Edison International, is the primary electricity supply company for much of Southern California, USA.

The objective of the project is to manage data of various energy sources.

Responsibilities:

•Developing applications using Hadoop Big Data technologies- Hue, Hive, Impala, Spark,Scala, Spark-Sql, Kafka, Map-Reduce, Scala, Hbase and Oozie.

•Developing Spark applications using Scala, RDD and SparkSql for processing of data.

•Importing data using Spark RDD and performing transformations, actions on RDD.

•Imported data from AWS S3 into Spark RDD and performed transformations and actions on RDD.

•Implementation and adding the performance tuning parameters for Kafka Producers,Broker and Consumers for optimum performance.

•Developing Kafka producers to stream the data from external rest APIs to Kafka Topics.

•Experience in Kafka Connect to fetch data from database into Kafka Topic through Kafka Connect Source and Sink Connectors and using Schema Registry to maintain the Schema level details for the Kafka.

•Development of KSQL scripts for querying the data recieved in Kafka.

•Integration of Spark along with Kafka to better movement of data.

•Experience in creating external Hive tables,partitioning,bucketing, loading and aggregating data.

•Worked with various file-formats like Avro, Parquet and ORC.

•Used Spark-Sql to load structured and JSON data and create schema RDD and loaded into Hive Tables using Spark-Sql.

•Imported data from database source using Sqoop into HDFS.

•Experience in using Redwood scheduler to schedule the jobs.

•Experience in designing and development of data lakes using Real time data ingestion into AWS S3.

•Creating HiveQL and Impala scripts for creating the tables and populating the data.

•Experience in using Hive Partitions, Bucketing in Hive and designed both Managed and External tables in Hive to optimize performance.

•Experience in understanding the data and designing/Implementing the enterprise platforms like Hadoop Data lake.

•Designed data Ingestion framework to load business validated data of around 3 million customers into Hadoop Data Lake to store Historical and Incremental data.

•Designing applications based on identified architecture and support Implementation design by resolving complex technical issues.

•Experience in Talend ETL Tool for handling huge volume of data.

•Involved in Customer discussions during requirement analysis with knowledge of requirement elicitation and analysis tools and techniques like story boarding, Agile user stories to understand system requirements and non-functional requirements like performance,security, availability and scalability etc.

•Creating source system data snapshot in Non-production Hadoop environment.

•Design and develop data routing techniques in the data ingestion framework and provide support to write test cases to valid the logic.

•The Enterprise data lake is used to provide support for various uses cases including Analytics, processing, storing and Reporting of voluminous, rapidly changing, structured and unstructured data.

•Processing large sets of structured, semi-structured and unstructured data and supporting systems application architecture.

Environment: Hadoop Big Data, Hue, Hive,Impala, Spark-Sql, Sqoop, AWS S3, AWS EC2,Hbase, Oozie, HiveQL, XML, Log 4j, Redwood, Autosys, JIRA, Perforce, Quick Build, Eclipse, UNIX

Company: Infosys Technologies Ltd -USA

Client: Capital Group, Irvine –CA

Duration: June 2107 to Nov-2017.

Role: Technology Lead - Hadoop.

Client manages the investments of Individuals and large institutions through mutual funds, separately managed accounts and pooled investment funds.

The objective of the project is to manage data of investments and funds.

Responsibilities:

•Developing applications using Hadoop Big Data technologies- Hue, Pig, Hive, Impala,Spark, Map-Reduce, Kafka,Hbase, Spark-sql and Oozie.

•Analyzing and Transformations of Data of Investments and Funds with Hive and Pig.

•Planning and Conducting collaboration sessions and requirements walkthrough with concerned stakeholders (Business SMEs, Architects, Tech Leads, Developers, QA).

•Experience in understanding the data and designing/Implementing the enterprise platforms like Hadoop Data lake.

•Creating HiveQL scripts for creating the tables and populating the data.

•Developing Map reduce jobs using apache commons components.

•Involved in Requirement Analysis, Design, and Development.

•Export and Import data into HDFS, HBase and Hive using Sqoop.

•Worked closely with the business and analytics team in gathering the system requirements.

•Load and transformation of large sets of structured and semi structured data.

Environment: Hadoop- Pig, Hive, Sqoop, Hbase, Oozie, HiveQL, XML, Log 4j, Autosys, JIRA, Perforce, Quick Build, Eclipse, UNIX

Company: Infosys Technologies Ltd -USA

Client: American Family Insurance, Madison WI

Duration: September 2016 to May 2017.

Role: Technology Lead - Hadoop.

American Family insurance provides term, universal and Whole life insurance; Personal and business Auto insurance, Personal umbrella insurance and Home insurance . This requires disparate data sources and high quality volume data loads in Hadoop Clusters.

The objective of the project is to standardize the data quality checks to be performed on the data received from various source systems of Policy, Billing, Claims etc.

Responsibilities:

•Analyzing and Transformations of Data of Auto Insurance and Life Insurance Policies with Hive and Pig.

•Experience in understanding the data and designing/Implementing the enterprise platforms like Hadoop Data lake.

•The Enterprise data lake is used to provide support for various uses cases including Analytics, processing, storing and Reporting of voluminous, rapidly changing, structured and unstructured data.

•Processing of large datasets using Hortonworks Hue Platform.

•Developing applications involving Hadoop Big Data technologies- Hue,Pig,Spark, Hive, Impala, Map-Reduce,Scala, Hbase and Oozie.

•Experience in Talend Big data tool for handling huge volume of data.

•Creating HiveQL and Impala scripts for creating the tables and populating the data.

•Developing Map reduce jobs using apache commons components.

•Supporting code/design analysis, strategy development and project planning.

Environment: Hadoop – Hortonworks, Pig, Hive, Sqoop, Hbase, Oozie, HiveQL, XML,Log 4j,Autosys, Cygwin,Eclipse, UNIX

Company: Cerebra Consulting Inc, Bensalem (PA) - April 2016 to September 2016

Role: Programmer Analyst- Hadoop.

Responsibilities:

•Worked on applications involving Hadoop Big Data technologies- Pig, Hive, Map-Reduce, Hbase, Spark and Oozie

•Worked on importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.

•Analyzing/Transforming data with Hive and Pig.

•Developed Oozie workflow for scheduling the job.

•Developing Java Web Applications using JSP and Servlets, Struts 2.0,Hibernate,Spring, REST Web Services, SOAP and Oracle.

•Configured using Hibernate for the Database Connection and Hibernate Query Language (HQL) to add and retrieve the Information from the Database.

•Created and executed test cases in Junit for unit testing of application.

Environment: Hadoop – Hortonworks,Pig, Hive, Sqoop, Hbase, Oozie, HiveQL, XML,Log 4j,Autosys, Eclipse, UNIX.

Company: TCS (TATA CONSULTANCY SERVICES), MUMBAI, INDIA DEC 2014 to FEB 2016

Client: GE CAPITAL

Project: GE CAPITAL

Role: Hadoop Developer.

TCS is a leader in the global marketplace and among the top 10 technology firms in the world .It operates across various domains in 46 countries and is placed among the ‘Big 4’ most valuable IT services brands worldwide.

Responsibilities:

•Worked on applications involving Hadoop Big Data technologies- Pig, Hive, Map-Reduce, Hbase, Oozie.

•Involved in understanding requirements, functional specifications, designing documentations and testing strategies.

•Developed Map reduce jobs in java using apache commons components.

•Developed UDFs in Java as and when necessary to use in PIG and HIVE queries.

•Provided support to the testing team and supporting UAT functional testing.

•Involved in development, build and testing.

•Coordinated with various stakeholders such as the End Client, DBA Teams, Testing Team and Business Analysts.

•Worked on importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.

•Involved in Unit Testing using Junit.

•Involved in UI designing, Coding, Database Handling.

•Provided support to the testing team and supporting UAT functional testing.

•Analyzed and resolved issues critical production issues (Severity # 1 issues).

Environment: Hadoop - Pig, Hive, Sqoop, Hbase, Oozie, HiveQL Java, JSP, Spring, Hibernate 3.0, Struts framework, XML,Log 4j,Eclipse, UNIX.

Company : MASTEK LTD, MUMBAI, INDIA DEC 2011 to NOV 2014

Client: COUNTRY FINANCIAL, STATE FARM.

Project: COUNTRY FINANCIAL, STATE FARM

Role: Senior Software Engineer.

Mastek operates with employee strength of 3200 people across multiple industry sectors. Some of its major clients include FSA, BT, Lloyds’, Country Financial etc.

Responsibilities:

•Developing Java Web Applications using JSP and Servlets, Struts 2.0, Hibernate, Spring, Rest Web Services, SOAP.

•Worked over the entire Software Development Life Cycle (SDLC) as a part of a team as well as independently.

•Responsible for doing validations and cleansing the data.

•Finding the right joins logics and creates valuable data sets for further data analysis.

•Developed various modules for Invoice Processing, Payment and Refund.

•Deployment and support of Web sphere and JBOSS servers in Putty using UNIX commands.

•Developed Java code with minimal defects.

•Developed and modified SQL code.

•Writing SQL queries to query the database and providing data extracts to users as per request.

•Updated technical documents as per the functionality

•Provided support to the testing team and supporting UAT functional testing.

•Prepared UTP for testing the changes in the system and supporting UAT functional testing.

•Performed knowledge transfer and training to the Application Development Team.

Environment: Java, JSP, Servlet, Spring, Hibernate 3.0, Struts framework, HTML, XML, Log 4j, Eclipse, UNIX.

Contact this candidate