Data Developer

srija soma

Hadoop Developer

Mobile: 425-***-****

Email: *****.******@*****.***

Summary:

6+years of extensive Professional IT experience, including 4+ years of Hadoop /Big data experience, 2years of Java Experience, capable of processing large sets of structured, semi-structured and unstructured data and supporting systems application architecture.

Expert in importing and exporting of data using Sqoop from HDFS to relational database system and vice-versa.

Expert level of scripting using Pig scripts and Hive queries for processing and analyzing large volume of data.

Experience with Oozie Workflow Engine in running workflow designing, job scheduling with actions that run Hadoop Map/Reduce and Pig jobs.

Good Experience in developing and implementing big data solutions and data mining applications on Hadoopusing Hive, PIG, Hbase, Hue, Oozie workflows and designing and implementing Java, Map Reduce programs.

Good knowledge on Hadoopcluster administration, Monitoring and managing Hadoop cluster using cloudera manager.

Knowledge in installing, configuring, and using Hadoopecosystem components like Hadoop Map Reduce, HDFS, HBase, Oozie, Hive, Sqoop, Pig, Flume, Apache Spark, Zookeeper and Kafka.

Experience in managing and reviewing Hadooplog files.

Hands on experience in converting Hive/SQL queries into Spark transformations using Scala.

Good Experience with flume tool for data ingestion from various data producers (webservers) into Hadoop.

Good knowledge in NOSQL databases HBASE, MongoDB.

Sound Relational Database Concepts and extensively worked with ORACLE, MySQL, SQL Server.

Good Experience with databases, writing complex queries and stored procedures using SQL and PL/SQL.

Experience in using sequence file, RC file and Avro file formats.

Good understanding of Classic Hadoopand Yarn architecture along with various HadoopDemons such as Job Tracker, Task Tracker, Name Node, Data Node, Secondary Name Node, Resource Manager, Node Manager, Application Master and Containers.

Very good experience with both MapReduce 1 (Job Tracker) and MapReduce 2 (YARN) setups.

Expert in Java MapReduce Jobs, User Defined functions for Pig and Hive.

Knowledge in handling messaging services using Apache Kafka.

Familiarity on real time streaming data with Spark for fast large scale in memory Map Reduce.

Experience in Business Intelligence tools such as Tableaufor visually analyzing the data.

Experience in building, maintaining multiple Hadoopclusters of different sizes and configuration and setting up the rack topology for large clusters.

Developedmachine learning algorithms using Mahout for clustering and data mining.

Involvement in all phases of SDLC from project proposal, planning, analysis, development, testing, deployment and support.

Experience in developing and implementing web applications using Java, JSP, CSS, HTML, HTML5, XHTML and Java script, JSON, XML, JDBC.

Experience in working in 24X7 Support and used to meet deadlines, adaptable to ever changing priorities.

Proven ability to work with senior technical managers and staff to provide expert-level support for the installation, maintenance, upgrading, and administration of full-featured database management systems.

Excellent interpersonal and communication skills, creative, research-minded, technically competent, result-oriented with problem solving as well and ability to work well with people and to maintain a good relationship with the organization.

Technical Skills:

Technology Tools

Big data and Hadoop, Apache/Cloudera HDFS 1.X/2.X, MapReduce, YARN, Sqoop, Flume, Spark, Scala, Hive, Pig, Oozie, Zookeeper, Kafka

Operating Systems

MS Windows, Linux, Ubuntu, CentOS

Programming Languages

C, C++, Java, SQL, PL/SQL,JavaScript,Unix Shell Scripting

Database

Oracle, MySQL, Microsoft Sql Server

NoSQL Databases: Hbase, MongoDB, Cassandra.

IDE Tools

Eclipse, Net Beans

Other Skills

Tableau,HTML5, Java Script, JSON, CSS XML, Apache Tomcat

Education:

Bachelor of technology in Computer Science from Jawaharlal Nehru Technological University, Hyderabad, India.

Certification: Certified Java Developer

Professional Experience:

Bed bath & Beyond, Union, NJ Oct 2017 – Mar 2018

Big Data/Hadoop Developer

Responsibilities:

Installed and Configured Apache Hadoopclusters for application development and Hadooptools like Hive, Pig, Oozie, Zookeeper, Hbase, Flume and Sqoop.

My most recent project I have worked on is for datalakes migration,the client started a new initiative to migrate all its data that it used for data analytics& reporting to migrate and store raw data in hadoop HDFS.

Implemented multiple Map Reduce Jobs in java for data cleaning and pre-processing.

Worked in a team with 40 node cluster and increase cluster by adding Nodes, the configuration for additional data nodes was done by Commissioning process in Hadoop.

Responsible for Cluster maintenance, adding and removing cluster nodes, Cluster Monitoring and troubleshooting, manage and review data backups and log files.

Responsible to manage data coming from different sources.

Managed and scheduled Jobs on a Hadoopcluster.

Implemented a script to transmit information from Oracle to Hbase using Sqoop.

Involved in defining job flows, managing and reviewing log files.

Installed Oozie workflow engine to run multiple Map Reduce, HiveQL and Pig jobs.

Participated in requirement gathering form the Experts and Business Partners and converting the requirements into technical specifications.

Created Hive tables to store the processed results in a tabular format.

Was done various compressions and file formats like snappy, Gzip, Avro, Sequence, text.

Wrote complex Hive queries and UDFs in Java and Python.

Created and exposed Hive views through Impala for the business Users

Involved in forecast based on the present results and insights derived from data analysis.

Involved in collecting the data and identifying data patterns to build trained model using Machine Learning.

Prepare Developer (Unit) Test cases and execute developer testing.

Implemented test scripts to support test driven development and continuous integration.

Developed and implemented some machine learning algorithms using Mahout for data mining for the data stored in HDFS

Created and maintained Technical documentation for launching HadoopClusters and for executing Hive queries and Pig Scripts.

Worked on visualization tool tableau for visually analyzing the data.

Environment: Hadoop, HDFS, Pig, Hive, Map Reduce, Impala, Sqoop, Flume, Oozie, Big Data, java, Python, Mahout, Junit testing, Oracle, MySQL, Tableau, LINUX, Windows.

American Airlines, Dallas, TX Jun 2016– Sep 2017

Java/Hadoop Developer

Responsibilities:

Worked on data migration from existing data sources to Hadoopfile system.

Interact with the Business Analysts to get the requirements and formulate them into system use cases.

Understand customer business use cases and be able to translate them to analytical data applications and models to implement a solution.

Created flow charts, sequence diagrams, schemas, data model of underlying system, pseudocode and class diagrams using Microsoft Vision.

Ability to create Hive Avro Tables to read and write the data with the appropriate compressions.

Created custom Database Encryption & Decryption UDF that could be plugged in while Ingesting data to External Hive Tables for maintaining security at table or column level.

Worked on No-SqlHbase database for processing very large semi structured and structured tables by defining column families, region servers etc.

ImplementedSpark SQL for faster testing and processing of data.

Knowledge on handling Hive queries using Spark SQL that integrate with Spark environment.

Worked on migrating MapReduce programs into Spark transformations using Spark.

Configured Spark Streaming to receive real time data and store the stream data to HDFS.

Developed map-reduce programs for different patterns of data on Hadoopcluster.

Developed java map reduce programs using core concepts like OOPS, Multithreading, Collections and IO. Compiled and built the application using MAVEN and used SVN as version control system.

Created data ingestion plans for loading the data from external sources using Sqoop.

Used Python for pattern matching in build logs to format errors and warningsZF-= j• Created flow charts, sequence diagrams, schemas, data model of underlying system, pseudo code and class diagrams using Microsoft Visio.

Environment: Hadoop, HDFS, Pig, Hive, Map Reduce, Sqoop, Oozie, Spark, Big Data, Hbase, java, Python,Unix Shell Scripting, Oracle 11g/10g, MySQL, MS Visio, LINUX, Windows.

Netwin Info Pvt. Ltd, India Jan 2012 – May 2016

Java Developer

Responsibilities:

Managed connectivity using JDBC for querying/inserting & data management including triggers and stored procedures.

Developed UI using HTML, JavaScript and JSP developed Business Logic and Interfacing components using Business Objects, XML.

Communicated with client to Analyze and Review of Business/Technical Requirements

Designed and developed Co-Branding framework to change UI for different clients using same application.

Used Struts, Spring (MVC) Framework to develop the application.

Used Factory, DAO, Singleton and DTO and Value Object, Business Delegate design patterns.

Designed and developed database layer using ORM technologies like Hibernate.

Developed Web services client to consume with Vender Web services

Generated Excel reports Using POI framework

Implementing the business logic and creating dynamic web pages with JSP and using JavaScript to incorporate client side and server-side validations and functionality.

Preparation and Review of Unit Test Plans, Scripts and Results

Used respective designed patterns to implement the reusable component of the application.

Implemented various GUI screens using JSP, AJAX Frame work and JQuery.

Worked with web methods for deploying and administering services.

Used Hibernate to interact with database.

Worked with requirement analysis team to gather software requirements for application development.

Provided support and maintenance after deploying the web application.

Resolved issues reported by the client.

Environment: Linux, Windows, Core Java, JSP, JavaScript Hibernate, Servlets, Spring MVC Framework, Hibernate, Oracle, JMS, Ajax, JQuery, XML, Log4j, Apache Tomcat and Eclipse.

Contact this candidate