Hadoop Developer

Atul Kumar

**** * ******** **. *** A* MT. PLEASANT, MI 48858 • 989-***-****

*******@*****.*** • https://www.linkedin.com/in/adwiv/

E D U C A T I O N

CENTRAL MICHIGAN UNIVERSITY Michigan, USA Master of Science in Computer Science - GPA: 3.50/4.00 Aug 2015 – Dec 2016

Graduate coursework: Big Data Analytics, Cloud Computing, Artificial Intelligence, Applied Data Engineering, Pattern Recognition and Data Mining, Application Development, Software Data Modeling and Databases.

RAJIV GANDHI PRODYOGIKI VISHWAVIDYALYA Bhopal, India

Bachelor of Engineering in Computer & Science -GPA: 3.6/4.00 Jul 2011 - May 2015

Undergraduate coursework: Compiler Design, Information Storage & Management, Distributed System, Computer Networking,

Microprocessor & Interfacing, Operating System, Computer Graphics & Multimedia.

S K I L L S

HADOOP ECOSYSTEM - HDFS, MapReduce, Pig, Yarn, HBase, Flume, Sqoop, Oozie, Cassandra, Zookeeper, SparkSQL, Spark Streaming, PySpark, MLLib, HIVEQL, Kafka, Nifi, Elastic Search, Kibana, Hortonworks Ambari, Amazon Web Services (AWS), cloudera Manager, hue.

WEB TECHNOLOGIES - HTML, CSS, XML.

NOSQL DATABASES - HBase, Cassandra.

METHDOLOGIES - Agile/Scrum, Waterfall.

DATABASES - MySQL, SQL Server, Oracle, Netezza.

PROGRAMMING LANGUAGES - Core Java, Python, Unix Shell, Scala.

OPERATING SYSTEM - Windows XP/Vista/7/8.x/10, Ubuntu, fedora, Debian, Red Hat Linux, CentOS, Mac.

MONITORING TOOLS - Nagios, Custom Shell scripts, Cloudera Manager.

OTHER TOOLS - Eclipse, GitHub, Latex, Weka, MS Excel, putty, WordPress.

P R O F E S S I O N A L E X P E R I E N C E

Brainyway Solutions Inc.

Hadoop Developer

Hyderabad, INDIA

Jan 2014 – Jun 2015

•Hands-on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop MapReduce, HDFS, YARN, HBase, Hive, Sqoop, Pig, Zookeeper, Oozie, Cassandra and Flume.

•In-depth knowledge of Hadoop architecture and its components like HDFS, Name Node, Data Node, Resource Manager, Yarn and MapReduce programming paradigm.

• Worked with data architecture including pipeline design of data ingestion, Architecture information of Hadoop, data modeling, machine learning and advanced data processing.

•analyzed data using Hadoop Ecosystem including HDFS, Hive, Spark, Spark Streaming, MLLib, Nifi, Elastic Search, Kibana, Kafka, HBase, Zookeeper, PIG, Sqoop, Flume.

•importing and exporting data using spark JDBC and Sqoop for Oracle, Netezza, DB2 and MS SQL databases.

•developed Spark Streaming applications. Getting the data using Nifi, writing the stream data into Kafka and analyzing the data through Spark.

•troubleshooting Spark/Map Reduce jobs.

•setting up clusters on Horton Works distributions of Hadoop.

• ETL: Data extraction, managing, aggressions and loading into HBase.

•Developed different kind of custom filters and handled pre-defined filters on HBase data using API.

• Handled data from different data sets, join them and preprocess using Pig join operations.

•Used Sqoop to import the data from RDBMS to Hadoop Distributed File System (HDFS) and later analyzed the imported data using Scala, and PySpark.

•Automated common maintenance and installation mechanisms.

•handled Avro data files by passing schema into HDFS using Avro tools and Map Reduce.

•Installed Oozie workflow engine to run multiple Hive and Pig jobs.

•Wrote python scripts to process semi-structured data in formats like JSON.

•Developed User Defined Functions (UDF) in the scripts to achieve the specific functionality defined by the business use cases.

•Responsible for analyzing and data cleaning using Spark SQL, Scala Queries.

R E L E V A N T P R O J E C T S

Running Hadoop on a Raspberry

Michigan, USA

Aug - Dec 2016

•Installing Hadoop alongside various utilities on multiple Raspberry Pi 3s.

•Implemented Map Reduce jobs on Hadoop Cluster

•Configured environmental variables in bash.bashrc.

•The implementation would result in a scalable, portable, and affordable solution for big data analysis.

Sentiment Analysis on Amazon Reviews Michigan, USA

Aug -Dec 2016

•Used scoring function and classification and compared the results to show the accuracy.

•Designed a scoring function to calculate score taking just the text as input.

•Used AFFIN word list is a sentiment dictionary for positive and negative words

•Used Naïve Bayes Algorithm for the classification

Big Data Analytics on Wiki Data

Michigan, USA

Jun - July 2017

•Analyzed wiki review data which was in XML format.

•Imported the data into hdfs using flume

•Filtered the data using Spark/Scala.

•Generate the reports using Hive.

Sentiment Analysis on Twitter and Wiki Data

Michigan, USA

Jan - Apr 2016

•In this project, we propose to calculate the sentiment of a proper noun by using twitter and Wikipedia data for training.

•Used Wikipedia Api to web crawling the data from Wikipedia page.

•Used Twitter Api to get the tweets from twitter.

•This Project is developed using Python, Twitter API, Beautiful soup, Tweedy, Text Blob

Image Steganography

Bhopal, India

Sep - Dec 2015

•To make our data secure we have designed an application using Stenography technique.

•An Image Steganography is used to hide the data in an image.

•Used LSB (least significant bit) to hide the data in images, audio, video.

I N T E R N A T I O N A L C O N F E R E N C E P A P E R S

Dwivedi A.K., Kaiya K., Botti P. (2016) Enigma of User Privacy in Android. In Smart Trends in Information Technology and Computer Communications. SmartCom.

Patel, Y.S., Vyas, S., & Dwivedi, A. K. (2015). An expert system based novel framework to detect and solve the problems in home appliances by using wireless sensors. In 2015 international conference on futuristic trends on computational analysis and knowledge management.

Contact this candidate