Resume

Sign in

big data consultant

Location:
Gurgaon, Haryana, India
Posted:
November 05, 2018

Contact this candidate

Resume:

Shubham Purwar

Big Data Consultant

Email: ac7ly3@r.postjobfree.com

Phone: +91-920*******

SKILLS

A passionate Implementation and Support Engineer with 2.6 years of experience being a Hadoop Administrator as well as a Hadoop Developer along with knowledge of Apache Spark. EXPERIENCE

Bharti Airtel, Gurgaon- Big Data Engineer (April 2018- Current) Xebia IT Architects Pvt Ltd- Big Data Consultant (Jan 2016- March 2018) JANUARY 2016 – PRESENT

Languages: Java, Python

Cloud Platform: Microsoft Azure Virtual Machines, Virtual Networks, Storage, Automation

DevOps: Docker, Ansible

Hadoop Ecosystem: Apache Spark, HDFS, MapReduce, YARN, Hive, Impala, Sqoop, Flume, Kafka, Apache Sentry, Apache Zookeeper, Pig, Cloudera Manager, Cloudera Director, Cloudera Navigator, Cloudera Management Service, Key Trustee Servers, HDFS at REST Encryption, Apache Storm, Apache NiFi, Grafana, InfluxDB, Telegraf

Linux: Linux Administration, Active Directory Integration, DNS Server, Kerberos

Working Methodology: Agile

Databases: MySQL, MongoDB

PROJECT SUMMARY

Project #1 (Xebia) Jan 2016- March 2018

Domain: Airline

Tools & Technologies: Microsoft Azure, CDH, Cloudera Manager, Apache Sqoop, Apache Spark, Hive, Kerberos, Linux, Sentry, Flume, Cloudera Impala, Zookeeper Description: Deployed a Cloudera Hadoop Cluster upon Microsoft Azure, being world’s first roduction implementation of CDH on Azure. Post deployment role was to administer and manage the cluster. Implementation Responsibilities:

Cluster design and pre-implementation considerations.

Deployed the infrastructure layer on Azure.

IPsec connectivity between On-premise and VNET

Installed CDH and the required services on Azure.

Integrated Jupyter Notebook with Apache Spark

Enabled Kerberos to work with Cloudera frameworks

Implemented the required configurations and the cluster backup strategy.

Managed the overall cluster health and performance. Support Responsibilities:

8 * 5 support for a distributed 10 node cluster.

Supports included managing issues at Cloud, Linux and Hadoop Level

Handled production issues primarily with Spark

Other support frameworks included Hive, Sqoop, YARN, Impala, HDFS

Daily job activities included cluster monitoring, configuration management, troubleshooting Project #2 (Xebia) October 2016- February 2017

Domain: Energy

Tools & Technologies: Cloudera Manager, Microsoft Azure, CDH, Sqoop, Spark, Hive, Kerberos, Linux, Cloudera Navigator, Key Trustee Servers, HDFS at REST Encryption, Cloudera Impala, Ansible Description: Deployed a Hadoop Cluster upon Microsoft Azure using Cloudera Director. Post deployment role is to administer and manage the cluster. Implementation Responsibilities:

Cluster Design

IAAS on Azure

Installed Cloudera Director, Cloudera Manager, CDH and the required services on Microsoft Azure

Ansible was used as a provisioning and configuration management tool

Configured Kerberos with Active Directory as KDC

Setup Cloudera Navigator and Key Trustee Servers

Configured HDFS and REST Encryption

Services configurations and high availability

Configured JupyterHub for multiuser login to work with Spark

Manage and monitor the health and performance of the cluster Support Responsibilities:

8 * 5 support for a distributed 15 node cluster.

Supports included managing issues at Cloud, Linux and Hadoop Level

Provided assistance in performance enhancement primarily in PySpark

Configuration management using Ansible

On demand cluster scaling using Cloudera Director

Other support frameworks included Hive, Sqoop, YARN, Impala, HDFS, Navigator, Key Trustee Servers

Daily job activities included cluster monitoring, configuration management, troubleshooting Project #3 (Bharti Airtel) April 2018 – Current

Domain: Telecommunication

Tools & Technologies: Hortonworks, Apache Sqoop, Apache Spark, Hive, Kerberos, Linux, Kafka, NiFi,Zookeeper, Grafana, InfluxDB, Telegraf

Project Description: Deployed a Hadoop Cluster on 120 nodes. Post deployment role is to administer and manage the cluster.

Responsibilities:

Cluster Design

Installed Hortonworks distribution and the required services.

Configured Kerberos with Active Directory as KDC

Services configurations and high availability

Configured JupyterHub for multiuser login to work with Spark

Manage and monitor the health and performance of the cluster

Set up NiFi cluster and import the workflow.

EDUCATION

University of Petroleum and Energy Studies, Dehradun - B.Tech (2012-2016) Bachelor Degree in Computer Science with Specialization in Telecom Informatics Technology sponsored by IBM

CGPA: 3.18/4. (83.6 %)

I hereby declare that I am ready to relocate to anywhere across India and abroad. Date: (Shubham Purwar)



Contact this candidate