Hadoop admin
Resume:
Viswesh Choppa *******.******@*****.***
PROFILE
More than 15+ years of experience in Information Technology, in different roles Hadoop Admin, Database Admin, Data Architect, Solutions Design and led and participated in the design and development of numerous IT initiatives that encompass Database app develop, server-side development using Big Data Technologies/Greenplum/Oracle/PLSQL/SQL/Unix/Ansible technologies in the Financial, Supply Chain and Banking industry. Most recent efforts have been focused on Building DAS architecture for Big data environment, Hadoop Administration and Performance Tuning Activities.
CORE COMPETENCIES CERTIFICATIONS
Cloudera/Hortonworks (HDP) Hadoop Administration
Hadoop Installation & Configuration
Cassandra Administration
Cluster Performance Tuning.
Data Migration to Hadoop from Greenplum/Teradata
Experience with developing and deploying applications on Cloud Amazon Web Services (AWS)
Develop Data Architectures and Models
Architectural Study, Design and Development
HDPCD Hortonworks Data Platform Certified Administrator Professional
HDPCD Hortonworks Data Platform Certified Developer Professional
OCP Oracle Certified Professional Certification
Sample IT Certifications include: PMP, MCSE, VCP, CISSP
These designations are earned from a professional society – they are not equivalent to certificates for class attendance
If you have no certifications, use this area for additional competencies; and delete the heading above
DO NOT include Security Clearances anywhere on your resume
EXPERIENCE SUMMARY :
Hands-on in setting up Data Lake Environment.
Experience of Deployment, configuration, troubleshooting, Securing Governance of production Hadoop Infrastructure.
Building DAS architecture for big data environment in DCA/non-DCA environments.
Installation, configuration, supporting and managing Cloudera/Hortonworks Hadoop cluster
experience in dealing with Apache Hadoop components like HDFS, MapReduce, HIVE, HBase, PIG, Sqoop, Spark, Kafka oozie, and Big Data and Big Data Analytics
Managing day-to-day activities of Hadoop clusters, Hadoop Cluster monitoring & health check
Hadoop Cluster Security management (Sentry & Kerberos)
Experience in Encryption, Kerberization, SSL/TLS, LDAP and Linux Admin activities
In-depth knowledge and understanding of Hadoop Architecture and HDFS including YARN
Developed UNIX/Python scripts to load the data from Oracle, Greenplum Production to HDP Hadoop. Data load operations
Backup automation, documenting all best practices, root cause analysis.
Data Migration to Hadoop (HDP) from GPDB using GPHDFS, and from Oracle/SQL Server to Hadoop
experience in Hadoop on Network Attached Storage like DELL EMC Isilon
Cassandra/Greenplum admin experience
Automation of Cluster setup with Ansible/python/Unix scripts.
Worked on Data pipeline (ETL tools) development using Informatica
Experience with CI/CD process in particular - GIT, Jenkins, Jira, and Confluence.
Experience in working on production big data solutions and experience in client-driven large-scale data lake projects
General operational expertise such as good troubleshooting skills, understanding of system’s capacity, bottlenecks, basics of memory, CPU, OS, storage, and networks.
PROJECTS:
Role : Big Data/Hadoop Consultant (IaaS )
Cloudera Hadoop Consulting for multiple customers in Healthcare, Telecommunications, Financial Services industries and Retail
Customers: BCBS Blue Cross Blue Shield ( Nov -2020 – Current)
Architect and administer the Cloudera installation and configuration on RHEL 7.4 and with HDFS. Provide customer with advice on best practices for deployment of Hadoop services for their production environment.
Cloudera Hadoop cluster setup and manage- technical expertise.
Installed, Configuration, HA, Troubleshooting and Deployed the CDH, applications on CDH Cluster.
Worked on installing cluster, Commissioning & Decommissioning of Data Node
Experience on managing Hadoop services HDFS, HBase, Hive, Impala, MapReduce, Pig, and Spark, Kafka systems.
Job scheduling, monitoring, debugging, and troubleshooting.
Ansible Script / Python/ Unix script development for the cluster Pre/Post Installation activities, maintenance, and Hadoop admin job automation, DevOPS (IaaS)
Monitored cluster for performance and, networking and data integrity issues.
Responsible for troubleshooting issues in the execution of MapReduce/Spark jobs
Installed/Configured/Maintained Apache Hadoop clusters for application development and Hadoop tools like
Hive, Pig, HBase, Spark,Nifi,impala,kafka and Sqoop.
Setup of Apache Ranger for Data Governance and Auditing.
Developing Data Pipelines to extract and Load data into the Bigdata Cluster using Python, Unix scripting
Ansible Script development for the cluster maintenance, and admin job automation like Hive, Hbase, Kafka with Control-M interface
Experience in Encryption, Kerberization, TLS/SSL, LDAP and Linux Admin
ETL tools Apache NiFi, Ab-initio setup and troubleshoot jobs and maintenance
Experience with AWS (EC2, S3, EMR)
Senior Advisor, Database Administrator @ Dell technologies
Customer: Dell Technologies, Round Rock, TX (Aug 2018 – Sep 2020 )
Architect and administer the Cloudera Hadoop cluster, installation and configuration on RHEL 7.5 and with HDFS. Provide customer with advice on best practices for deployment of Hadoop services for their production environment.
Cloudera Hadoop cluster setup and manage- technical expertise.
Installed, Configuration, HA, Troubleshooting and Deployed the CDH, applications on CDH Cluster.
Worked on installing cluster, Commissioning & Decommissioning of Data Node
Experience on managing Hadoop services HDFS, HBase, Hive, Impala, MapReduce, Pig, and Spark, Kafka systems.
Job scheduling, monitoring, debugging, and troubleshooting.
Secured the Cloudera Hadoop Cluster using Active Directory Kerberos KDC
Upgrade CDH from 5.15 to 6.3.3
Configured Sentry to Authorize Hadoop users.
Monitored cluster for performance and, networking issues.
Data Migrations/transfer between Hadoop clusters using HBase export snapshots, distcp.
Responsible for Debug/Troubleshooting environment failures or job failures
Elastic Search/Kibana Cluster setup and maintenance.
Ansible Script development for the cluster Pre/Post Installation activities, maintenance, and admin job automation.
ETL tools Streamsets/Apache NiFi setup and troubleshoot jobs and maintenance.
Experience with CI/CD process in particular - GIT, Jenkins, Jira, and Confluence.
Experience working with AWS Cloud Infrastructure (EC2, S3,EMR)
Experience in Encryption, Kerberization, SSL, LDAP and Linux Admin
Role : Hadoop Administrator
Hortonworks Hadoop Consulting for multiple customers in Healthcare, Telecommunications, Financial Services industries and Retail
Customers: SSTECH, Mitre, DAI inc ( Feb -2018 – Aug 2018)
Architect and administer the Hortonworks HDP 2.6.2 installation and configuration on RHEL 7.4 and with HDFS. Provide customer with advice on best practices for deployment of Hadoop services for their production environment.
Ambari Manager setup and manage- technical expertise.
Installed and Deployed the Spark & Apache NiFi application on Cluster.
Knowledge or experience with AWS (EC2, S3,EMR)
Setup Nifi for Data loading into Hadoop and involved in developing data mappings.
Worked on installing cluster, commissioning & decommissioning of Data Nodes, Name Node recovery,
Installed, configured, and administered huge Hadoop clusters.
Data Migrations/transfer between Hadoop clusters using HBase export snapshots
Monitored cluster for performance and, networking and data integrity issues.
Responsible for troubleshooting issues in the execution of MapReduce jobs by inspecting log files.
Installed/Configured/Maintained Apache Hadoop clusters for application development and Hadoop tools like
Hive, Pig, HBase, Zookeeper and Sqoop.
Ansible Script development for the cluster Pre/Post Installation activities, maintenance, and admin job automation.
ETL tools Streamsets/Apache NiFi setup and troubleshoot jobs and maintenance
Experience with CI/CD process in particular - GIT, Jenkins, Jira, and Confluence.
Responsible for developing data pipeline using flume, Sqoop and pig to extract the data from
Weblogs and store in HDFS.
Sr Consultant, Professional Services @ DellEMC ( Feb-2015 to 02/02/2018 )
Role : Hadoop Administrator
Hortonworks Hadoop Consulting for multiple customers in Healthcare, Telecommunications, Financial Services industries and Retail
Customer: LA County, CA
Architect and administer the Hortonworks HDP 2.6.2 installation and configuration on RHEL 7.2 and with HDFS. Provide customer with advice on best practices for deployment of Hadoop services for their production environment.
HDFS technical expertise
Ambari Manager setup and manage- technical expertise
Setup of Apache Ranger and Atlas for Data Governance and Auditing.
Integrated Hadoop with AD/KDC server (kerberizing the cluster).
Installed and Deployed the Spark & R application on Cluster.
Setup Apache Nifi/ETL for Data loading into Hadoop and involved in developing data mappings
Post-Production successfully mentored and transitioned the operating model for continued maintenance.
Worked on importing and exporting data from Oracle and DB2 into HDFS and HIVE using Sqoop.
Worked on installing cluster, commissioning & decommissioning of Data Nodes, Name Node recovery,
Installed, configured, and administered a small Hadoop clusters consisting of 10 nodes.
Data Migrations/transfer from Oracle/SQL server to Hadoop Cluster.
Experience with CI/CD process in particular - GIT, Jenkins, Jira, and Confluence.
Monitored cluster for performance and, networking and data integrity issues.
Responsible for troubleshooting issues in the execution of MapReduce jobs by inspecting and reviewing logfiles.
Installed/Configured/Maintained Apache Hadoop clusters for application development and Hadoop tools like
Hive, Pig, HBase, Zookeeper and Sqoop.
Responsible for developing data pipeline(ETL) using flume, Sqoop and pig to extract the data from
weblogs and store in HDFS.
Experience as Linux Admin
Customer: Walmart, Bentonville, AR
Role : Big Data Admin Consultant/Greenplum DBA
Administered and Managed the Bigdata environment
Installing, managing and administering the overall Hadoop HDP infrastructure.
Implemented the Kerberized Security model for HDP cluster.
Involved in migration of ETL processes from Oracle to Hive to test the easy data manipulation.
Install Hadoop updates, patches, and version upgrades as required.
Managed log files, backups and capacity.
Found and troubleshot Hadoop errors
Created Ambari Views for Tez, Hive and HDFS.
Architecture and designed Hadoop 24 nodes Innovation Cluster with SPARK, HDP 2.6.2.
Teradata Resource Management Experience
Performance tuning and optimization to handle and process large volume of data in Teradata
Managed 350+ Nodes HDP 2.2.4 cluster with 4 petabytes of data using Ambari 2.0 and Linux Cent OS 6.5 on DCA
Complete end to end design and development of Apache Nifi flow
Design Data Lake, Data reservoir solutions based on Hadoop & Greenplum for client BI Analytics needs.
Data Migration to Hadoop (HDP) from GPDB using GPHDFS.
Used Ansible in deploying, configuring, managing, maintaining, a server machine.
Experience with CI/CD process in particular - GIT, Jenkins, Jira, and Confluence.
Experience as Linux Admin
Sr.Consultant @ Cyber Resource Group LLC ( Oct-2013 to Jan/2015 )
Hadoop Consultant
Customer : The Depository Trust & Clearing Corporation ( DTCC)
Role : Greenplum / Hadoop Administrator
Technical lead and architect for a Finance company’s various projects.
Greenplum / Oracle Database Administration & Performance Tuning on MPP Data Computing Appliance (DCA)
Designing Greenplum Database and application development
Greenplum Upgrade & Configuration
Data Migration to Hadoop (HDP) from GPDB using GPHDFS.
Involved in Application Performance Testing and SQL and Database Tunings
Involved in setting up and implementing database
Managing users, roles and privileges
Configure and monitor security, and resource queues.
Install Hadoop updates, patches, and version upgrades as required.
Perform routine cluster maintenance, such as provisioning new nodes and performing HDFS backups and restores.
Develop scripts and tools to automate common Hadoop administration tasks.
Participate in an on-call rotation with 24x7 availability requirements.
Configure and monitor security, and resource queues.
Install Hadoop updates, patches, and version upgrades as required.
Perform routine cluster maintenance, such as provisioning new nodes and performing HDFS backups and restores.
Develop scripts and tools to automate common Hadoop administration tasks.
Oracle Golden gate and ETL tools like Informatica Data Migration, Troubleshooting work experience.
Admin work automation using Ansible/Python/Unix scripts.
Participate in an on-call rotation with 24x7 availability requirements.
Performance tuning of SQL/Stored procedures on Greenplum
Database Administrator ( DBA), Consultancy Services @ TCS ( Feb-2004 to Oct 2013 )
Role : Database Admin/Architect
Customer : Moody’s Corp
Successfully implemented the projects of a leading Finance company .
Administering and Maintaining Cloudera Hadoop Clusters
Management and support of Hadoop Services including HDFS, Hive, Impala, and SPARK. Primarily using Cloudera Manager
Red Hat Enterprise Linux Operating System support including administration
Performance tuning of Hadoop clusters and Hadoop MapReduce routines
Screen Hadoop cluster job performances and capacity planning
Monitor Hadoop cluster connectivity and security
File system management and monitoring
HDFS support and maintenance
Diligently teaming with the infrastructure, network, database, application and business intelligence teams to guarantee high data quality and availability
Collaborating with application teams to perform Hadoop updates, patches, version upgrades when required
Role : Database Admin
Client : TCS Banks, Bangalore
Successfully developed and implemented the Finance product-based company for Java/J2EE applications.
Installed configured and maintained Oracle 10g /9i on Sun Solaris and upgraded to 9i to 10g.
Installed and configured of Oracle 10g/9i RAC (Real Application Clusters) on Redhat Linux. Add new nodes to RAC 10g Cluster to distribute the load.
Performed Oracle Clusterware installation, Configuration and monitor Oracle Clusterware resources.
Provided performance tuning on a 10G RAC database on a Linux platform.
Created and Managed Services for RAC DB Load balancing.
Setup and supported HA solutions Standby servers, Data Guard Setup, Configured Oracle Data Guard using physical standby database.
Creating, maintaining and refresh materialized views to maintain replication environment and resolved out-of-sync issues
Performed DB performance monitoring, generated Stats pack reports to observe overall DB performance and top wait events, and SQL optimization/tuning using Explain Plan.
Created shell scripts to refresh database schema using Export/Import utility and Clone/Refresh of Production databases to Testing environment.
Implemented RMAN backup/recovery procedures using shell scripts and involved in Perl scripting.
Responsible for maintaining application schema, schema changes.
Role : Database Administrator
Successfully implemented the projects of a leading Product based company for Java/J2EE applications.
Provided Primary DBA support for Bea Applications Development and Administration
Provided input to the project and enhancement teams to define database storage structures (tablespaces) after application developers have designed an application
Created primary objects (tables, views, indexes, snapshots, database links) once application developers have designed an application
Modified the database structure as necessary, from information given by application developers
Took database backups and recovery using Import/Export, Data Pump tools
Monitored and optimized the performance of the database and maintained archived data
Supported Dev teams for the application development process
Tuned SQL, I/O, Application and Database
Reviewed Data Fragmentation and Performance Tuning on Oracle and DB2
Created Development/Test Instances
Used SQL/PLSQL and UNIX Shell Scripting for monitoring space and extent growth
Managed and supported SQL Server instances/databases using SQL Server 2000/2005
Installed and maintained oracle database servers and client’s different versions 8.1.7, 9i R2, 10g R1/R2 on platforms Red hat Linux, Windows servers and different Unix platforms
Installed DB2UDB 9.1 Fix Pack 3 / 8.2 FP2, SYBASE, MSSQL server 2000 on AIX, HPUX and windows 2000 servers
Took backup of database and restored test data for testing purpose.
Solved Day to day issues related to all the databases such as ORACLE, DB2, SYBASE and MSSQL
Created the databases and users for testing purpose with right privileges
TECHNICAL SKILLS SUMMARY
Hardware
EMC DCA v2/v3
Languages & Development Tools
SQL, PL/SQL, Shell Scripting, Python
Operating Systems, Utilities & Virtualization Tools
Red Hat Enterprise Linux 64-bit 5.x and 6.x and Windows 95/98/NT/2000/XP
Database Management Systems
Oracle, DB2, Sybase, Greenplum 4.3.5.1,
Big Data / Hadoop technologies
HDFS, Pig and Hive programming, HDFS Hadoop HDP Hortonworks Administration, Ambari
Other Tools
Greenplum Control center, Pgadmin 3, OEM, SQL*Loader, EXP, IMP, expdp, impdp, TOAD, DBCA and RapidSQL, EXPLAIN PLAN, STATSPACK/AWR
Database Modelling
ER Studio 8.5
EDUCATION
Andhra University Engineering College, Visakhapatnam, India – M.Tech
Sri Krishnadevaraya University, Anantapur India – M.Sc.
Contact this candidate