Senior Hadoop Administrator and Cloud Architect

Koushik Reddy

Sr. Hadoop Administrator

Email: *****.***@*****.***

Phone: 313-***-****

Professional Summary:

Having 9+ years of IT experience in Maintaining, deploying, configuring and managing large-scale MapR Hadoop clusters on both (Cloudera Manager and Ambari Hortonworks) platforms such as (DEV, IT, PROD and DR) environments for the Automative & Banking Industries through Linux and Kubernetes Operation systems.

Great knowledge and experience on Cloud Platforms such as (Azure Databricks, Data Lake, Snowflake, AWS EMR and Cloudera Public Cloud).

I’m also Certified Cloudera, Databricks and AWS Solutions Architect Professional.

Skill Set: HDFS, MapReduce, Impala, Hive Metastore, HiveServer2, Yarn, Hive, Spark, HBase, Impala, Zookeeper, Pig, Cloudera Management, Kafka, Nifi, Kerberos, Airflow, Sqoop, Ranger, Pyspark, Sentry, Oozie, Grafana, Splunk, AWS EMR, EC2, RDS, Lambda, Route53, Airflow, IAM, CloudWatch, Batch job, Terraform, S3, CloudFormation, Java, Python, Delta lake, Ansible.

Technical Skills:

Big Data Ecosystem: AWS EMR, Ambari, Cloudera Management, Hadoop MapReduce, Airflow, Batch Job, HiveServer2, HiveMetastore, Yarn, HDFS, Hadoop MapReduce, Zookeeper, Airflow, HBase, Spark, Sqoop, Oozie, Kafka, Nifi, Ranger, Ranger, Lambda, Route 53, Terraform, HDFS, S3, EC2, EKS, IAM, CloudFormation, Databricks, Snowflake, and Sentry.

Database: MySQL, PostgreSQL, RDS.

Scripting Languages: Bash, JavaScript, Python, shell scripting.

Operating Systems: Rhel, CentOS, OpenShift and Kubernetes.

Platform Management: Cloudera Manager and Hortonworks Ambari.

Monitoring Tool: MCS, CloudWatch, Unravel, Grafana, Splunk, Icinga.

Cloud Computing: AWS EMR, Cloudera Public, Databricks.

AI Tools: Copilot, Gemini.

Certified: Databricks, Cloudera and AWS Solutions Architect Professional.

Education:

Bachelor’s in information technology – GITAM University, INDIA (2009-2013)

Master of Science in Information Systems – Stratford University, Washington DC, USA (2014-2016)

Professional Experience:

U.S. Bank, Minneapolis, MN Aug 2023 – Dec 2025

Role: Sr. Hadoop MapR Administrator

Responsibilities

Hands-on experience in Managing, configuring, Monitoring, patches and upgrading Hadoop MapR clusters on Cloudera Manager (CDP) and Hortonworks Ambari (HDP) Platforms.

Great knowledge on Cloud Platforms such as (Databricks, Data Lake, Snowflake, AWS EMR, Cloudera Public Cloud, EC2, EKS, IAM, S3, Terraform).

Hands on experience integrating Hadoop with third-party tools like Informatica tools, Jupiter notebook, Terraform, IAM, S3, CloudWatch, Grafana and Unravel.

Hands on Experience in multiple domains which includes cloud computing, security, identity and access management through Kerberos, LDAP, Sentry and Ranger.

Hands-on experience in Managing on-prem Cloudera (CDP) and Ambari Hortonworks (HDP) components such as (HDFS, Mapreduce, Yarn, Hiveserver2, HiveMetastore, Spark, HBase, Impala, Zookeeper, Pig, Cloudera Management, Kafka, Nifi, Airflow, Sqoop, Ranger, Sentry, Pyspark and Oozie).

Hands-on experience in taking and upgrading Database such as (MariaDB, PostgreSQL).

Hands on experience in Code (IaC) using Terraform, Ansible, and CloudFormation for repeatable and auditable deployments.

Experience in Monitoring Hadoop clusters by using (Airflow, Unravel, Grafana, Splunk and CloudWatch).

Experience in performing backup and disaster recovery of Name Node/MapR snapshots and failover configurations on Hadoop cluster from backup recovery.

Having Strong knowledge on MapR-DB, MapR-FS image, MapR Snapshots etc.

Strong knowledge on Hadoop, Spark, Hive and Map-Reduce framework.

Good understanding of Hive, Spark, MapReduce jobs and optimizing performance of memory management.

Hands-on Experience in managing Yarn Queues, memory management, queue allocation, and distribution experience in Hadoop MapR Production environments.

Experienced with MapR Jobs improving the performance and optimization of the existing algorithms Hadoop in using Spark, MapReduce and YARN.

Hands on experiencing in Managing and enabling security features like (SSL, AD, ACLS, Kerberos, Ranger and Sentry) for user role authentication.

Hands-on experience in developing Automation (CI/CID) pipelines using Ansible, Terraform and CloudFormation to automate data pipeline deployment, testing and integration with other services.

Hands-on experience in working production issues like (Incidents, Problem records and deployments) across all the environments to support users and fix solutions.

Hands-on experience in working troubleshoots and resolving the issues with infrastructure teams.

Back up or copying data from one host to another using scp, distcp, Terraform and Ansible tools from disaster recovery.

Familiar with CI/CD pipelines using Ansible, GIT, Jenkins for repositories.

Hands-on experience with Batch Processing scheduling (Batch job, AutoSys) jobs.

Hands-on experience in creating Kafka topics, granting access and troubleshooting user end fixing broker issues.

Hands-on experience in Managing and maintaining Windows Server environments, Active Directory, and group policies.

Experience in cluster capacity planning, performance tuning, cluster monitoring, and Troubleshooting Hadoop cluster.

Experience in managing Oozie workflows and Job Controllers for job automation - shell, hive, and Cron jobs.

Experience in Configuring ODBC configuration and troubleshooting closely working on Jupiter Notebook, Tubleau, SAS, to support Informatica, Data science teams.

Environment: Hadoop MapReduce, Cloudera Management, Hortonworks Ambari, AWS EMR, EC2, CloudWatch, S3, IAM, CloudFormation, Redshift, Cloudera, Hortonworks, Spark, HDFS, Map Reduce, HiveServer2, HiveMetastore, HBase, Sqoop, Ranger, Solr, Spark, Oracle, MySQL, Kafka, Unix, Linux, Java, Shell, Python Scripting.

Ford Motor Company, Dearborn, MI Sep 2018 – Aug 2023 Role: Sr. Hadoop Administrator

Responsibilities

Hands-on experience in upgrading Cloudera CDH and Hortonworks platform to latest versions.

Hands-on experience in Cloudera (CDP) and Hortonworks (HDP) upgrades, patches and installing Hadoop Components.

Hands on Experience in multiple domains which includes cloud computing, security, identity and access management through Kerberos, LDAP, Sentry and Ranger.

Experience as a Data platform engineer, which includes Kubernetes, Python and Jenkins.

Performed and involved in Upgrading HPE EDF, CDP and HDP to latest versions.

Hands-on experience in Managing on-prem Cloudera (CDP) and Ambari Hortonworks (HDP) components such as (HDFS, Yarn, Hive, Spark, HBase, Impala, Zookeeper, Pig, Cloudera Management, Kafka, Sqoop, Ranger, Sentry, Pyspark and Oozie).

Hands-on experience in taking Database Backups (MariaDB, PostgreSQL) during upgrades.

Hands-on experience in upgrading the cluster/patches on Cloudera private and public, Hortonworks platforms.

Hands-on experience in working with infrastructure teams to manage/upgrade server patching and system fixes.

Hands on experience in Managing and maintaining Windows Server environments, Active Directory, and group policies.

Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage &review log files.

Hands-on experience in migrating Rhel servers and services from one host to another.

Hands-on experience in Managing and maintaining Windows Server environments, Active Directory, and group policies.

Responsible for memory management, queue allocation, and distribution experience in Hadoop/Cloudera environments.

Hands on experience in Install, configure, monitor, and maintain Hadoop ecosystem components (HDFS, YARN, Hive, Spark, etc.).

Hands on experiencing in enabling SSL/TLS, LDAD/Kerberos through Ranger, Sentry security authentication.

Experience with Batch Processing scheduling (Oozie, AutoSys) jobs.

Experience in Configuring ODBC configuration and troubleshooting closely working on Jupiter Notebook, Tableau, SAS to support Informatica and Data science teams.

Experience in Responsible Hadoop clusters for memory management, queue allocation and performance tuning user workloads.

Hands-on experience in Migrating Data from active cluster to another cluster and during cloud Migration using distcp.

Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage &review log files.

Hands-on experience in troubleshooting and resolving hardware, software, and network issues across diverse environments, Opening HPE cases as well.

Hands-on experience in programming and scripting languages, such as Java, Scala, and Bash.

Day to day responsibilities include solving developer issues, deployments moving code from one environment to another environment, providing access to new users and providing instant solutions to reduce the impact and documenting the same and preventing future issues.

Experienced with Spark improving the performance and optimization of the existing algorithms Hadoop in using Spark Context, Spark-SQL, Data Frame, and YARN.

Hands on Experience with Hadoop ecosystems such as (Hive, Spark, MapReduce, Ranger, Kafka, Oozie, Sqoop, Solr).

Experienced in adding/installation of new components and removal of them through Hortonworks Ambari and Cloudera Manager.

Hands-on experience in Monitoring systems and services through (Unravel, Splunk, Grafana) dashboard to make the clusters available for business.

Performance tuning the Spark, MapReduce, Hive jobs by changing the configuration properties and using broadcast variables.

Environment: Cloudera, Kubernetes, Hortonworks Ambari, Spark, HDFS, Map Reduce, Hive, HBase, Sqoop, Ranger, Solr, Kafka, Oozie, Spark, Oracle, MySQL, Rhel (7,8,9) Servers, Unix Linux Java, Shell Scripting, Ansible, Grafana, Splunk, Unravel, Sientry.

Vanguard, Charlotte, NC Sep 2017 – Aug 2018

Role: Hadoop Administrator

Responsibilities:

Worked as an admin in Horton works (HDP 2.2.4.2) distribution for 4 clusters ranges from POC to PROD.

Worked on installing and configuring HDP, Cloudera services for (Hue, Ambari views).

Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage &review log files.

Day to day responsibilities include solving developer issues, deployments moving code from one environment to another environment, providing access to new users and providing instant solutions to reduce the impact and documenting the same and preventing future issues.

Experienced in adding/installation of new components and removal of them through Ambari.

Installed and Configured Hortonworks Data Platform (HDP) and Apache Ambari

Installed and Configured Hadoop Ecosystem (MapReduce, Pig, and Sqoop. Hive, Kafka) both manually and using Ambari Server.

Implemented and Configured High Availability Hadoop Cluster (Quorum Based)

Installed and Configured Hadoop monitoring and administrating tools: Nagios and Ganglia.

Back up of data from active cluster to a backup cluster using distcp.

Periodically reviewed Hadoop related logs and fixing errors and preventing errors by analyzing the warnings.

Hands on experience working on Hadoop ecosystem components Hadoop Map Reduce, HDFS, Zookeeper, Oozie, Hive, Sqoop, Pig, Flume, Atlas.

Experience in configuring Zookeeper to coordinate the servers in clusters to maintain the data consistency.

Experience in using Flume to stream data into HDFS - from various sources.

Used Oozie workflow engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as Java map-reduce, Hive, Atlas and Sqoop as well as system specific jobs.

Installed Oozie workflow engine to run multiple Hive and pig jobs.

Worked on analyzing Data with HIVE and PIG +9.

Helped in setting up Rack topology in the cluster.

Implemented automatic failover zookeeper and zookeeper failover controller.

Environment: Hadoop Horton works (HDP 2.2.4.2), Cloudera, Spark, HDFS Map Reduce, Atlas, Pig Hive HBase Flume Sqoop, Windows 2000/2003 Unix Linux Java, Shell Scripting., Kafka, Oozie, Java, Oracle 10g, MySQL, Impala, Nagios, Ambari.

HP, Texas Sep 2016- Aug 2017

Role: Hadoop/Linux Administrator

Responsibilities:

Installation, Maintenance, Administration and troubleshooting of Sun Solaris 8,9 and Redhat 9, AS 3.0 servers on various hardware platforms that include Sun 4800, V480, 280R, 4500, 3500, Dell 6400, 2400, 1800 etc.

Installed and Configured Hortonworks Data Platform (HDP) and Apache Ambari

Installed and Configured Hadoop Ecosystem (MapReduce, Pig, and Sqoop. Hive, Kafka) both manually and using Ambari Server.

Performed automated installations of Operating System using Jumpstart for Solaris and Kickstart for Linux.

Worked on installing and configuring HDP, Cloudera services for (Hue, Ambari views).

Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Manage and review data backups, Manage &review log files

Extensively worked on hard disk mirroring and stripe with parity using RAID controllers

Implemented a High Availability Cluster using 2 V480s, a T3 and Veritas Cluster Server.

Worked on setting up Kerberos and used it to grant access to the users.

Installation, Management, Configuration of LAN/WAN systems utilizing Cisco switches and routers.

Configured various services, devices as well as applications of UNIX servers and worked with application team to customize the environment. Worked with Apache and developed several UNIX scripts to automate web tasks

Configured firewall based on red hat Linux and FreeBSD 4.x that has three network interfaces.

Managed existing documentation for systems and created new procedures to support new products. Created documentation for disaster recovery project

Managed server on VMware and provided test environments on virtual machines.

Provided IT support to internal staff members.

Used Puppet to create Modules

Provided application support to large user groups.

Installed hardware, installed RHEL 3.0 OS and configured required network on 1000 Node HPC cluster.

Managed HPC cluster, performed hardware, BIOS and application upgrade.

Configured and managed Apache web server.

Managed software and hardware RAID systems.

Configured and maintained FTP, DNS, NFS and DHCP servers.

Manage user accounts and authentication process by NIS service.

Managed System Firewall utilizing IP Chains, IP Tables. Implemented SSH SSL.

Managed user disk usage by setting up quotas.

Updated software packages and applied security patches.

Performed hardware maintenance, upgrades and troubleshooting on workstations and servers.

Environment: Solaris 8,9, Red hat Linux, AS 3.0, Veritas Volume Manager 3.x, 4.0, Veritas Cluster Server 4.1, Cisco Routers, Sun 4800, V480, 280R, 4500, 3500, Dell 6400, 2400, 1800 Redhat 8, 9

Contact this candidate