Big Data Infrastructure Management

VISHWAKSEN DOOSA

**** ***** **, *******, **********, 90630 **********.*****@*****.*** (510) 359 - 8974 CAREER SUMMARY

● Big Data Ops Engineer with over 10 years of IT experience,specializing in Linux,Hadoop,and AWS infrastructure,and proven expertise in Big Data architecture,Hadoop administration,and cloud-based data infrastructure management.

● ProficientincreatingCloudFormationandTerraformtemplatestodeployAWSserviceslikeEMR,S3,IAM, Elastic Kubernetes Services, and other resources.

● Experienced in troubleshooting EKS,EMR,and Hadoop clusters for Spark workloads with strong knowledge of infrastructure management on AWS.

● ProficientinusingGrafana,Splunk,PepperdataandDataDogforapplicationperformancemonitoringon EMR, Hadoop and Kubernetes clusters.

TECHNICAL SKILLS

Hadoop Ecosystems :HDFS, HBase, MapReduce, Zookeeper, Yarn, Hive, Spark, Impala, Kafka, Oozie Database :MySQL, Postgres, NoSQL, Oracle

Scripting Languages :Python, Shell scripting

Operating Systems :RHEL, CENTOS, UNIX, LINUX, VMware, Windows Hadoop Distributions :Cloudera Manager, Ambari

Cloud Computing :AWS, EKS

Cloud Services :EMR, EC2, S3, CloudWatch, Glue, IAM, RDS, Lambda, CloudTrail ETL Tools :Informatica, Talend

Testing Tools :HP QC, Jira, One Jira

No SQL Databases :HBase, Cassandra

PROFESSIONAL EXPERIENCE

Client: Apple, Cupertino, CA, USA March 2022 – Present Hadoop/AWS Site Reliability Engineer

● Created EMR clustersforvariousteams,configurededgenodes,andsupportedmigrationfromHadoop to EMR and Elastic Kubernetes clusters.

● Used AWS CloudFormation templates and terraform to automate deployments for EMR,S3,VPC endpoints, CloudWatch Alarms, and IAM roles.

● Built and maintained Helm charts to simplify Kubernetes application deployment,ensuring consistent configuration and reducing deployment time.

● Developed Bash scripts for automating system updates,backups,log rotation and developed Python-based Lambda functions to monitor EMR health,withemailalertstriggeredbyCloudWatchfor any warnings or failures.

● Experienced in creating and managing S3 buckets,setting up S3 replications,and configuring cross-accountS3accesstomeettherequirementsofvariousapplicationteams.Additionally,I’vecreated and managed roles and policies in AWS IAM, ensuring secure access to AWS.

● Worked with the AWS services team on infrastructure issues,including EMR-managed scaling,metrics collection, and instance state issues, and applied patches as needed.

● Configured andmaintainededgenodesonClouderaHadoopclusters(CDH5.16.1,CDH6,CDP7.1.7,and CDP 7.1.9)across both development and production environments.Managed over 30Hadoopclusters with over 10,000 nodes, providing continuous 24x7 monitoring and support.

● Monitoredapplicationperformance,resourceusage,andsystemhealthofHadoop,EMR,andKubernetes clusters using Grafana, Splunk, DataDog, Hubble, and Kibana.

● Configured Yarn containers and queues for multi-tenant clusters and managed resource quotas in HDFS.

● Provided application support on Hadoop and AWS fordevelopmentteams,managedsystemupgrades, andresolvedissuesinEMRandEKSenvironments.Offeredon-callsupportforincidentmanagementand troubleshooting, solving developer issues, and handling user access requests. Client: Dish, Englewood, CO Oct 2020 – Mar 2022

Hadoop/AWS Cloud Administrator

● ManagedAWSservicesincludingEC2,S3,IAM,andEMR,ensuringsecureandoptimizedconfigurations, and Supported the migration of on-premise Hadoop clusters to AWS, improving resource management.

● Automated routine tasks like starting and stopping Ec2 instances,and RDS databases using lambda functions written in Python and triggered using CloudWatch and created bootstrap scripts to install cleanup tool, spark versions, etc. For EMR cluster creation.

● CreatingEC2,EMR,RDS,Lambdafunctionsandhandlingsoftwareinstallations,backups,andpatchesas well as routine administrative tasks.

● Used AWS Terraform to automate PostgreSQL deployments and configurations on AWS.

● Responsible for estimating the cluster size,monitoring,and troubleshooting the Spark and Hive applications,andInvolvedinAnalyzingsystemfailures,identifyingrootcauses,andrecommendedcourse of action.

Client: TransAmerica, Plano, TX Aug 2018 – Sep 2020 Hadoop Administrator

● Upgraded CDH clusters, resolved post-upgrade connectivity issues, and automated deployments.

● Set up Git and Jenkins for CI/CD processes, ensuring code stability across environments.

● Provided 24x7 production support, managing HDFS, Hive, Impala, and ensuring high availability.

● Monitored workload, job performance and capacity planning using Cloudera Manager.

● Working withdatadeliveryteamstosetupnewHadoopusers.ThisjobincludessettingupLinuxusers, Setting up Kerberos principals and testing HDFS, Hive, Impala and MapReduce access for the new users.

● Used Informatica ETL tools like Powercenter_designer,Powercenter_worflow_manager,Informatica Developer for the development activity.

● Performed HDFS cluster support and maintenance tasks like adding and removing nodes without any effectonrunningnodesanddata.Monitoredandcontrolledlocalfilesystemdiskspaceusage,logfiles, and cleaning log files with automated scripts.

Client: Morgan Stanley, Manhattan, NY July 2017 – July 2018 Hadoop Administrator

● Deployed Hadoop clusters on AWS EC2 instances and configured high availability.

● Collaborated with third-party vendors,including Securonix and SAS Viya,to deploy and integratetheir applications on Hadoop clusters,enhancing Spark resource efficiency and optimizing overall cluster performance.

● Supported Kafka clusters for data ingestion and set up LDAP and Kerberos integrations for security.

● ConfiguredHighAvailabilityforHadoopservicesandsetupLoadbalancersforBigDataserviceslikeHive and Impala.

Cognizant Technology Solutions, India May 2012 – July 2015 QA Analyst/System Admin

● Involvedinthepreparationoftestplans,teststrategiesandtestmethodologiesandwrotetestcasesto test the application manually.

● Created and executed Test Scripts and Test Cases based on business requirements and used Quality Center/ALM for bug tracking and reporting.

● Created SQL queries for quality assurance and analysis.

● Install,configure,andmaintainLinuxserversincludingwebserver,mailserver,applicationanddatabase server.

● Managed,monitoredandtestedindividualandgroupuseraccessprivilegesandsecurityandmonitorthe UNIX server utilization,network links utilization,generated customized resource utilization reports for servers and network links using the tool Zabbix through TGIM.

● Schedule jobs through Crontab to backup Project shared folders.

● Installed the daily updates in windows server 2003, 2008 & windows. Education:

Masters in Information Systems Security, Universityof the Cumberlands, KY, USA Bachelors in Information Technology, Vellore Instituteof Technology, TN, India

Contact this candidate