Data Security

Name: Pavan Kumar Thaduvai

Email:************@*****.*** Contact: +1-747-***-****

Professional Experience

o4+ years of professional IT experience this includes 4 years of proven experience in Kafka Administration on Cloudera (CDH), Hortonworks (HDP) Distributions and in AWS, Confluent Cloud and Linux Administration.

oProficient with Shell and scripting languages.

oConfigured Elastic Load Balancing (ELB) for routing traffic between zones, and used Route53 with failover and latency options for high availability and fault tolerance.

oSite Reliability Engineering responsibilities for Kafka platform that scales 2 GB/Sec and 20 million messages per sec.

oConfigured Elastic Search for log collections and Prometheus & Cloud watch for metric collections

oBranching, Tagging, Release Activities on Version Control Tools: SVN, GitHub.

oTeam Player and self-starter possessing effective communication, motivation and organizational skills combined with attention to detail and business process improvements, hard worker with ability to meet deadlines on or ahead of schedules.

oExperience in Implementing High Availability of Name Node and Hadoop Cluster capacity planning, Experience in benchmarking, performing backup and disaster recovery of Name Node metadata and important and sensitive data residing on cluster.

oTeam Player and self-starter possessing effective communication, motivation and organizational skills combined with attention to detail and business process improvements, hard worker with ability to meet deadlines on or ahead of schedules.

Technical Skills

Operating Systems: UNIX, Linux, Windows XP, Windows Vista, Windows 2003 Server

Servers: Web logic server, WebSphere and Jboss.

Programming Languages: Java, Shell Scripting and Python.

Tools: Jenkins and GitHub.

Database: Oracle.

Processes: Incident Management, Release Management, Change Management

Monitoring Tools: Confluent Control Center (C3), Sysdig, RTView and ELK-logging

Work Experience

Big Data Engineer-Kafka Administrator

Greater Chicago Area

Northern Trust Corporation

March 2018-Present

Northern Trust Bank is a financial services company headquartered in Chicago, Illinois that caters to corporations, institutional investors, and ultra-high net worth individuals. To route the millions of swift messages per day in the bank from the across the globe and after the day closing cash report generations need to route to different application teams; they chose Kafka as distributed messaging platform which provides low latency and high throughput

oInstalled the Apache Kafka open source and Confluent Kafka, both open source and enterprise editions in different environments from scratch to production environments.

oLead and involved while doing the capacity planning, architecture and hardware/software procurement for the Kafka installations.

oBuilt the PROD cluster in Stretch cluster mode between two data centers which helpful for maximum HA.

oInstalled and developed different POC's for different application/infrastructure teams both in Apache Kafka and Confluent open source for multiple clients.

oInstalling, monitoring and maintenance of the clusters for the entire environments and support available 24/7.

oInstalled both the single node-single broker and multi-node multi broker clusters and encrypted with SSL/TLS, authenticate with SASL/PLAINTEXT, SASL/SCRAM and SASL/GSSAPI (Kerberos) and also exclusively 2-way SSL.

oIntegrated topic-level security using ACLs and the cluster full up and running for 24/7.

oPerforming rolling restart at the time of software upgrade or linux patching activity scheduled.

oDo have experience on installing and running single node Kafka in PKS as well.

oInstalled and configured different monitoring tools like Confluent Control center, RTView and Sysdig (for Infrastructure health checks and app team’s data flow).

oAlso successfully integrated the logging effort of Kafka to Splunk and ELK.

oResponsible for installing and support for the components like Kafka Connect, Schema-registry and KSQL.

o Do have experience replicating data between two different data centers in a distributed mode using Kafka Connect.

oSupported and worked with the Docker team to install both open source and confluent Kafka single node and enabled security in the DEV environment.

oInstalled open source tool “Kafka Tool” for DEV environment and help the application teams to check their consumer lags and monitoring Kafka metrics like adding/viewing the topics, Partitions etc.This is just for POC purposes but no live environment is using it though.

oSuccessfully generated consumer group lags from Kafka using their API.

oSuccessfully did set up a no authentication Kafka listener in parallel with Kerberos (SASL) Listener. In addition, I tested non-authenticated user (Anonymous user) in parallel with Kerberos user.

oInstalled Ranger in all environments for Second Level of security in Kafka Broker.

oInvolved in Data Ingestion Process to Production cluster.

oInstalled Docker for utilizing ELK, Influx dB, and Kerberos.

oGood experience in documenting and implementing best practices and optimizing Kafka, Zookeeper and JVM.

oDesigned and implemented by configuring Topics in new Kafka cluster in all environment.

oWhile adding the new node, using the Rebalancer migrated partition’s across the cluster successfully.

oImplemented Kafka security features using SSL and without Kerberos. Further, with finer grain security. I set up Kerberos to have users and groups this will enable more advanced security features.

oExperience on DR capabilities of the cluster.

oCreated an automated scripts and deployed in enterprise Docker available in the bank and using the postman service able to create/verify the topic(s), ACL(s), logging and Kafka connect status.

oIntegrated all Kafka environment clusters with different monitoring tools like Confluent control center, Sysdig, RTView and open source Kafka manager.

oResponsible for weekend changes, upgrades with the confluent Kafka software in rolling restart fashion and Linux patches.

oDid some POC work for Solace and MQ using Hermes JMS and SolAdmin.

oAlso do support, install and configure the MQ and solace systems.

oWorked on disk space issues in both the non-prod/prod environments by monitoring how fast the disk space will reach to max peak load, and review what is being logged created a long-term fix for this issue (Minimize Info, Debug, Fatal Logs, and Audit Logs).

oUsing ansible tower installed and configured the Kafka and other components.

oWorking closely with Vendor, in case if any issues comes up.

oBeing a Middleware consultant, responsible for Governance, administrative and support for the app teams.

Environment: Deployed Confluent Kafka on various environments like POC/Sandbox, INT, SYS, UAT and Production environments.

Kafka Administrator-Associate Consultant

Anthem

(Alpharetta, Georgia)

May 2016 – April 2017 (1yr 1 months)

Western Governors University is a private, nonprofit, regionally accredited, online university based in Salt Lake City, Utah. The university uses an online competency-based learning model as opposed to the traditional, cohort-based class model present at most universities. WGU went for Kafka to route the messaging solutions for 60K students average and help them to get the University updates real time and later tweaked with some Devops solutions as well.

oInstalled Kafka on AWS and Hortonworks distributions.

oAdministering and Maintaining Cloudera Hadoop Clusters provision physical Linux systems, patch, and maintain them.

oLearned the messaging platforms like Solace, MQ, Kafka and Adaptris.

oDesigned and installed Apache Kafka in low-level environments like POC and SYS.

oExperience on enabling the Kafka security like open SSL and Kerberos.

oResponsible for creation of topics and making sure cluster is up and running.

oResponsible for all the design, documentation and support model.

oWorked as an L1 and L2 support for the teams using Kafka open source application/.

oPerformed optimization, capacity planning of a large multi-tenant cluster.

oWorked end to end with platform, Infrastructure and application teams and supported them 24/7 for any fire call issues.

Hadoop and Linux/Unix Administrator

Ciskon Technologies Ltd

Hyderabad, India

Jan 2014-Aug 2015(1yr, 8months)

Ciskon Technologies is a proven data staging, analytics and real time operations enterprise located in Hyderabad, India. They choose Linux platform a key role to play in the organization to provide all the needs for Infrastructure and application teams to lead their business.

oInvolved in the installation of CDH5 and up-gradation from CDH4 to CDH5

oCloudera Manager Up gradation from 5.3. to 5.5 version

oCreated POC on Hortonworks and suggested the best practice in terms HDP, HDF platform

oSet up Hortonworks Infrastructure from configuring clusters to Node

oInstalled Ambari server on the clouds

oSetup security using Kerberos and AD on Hortonworks clusters/Cloudera CDH

oExtensive experience in cluster planning, installing, configuring and administrating Hadoop cluster for major Hadoop distribution’s like Cloudera and Hortonworks.

oInstalling, Upgrading and Managing Hadoop Cluster on Hortonworks

oHands on experience using Cloudera and Hortonworks Hadoop Distributions.

oResponsible for implementation and support of the Enterprise Hadoop environment.

oResponsible for building scalable distributed data solutions using Hadoop.

oUsed Scala functional programming concepts to develop business logic.

oSpark scripts by using Scala shell commands as per the requirement.

oProcessing the schema oriented and non-schema oriented data using Scala and Spark.

oDeveloped and designed system to collect data from multiple portal using Kafka and then process it using spark.

oIntegrated LDAP Configuration this includes integrating LDAP for securing Ambari servers and manage authorization and securing with permissions against users and Groups.

oInstalled and configured Ambari Log Search under the hood it will required a SOLR instance that can collect and index all cluster-generated logs in real time and display them in one interface.

oInstalled Ansible 2.3.0 in Production Environment.

oImplemented KNOX, RANGER, Spark and Smart Sense in Hadoop cluster.

oInstalled HDP 2.6 in all environments.

oWorked on Micro Strategy report development, analysis, providing mentoring, guidance and troubleshooting to analysis team members in solving complex reporting and analytical problems.

oExtensively used filters, facts, Consolidations, Transformations and Custom Groups to generate reports for Business analysis.

oLeveraged with the design and development of MicroStrategy dashboards and interactive documents using Micro Strategy web and mobile.

oExtracted data from SQL Server 2008 into data marts, views, and/or flat files for Tableau workbook consumption using T-SQL. Partitioned and queried the data in Hive for further analysis by the BI team.

oManaged Tableau extracts on Tableau Server and administered Tableau Server.

oExtensively worked in data Extraction, Transformation and Loading data using BTEQ, Fast load, Multiload from Oracle to Teradata

oExtensively used the Teradata fast load/Multiload utilities to load data into tables

oUsed Teradata SQL Assistant to build the SQL queries

oDid data reconciliation in various source systems and in Teradata.

oInvolved in writing complex SQL queries using correlated sub queries, joins, and recursive queries.

oWorked extensively on date manipulations in Teradata.

oTested and Performed enterprise wide installation, configuration and support for hadoop using MapR Distribution.

oSetting up cluster and installing all the ecosystem components through MapR and manually through command line in Lab cluster

oSet up automated processes to archive/clean the unwanted data on the cluster, in particular on Name node and Secondary name node.

oInvolved in estimation and setting-up Hadoop Cluster in Linux.

oExtracted the data from oracle using sql scripts, loaded into teradata using fast/multi load, and transformed according to business transformation rules to insert/update the data in data marts.

oInstallation and configuration, Hadoop Cluster and Maintenance, Cluster Monitoring, Troubleshooting and certifying environments for production readiness.

oExperience in Implementing Hadoop Cluster Capacity Planning.

oExperience installing, upgrading and configuring RedHat Linux 4.x, 5.x, 6.x using kick start servers and Interactive Installation.

oResponsible for creating and managing user accounts, security, rights, disk space and process monitoring in Solaris, CentOS and Redhat Linux.

oPerformed administration and monitored job processes using associated commands.

oManages systems routine backup, scheduling jobs and enabling cron jobs.

oMaintaining and troubleshooting network connectivity.

oManages Patches configuration, version control, service pack and reviews connectivity issues regarding security problem.

oConfigures DNS, NFS, FTP, remote access, and security management, Server hardening.

oInstall the upgrades and manages packages via RPM and YUM package management.

oLogical Volume Management maintenance.

oExperience administering, installing, configuring and maintaining Linux

oOversee their Linux systems – installing, monitoring and fine-tuning them and sorting out any system or network problems

oSupport users on everything from setting up new accounts to updating the DNS, and troubleshoot email environments and apache configurations

oBuild new servers, set up test servers for new applications and develop new Linux-based architecture.

oLearned out third-party products, keep on top of advances in technology, and make sure we always work in the most effective ways.

Contact this candidate