Engineer Data

BHARATH PUSULURI

E-Mail: **************@*****.***

Phone: 571-***-****

PROFESSIONAL SUMMARY:

●Overall 14+ years of experience in Software Design, Development and Infrastructure Administration.

●Worked in different bigdata technologies Teradata, Hadoop, Spark, Kafka, Hbase, Hive, Sqoop and search engine tools like Elasticsearch and Solr.

●Worked as an ETL Designer, build both batch and real-time data pipelines based on the business requirements.

●Implemented CICD from lower environments to prod environments.

●Worked as a Teradata DBA and Hadoop Administrator and provided 24/7 support for a production environment.

●Troubleshoot the existing pipelines and helped with optimization techniques and decreased the footprint of memory and running time.

●Worked as Data Modeler in designing the Logical Data models from the business metadata.

●Good experience in leading the team, independent contributor and perform Project Management activities.

●Good experience in handling client requirements and providing solutions by adapting new technologies based on requirement.

●Experience in writing the Solution Documents.

●Proactive, Positive-minded, quick learner and highly passionate towards work.

●Excellent analytical and communication skills and ability to work independently with minimal supervision and perform as part of a team.

●Enthusiastic about learning new concepts in emerging technology and apply them suitably.

TECHNICAL SKILLS:

BIG Data Platform : HDP 2.7, HDP 3.1

Hadoop Ecosystem & Tools : HDFS, YARN, Tez, Hbase, Phoenix, Hive, Kafka, Spark, Ranger, Zookeeper, LLAP, Atlas, Druid, Hue, Knox, Oozie, Airflow

RDBMS : Teradata, SQL Server, MySQL, Postgresql

Search Platforms : Elasticsearch, Solr

Reporting Tool : Kibana

Version Control System : Git

Build Tools. : Maven, SBT, Gradle

CICD : Vela, Drone, Ansible, Chef

Operating Systems : RHEL6.x, Centos, suselinux, Windows Server

Languages : Shell Scripting, SQL, Scala, Python

CERTIFICATIONS:

●Teradata 12 basics certified

●CCA 500 CLOUDERA ADMINISTRATION CERTIFIED

EDUCATION: BACHELOR OF TECHNOLOGY & SCIENCE

PROJECT EXPERIENCE:

Adaequare INC Aug 2019 to till date

Client – TARGET CORPORATION

Role: Software Developer Operations Engineer - Mid

The objective of the project is to serve the BigData requirements throughout the organization from acquiring products, purchasing items, marketing, sales, inventory ...etc. in secure and centralized file system that allows real time processes and improve the company profit with good decisive reporting.

BigData Platform is one of the highly technical platforms serving all the purposes of Target to achieve its profits to customer satisfactions. There are many new tools integrated like Atlas, Ranger to give good authorization and security management.

Roles and Responsibilities:

Conduct workshops with distinctive subject matter experts in Target to elicit requirements, perform gap analysis and translate business requirements and business processes into functional design specifications for Hadoop configuration and custom development work.

Work closely with development teams to clearly communicate the latest technical improvements and closely monitor the development progress to ensure the timely completion of custom developments.

Develop the frameworks needed for spark streaming, spark batch process with customized components.

Developing Oozie workflows for job automation and schedule via coordinators.

Serve as technical point of contact for the applications runs on Hadoop environments and plan for allocating resources with respect to requirement.

Thoroughly performs code reviews to make sure resources are optimally utilized by considering the stability of the system.

Work with analytical, machine learning supported client tools like Jupyter and resolve the issues raises by customers.

Configure the properties of the tools in production environment by performing different level of test scenarios and verify in lower environments.

Design and develop the reports for the cluster health monitoring using different tools like ELK, Nagios, Uchiwa.

Develop training material for the business key users & conduct key user training.

Drive and assist with the integration testing across all systems for different components like Hive, Hbase, Kafka, Druid.

Perform the performance tuning of Hadoop clusters across different environments.

Diligently teaming with infrastructure, network, and security to guarantee high system availability for customers.

Deploying and maintaining a Hadoop cluster, adding and removing nodes using cluster monitoring tools and configuring the NameNode high availability and keeping a track of all the running Hadoop jobs.

Gaining optimum performance with data compression, region splits and by manually managing compaction in HBase

Automate daily routine works to decrease the time footprint on daily activities and make sure to develop new customized tools for organization benefits.

Work with analytical, machine learning supported client tools like Jupyter and resolve the issues raises by customers.

Testing and applying Bug fixes in Hadoop cluster from upstream apache.

Using docker containers created needed python rpm packages.

Environment: Hadoop, Yarn, Hive, Hbase, Kafka, ElasticSearch, Druid, Spark, Python, Scala, JSON, XML, Git, Linux & Windows OS.

STANDARD CHARTERED BANK, Malaysia Sep 2016 to Aug 2019

Role: Senior Hadoop Admin Teradata DBA

EDMp - The EDMp is an enterprise wide information management platform that was built for bringing up a bank wide Data Model, with both structured and unstructured data. It is a One Stop Data and MIS Reporting Solution for SCB.

RDA – Retail Application processes the CCMS data and performs campaigning based on Business needs.

Roles and Responsibilities:

Hadoop:

●Responsible for Cluster maintenance, commissioning and decommissioning Data nodes, Cluster Monitoring, Troubleshooting, Manage and review data backups, Manage & review Hadoop log files.

●Providing subject matter expertise for the new use cases to the operations team w.r.t capacity and performance.

●User access permission to business users using tools such as Ranger.

●LDAP integration with Ranger.

●Troubleshooting issues related to real time data flow.

●Ensure smooth functioning of Hadoop components such as HDFS, HBASE, HIVE, KAFKA and SPARK.

●Liaising with hardware vendors and other service providers.

●Ongoing administration, troubleshooting and monitoring of the cluster.

●Monitoring and troubleshooting job and service failures.

●Analyzing the Hadoop Service (Daemon) Failures by checking for errors in logs and sharing the analysis.

●Using BMC Remedy and Service now for incident management and change management like minor and standard changes for service as per the SLA following with ITIL managements.

●Hadoop Configuration and Database backup.

●Raising cases using Hortonworks and coordinating with Support team for troubleshooting analysis.

●Automated few monitoring tasks using Python and Shell scripting.

●Written data pipelines with scala to move the data from HDFS to Elasticsearch.

Elasticsearch:

●Responsible in configuring metricbeat, filebeat indices to monitor the Hadoop systems.

●Responsible in creating the Kibana dashboards and work with users to resolve the issues. Responsible in maintaining the Elastic Cluster and respective DR sync strategies.

Teradata:

●Optimized the queries which are impacting the system by required Partitioning, Indexing.

●Tuned the TASM to avoid flow control and utilize the system efficiently.

●Tuned the parameters on the system to avoid run time errors and effectively utilize the system with less manual interventions.

●PDCR performance reports and Performance tuning.

●Worked on the DSA jobs implemented to Quantum Tape Library and automated scripts in extending the policies of the backup.

●Implemented the DSA jobs to copy from PROD and DR systems via file system.

●Tuned the network settings on DM server and done capacity analysis on the Datamover.

●Provided the capacity estimations on Teradata servers in terms of Storage and CPU.

●Worked with Teradata GSO teams in reviewing the NTA’s, upgrading the patches based on the bug fixes and features.

Environment: Teradata 15.10, 16.20, Intelliflex system, Backup netvault, PDCR, DSA, quantum storage, Hdp 2.4, 2.7. Hortonworks distribution, Kafka, hbase, spark, hive, ELK 5.2, BMC Remedy tool, Kibana, NiFi, Esxi servers, ITIL process.

TERADATA, Hyderabad, India July 2014 to Aug 2016

Role: Software Engineer II

SIT EcoSystem - The Ecosystem Project is to work with Big Data technologies in combination of Teradata Data warehouse. Team is responsible for developing the different configurable clusters and installs the Hadoop components to run the business cases efficiently on the Hadoop machines. Monitor the Hadoop machine via Ambari, monitoring the logs and make sure there is no discrepancy for the jobs running. Apart from monitoring the Hadoop machines, also monitor the TD system components via newly added features in ViewPoint portlets and make sure system is behaving as expected. The team certifies the new connectors developed for Teradata and Hadoop.

Roles and Responsibilities:

●It’s a R&D Platform, where the focus is on the new feature’s implementation and respective Proof of Concepts.

●Performed the Hadoop installation on the VM’s both single and dual machine clusters both manual and automated scripts.

●Configured the Hadoop components like sql-h, tdch connections.

●Performed admin activities on Hadoop via Ambari.

●Responsible for trouble shoot the DBS server problem at all the components level (OS, PDE, FileSys, DBS, TDWM and Viewpoint) and escalate the issues to support team if needed.

●Good experience in data consistency check via Ferret/CNS utilities (Checktable & Scandisk).

●Worked with architects in designing the workloads and scripts.

●Managing team of 5 members to perform the activities of the project.

Environment: Teradata 15.10, Hadoop 1x, Loom, suse Linux, DSA, Datamover, Teradata Utilties, Viewpoint, PDCR, Esxi servers, SDD generator.

IBM, Bangalore, India Sep 2012 to July 2014

Role: Application Developer

WellPoint - WellPoint is a largest managed healthcare. There are many modules in WellPoint and I have worked in Provider subject area and had knowledge on Claim, RHI subject areas. Our project goal is to combine the 7-source systems data and generate a single source of truth.

Roles and Responsibilities:

●Analyzed the client’s requirements mentioned in BTRD and written BSD, TSD documents

●Worked with DA team to understand the model and helped in fixing the model according to the requirements.

●Identified business user requirements and prepared ETL strategies for data extraction, transformation and Loading into the data warehouse.

●Managed the team of 3 members. Worked on creating Semantic Views as per Business requirement. Worked on creating the Security views as per Business requirement.

Environment: Teradata 12.0, Informatica, Linux, Erwin tool, Teradata Utilities, Viewpoint, TASM, SQL assistant, Clear case

TERADATA, Hyderabad, India Feb 2010 to Sep 2012

Role: Software Engineer

SIT R - The Solutions Integration and Sustaining Certification team is responsible for developing SQL based jobs and execute those jobs on TD machines with latest versions and certifying the integrated Teradata Warehouse, including platform, OS, and Teradata releases. The team is responsible to certify both new pre-GCA releases as well as sustained releases.

Roles and Responsibilities:

●Responsible for DBS upgrades/configuration and trouble shooting.

●Developed the jobs [created business queries with SQL and Teradata Utilities].

●Involved in the design of physical models and logical models for the database.

●Prepared the data w.r.t customer data. Developed tables, views, macros required for jobs.

Environment: Teradata 12.0, 14.0, Suse Linux, Teradata Utilities, Viewpoint, TASM, SQL assistant

VALUELABS, Hyderabad, India July 2006 to Feb 2010

Role: Senior Software Engineer

Nielsen – Nielsen (formerly: Nielsen Mobile’s) mission is to help customers turn data into superior business results by providing high-quality measurement products and integrated analysis. We serve our customers by providing measurement data that is Independent, Comprehensive, Integrated.

Roles and Responsibilities:

●Developed all the required database objects.

●Worked extensively with client gathering requirements to generate customized SQL reports. Created Store Procedures as per the requirement.

●Worked on fixing the bugs w.r.t performance tuning.

●Generated SQL patches for intermediate requirements

Environment: SQL Server 2005, SVN, Teradata, Cognos, Microstrategy.

Contact this candidate