Hadoop Data

Location:

Keller, TX

Salary:

As per the market price.

Posted:

August 21, 2020

Contact this candidate

Resume:

VEERA B DASARI

********@*******.***

+1-657-***-****

https://www.linkedin.com/in/dr-veera-b-dasari-95995510/ SUMMARY

• 24+ years of IT and Business Strategic experience with excellent insight into Hadoop / Big Data / Data Science/Cloud (AWS and GCP) Technology and Architecture, IT Program and Technical Management, ITIL Process, Service Operations, Leading the technology groups and proven success in driving improvement initiatives and delivering results.

• Experienced Technology Leader with strong experience in Data and Information architecture implemented on modern technology platforms enabling Digital and Big Data capabilities.

• Experienced in diverse data disciplines ranging from Analytics, Operational Reporting, Data Quality, Data Governance, Data Cataloguing, Database Design and Data Science. Responsible for setting Data Strategy and devising the Data Multi Year Planning activities to align changing Business priorities with Data and Information Strategy.

• Experience with Hadoop Ecosystem in installation, development, Architecture and configuration of different Hadoop eco-system components in the existing cluster.

• Experience in Hadoop Ecosystem (HDFS, MapReduce, Spark, HIVE, HBase, MongoDB, Kafka, PIG, SQOOP, Flume, OOZIE, and HBase), Zookeeper, Ambari and NoSQL.

• Setting up automated 24x7 monitoring and escalation infrastructure for Hadoop cluster using Nagios and Ganglia.

• Worked with Sqoop in Importing and Exporting data from different databases like MySQL and Oracle into HDFS and Hive.

• Extensive experience in migrating, managing, architecting, implementing AWS and Google Cloud

• Environments – S3, ECS/Docker/Fargate, RDS - SQL Server, DynamoDB, Redshift, Glue, Lambda, Python, API GW, VPC, ELB, ASG, CloudWatch, Big table, DevOps, DataOps

(Zaloni / AWS/GCP), Snowflake, Cloud Trail and CI/CD.

• Cloud – AWS. Azure and GCP.

TECHNICAL SKILLS:

Operating System: Red Hat, CentOS, Ubuntu, Solaris, SUSE and Windows Hardware: Sun Ultra Enterprise Servers (E3500, E4500), SPARC server 1000, SPARC server 20 Enterprise Servers.

Languages: C, C++, Core Java, SQL, Scala, Python, R and JDK 7/8 Web Languages: HTML, CSS, and XML

Hadoop Distribution: Cloudera and Hortonworks Ecosystem Hadoop Map Reduce, YARN, HDFS, Sqoop, Spark, Hive, Pig, HBase, Sqoop, Flume, Zookeeper and Oozie. Tools: JIRA, Putty, WinSCP, FileZilla, Cmder, etc. Virtualization: Vagrant, Virtual Box and VMware.

Database: HBase, RDBMS Sybase, Oracle 7.x/8.0/9i, MySQL, SQL, Cassandra, Big Query and Big Table.

Protocols: TCP/IP, FTP, SSH, SFTP, SCP, SSL, ARP, DHCP, TFTP, RARP, PPP and POP3 Shell Scripting Bash.

Cloud Technologies: AWS, Azure and GCP

PROFESSIONAL EXPERIENCE:

Mphasis Corp / Charles Schwab, Westlake, TX February 2020 – Present GR. Manger (Tech) & Sr. Architect (BIG DATA – HADOOP / Data Science /CLOUD)

• SME for Hadoop Ecosystem, Splunk and Cloud Architecture (AWS & GCP).

• Responsible for the Big Data Environment and Big Data Hub Support with MapR clusters

(Hadoop Distribution) for managing the full lifecycle of a Hadoop solution.

• Requirements analysis, the platform selection, design of the technical architecture, design of the application design and development, testing, and deployment of the proposed solution.

• Lead and SME for the entire BDE and BHDS support team, Platform engineering team.

• Involving Installation, Volume creation, Trouble shooting, Balancing, HA, DR, Patching of the OS and other tools in Hadoop Architecture.

• Straight shooter for BDE (Hadoop) Architecture.

• Finding pain points and providing solutions in the entire BDE/BDH architecture.

• Migrating Cloud (GCP, AWS) and Big Table.

• Drive internal process improvements across multiple teams and functions.

• Interface with Engineering teams and cross-functional partners for project requirements and scope

• Handling any technical accounting matters as well as providing effective technical accounting guidance and support to wider finance teams. Wells Fargo, Charlotte, NC Aug 2019 – February 2020 Big Data Architect

• Responsible for developing, administration and architecture of Hadoop Ecosystem for Big Data using Hadoop stack in the Hadoop cluster with various tools like HDFS, MapReduce, Hive, PIG, Sqoop, Spark, Zeppelin, Atlas, Flume, Solr, Kafka, Zookeeper, MongoDB etc.

• Working with multiple platforms, architectures and diverse technologies gathering requirements and design and develop ETL process to transform information from multiple sources. Working with a team of talented engineers to deploy, configure and develop a highly scalable, fault tolerant, and responsive big data ETL platform with next generation streaming data technologies.

• Architect, Evangelize, Design and build Enterprise Class Big Data platform using Hadoop and cloud eco-system.

• Design and build reusable frameworks to help development teams.

• Subject Matter Expert for Spark, Scala, Kafka, MongoDB, Hive, HBase and R.

• Facilitate/lead reviews (walkthroughs) of technical specifications and program code with other members of the technical team, communicating design, requirements, features of the technical team, communicating design, requirements, feature set, functionality and limitations of systems/applications to development teams.

• Defining job flows in Hadoop environment using tools like oozie for data scrubbing and processing.

• Working in configuring zookeeper to provide cluster co-ordination services.

• Working the security requirements for Hadoop and integrating with Kerberos authentication, infrastructure KDC server setup, creating and managing the realm domain, configuring and authentication of LDAP and Active Directory.

• Working with cloud infrastructure with Azure, AWS, EMR and RDD to deploying a Hadoop cluster.

Boeing, Bellevue, WA Oct 2018 – Aug 2019

SR. Technical Architect (BIG DATA – HADOOP)

• Architect, Evangelize, Design and build Enterprise Class Big Data platform using Hadoop and cloud eco-system.

• Design and build reusable frameworks to help development teams.

• Subject Matter Expert for Spark, Scala, Kafka, MongoDB, Flume, Hive, HBase and R.

• Defining job flows in Hadoop environment using tools like oozie for data scrubbing and processing.

• Working in configuring zookeeper to provide cluster co-ordination services.

• Loading logs from multiple sources directly into HDFS using Flume.

• Configuring Zookeeper to provide cluster co-ordination services.

• Working with cloud infrastructure with Azure, AWS, EMR and RDD to deploying a Hadoop cluster.

Run Ren, Inc., Carlsbad, CA Jul 2018 – Sep 2018

Big Data Architect

• Responsible for setting up Kerberos enabled secured, high available Pivotal HD/HAWQ clusters in DEV, TEST and PROD environments

• Developed a custom Data Copy tool and copied data in petabytes of size from Cloudera HD clusters to Pivotal HD clusters

• Migrated different applications from Hadoop 1.0 (CDH 4.2) to Hadoop 2.0/YARN (Pivotal HD)

• Created HAWQ equivalents to Hive queries. Benchmarked various tests against Hive and HAWQ.

ViaSat, Inc. – Carlsbad, CA April 2018– July 2018

Hadoop Administrator

• Responsible for implementation and administration of Hadoop infrastructure.

• Aligning with the systems engineering team to propose and deploy new hardware and software environments required for Hadoop and to expand existing environments.

• Working with data delivery teams to setup new Hadoop users.

• Setting up Linux users, setting up Kerberos principals and testing HDFS, Hive, Pig, and MapReduce access for the new users.

• Cluster maintenance as well as creation and removal of nodes using tools like Ganglia, Nagios, Cloudera Manager Enterprise, Dell Open Manage, and other tools.

• Performance tuning of Hadoop clusters and Hadoop MapReduce routines.

• Screen Hadoop cluster job performances and capacity planning Monitor Hadoop cluster connectivity and security Manage and review Hadoop log files.

• File system management and monitoring.

• HDFS support and maintenance.

• Diligently teaming with the infrastructure, network, database, application, and business intelligence teams to guarantee high data quality and availability.

• Collaborating with application teams to install the operating system and Hadoop updates, patches, version upgrades when required.

• Point of Contact for Vendor escalation

Chisrin Corp, New York, NY Jun 2014 – Mar 2018

Hadoop Administrator and BI Consultant

• The main goal of this project is to replace existing Legacy System with Hadoop. We have data residing in multiple sources like Oracle, SQL Server, Teradata, DB2 and MySQL where we pull the data to Hadoop and perform necessary transformations and aggregations in Hadoop and then load the necessary data to target Netezza system with the reports to the Business.

• Installed and configured multi-nodes on fully distributed Hadoop cluster.

• Involved in Hadoop Cluster environment administration that includes De-commissioning and commissioning nodes, cluster capacity planning, balancing, performance tuning, cluster Monitoring and Troubleshooting.

• Configured Fair Scheduler to provide service level agreements for multiple users of a cluster.

• Worked on installing Hadoop Ecosystem components such as Sqoop, Pig, Hive, Oozie, and Hcatalog.

• Involved in HDFS maintenance and administering it through Hadoop Java API.

• Loaded data into the cluster from dynamically generated files using Flume and from relational database management systems using Sqoop.

• Proficient in writing Flume and Hive scripts to extract, transform and load the data into Database.

• Responsible for maintaining, managing and upgradation of Hadoop cluster connectivity and security.

• Worked on Hadoop CDH upgrade from CDH4.x to CDH5. x. Lotus Digitech, Inc, Jersey City, NJ, USA Feb 2012 – Jun 2014 Hadoop Administrator

• Installing, configuring, and monitoring HDP stacks 2.1, 2.2, and 2.3.

• Installed and configured Hadoop ecosystem components like Hive, Pig, Sqoop, Flume, Oozie and HBase.

• Used Horton Works distribution of Hadoop to store and process their huge data generated from different enterprises.

• Responsible for cluster HDFS maintenance tasks: commissioning and decommissioning nodes, balancing the cluster, and rectifying failed disks.

• Responsible for cluster Map Reduce maintenance tasks: commissioning and decommissioning task trackers and map reduce jobs.

• Experience in using Sqoop to import and export data from external databases to Hadoop cluster.

• Used flume to get log files into the Hadoop cluster.

• Experience in administration of NoSQL databases including HBase and MongoDB.

• Communicating with the development teams and attending daily meetings.

• Addressing and Troubleshooting issues on a daily basis.

• Experience in setting up Kerberos in Horton works cluster. Lotus Digitech Ltd, Hyderabad, India Jan 2007 – Feb 2012 LINUX and HADOOP Engineer

• Involved in making Design documents by gathering the information from Business teams

• Developed and integrated various software components like Hive, Pig, Sqoop, Oozie, and Pig and requested by Business user as a Proof of concept.

• Development of HQL and Pig Latin scripts for Fraud and Risk teams.

• Processed XML files using Pig.

• Written Map reduces programs for cleansing of data in HDFS.

• Implemented Netezza queries in HDFS with Hive and Pig.

• Developed Map reduces code for calculating Histograms.

• Development experience with Data Meer.

• Implemented custom python UDF's in Hive and Pig

• Involved in migrating the code to Production and support for any bug fixes.

• Developed Python scripts for Streaming on Hadoop and Pig scripts for various ETL activities.

• Developed scripts to automate application deployments, Hadoop cluster performance tuning and monitoring.

Microsystems R&D India Pvt ltd., Hyderabad, Telangana, India (Dec 2006 – May 2009) CEO - Technical Head

• General & Technical Administration and Project Management Kanika overseas ltd., India (Jul 2004 – Dec 2006)

Vice President - Technical head

• General Administration Technical Advise & Business Development INCOR, Visakhapatnam, AP, INDIA (Jan 1999 – Jul 2004) Project Director

• R & D work in offshore and GIS

EDUCATION:

Post Graduate Diploma (Project Management)

Pune, Maharashtra, India

M.B.A (IT Management)

Jaipur National University - Jaipur, Rajasthan, India M.S. (I.T)

Canterbury University- Canterbury, Kent, UK

B. Tech (Geo-Informatics)

Andhra University-Visakhapatnam A.P, India

M.Tech (Remote Sensing)

Andhra University-Visakhapatnam, A.P., India

PhD. (Remote Sensing & GIS)

Andhra University-Visakhapatnam, AP, India

Certification of IT Project Management

Indian School of Business (ISB)

CERTIFICATIONS:

• PMP® certification –Project Management Institute (pmi.org PMP® #2774391)

• GCP – Professional cloud architect :

https://googlecloudcertified.credential.net/profile/ce764e5e23c41a3bb1748f45108a94bc8ef4e e17

• AWS – Certified Solutions Architect – Professional ( Validation Number W42EM882G1E41MGB) http://aws.amazon.com/verification

• Master Program in Hadoop - Big Data Architect.

• Big Data: https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+BD0101EN+2016

• Hadoop: https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+BD0111EN+2016

• Big Data and Hadoop Admin: https://accounts.simplilearn.com/user/profile/manage- certificates#

• Map Reduce and YARN:

https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+BD0115EN+2016

• Apache Pig: https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+BD0121EN+2016

• Oozie : https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+BD0133EN+2016

• Zookeeper: https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+BD0135EN+2016

• Hive: https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+BD0141EN+2016

• Spark: https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+BD0211EN+2016

• https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+BD0212EN+2016

• Data Science: https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+DS0101EN+2016

• Python: https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+ML0101EN+2016_T2

• R Language: https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+RP0101EN+2016

• Scala: https://courses.bigdatauniversity.com/certificates/user/344075/course/course- v1:BigDataUniversity+SC0101EN+2016

Contact this candidate