Principal Big Data Engineer - Spark/Kafka/Hadoop AWS cloudExpert

MRUTUNJAYA CHAVADI

#B*, Subiksha Habitat, Next to ayyappa swamy temple, Near kaggadasapura Railway Cross, Bengaluru. ******@*****.*** +919*********,+919*********.

PROFESSIONAL SUMMARY

● High-performing, self-motivated and passionate professional with 14+ years of experience in SDLC with key emphasis on the Big Data Technologies - Spark,Kafka,Hadoop,Hbase,Hive,Druid.

● Architect, Design and develop Big Data Solutions using Hadoop and its ecosystem.

● Designed and developed robust data pipelines using Spark, Kafka and druid for digital marketing analytics.

● Excellent understanding of Hadoop architecture and underlying framework including storage management.

● Good working experience with RDBMS(PostgreSQL,Oracle SQL) and NOSQL databases such as Hbase and Druid.

● Designed and developed Rest API's using Java and Spring Restful webservices for real-time and reporting analytics.

● Evaluate new technologies, develop proof-of-concepts to validate the proposed architecture.

● Apply best practices for software development and documentation, assure designs meet requirements, and deliver high- quality work in timely manner.

● Proactively look for improvements/optimizations in existing/new solutions for enhanced data ingestion and improved reporting capabilities.

● Worked in different stages of Data Analytics solution – from Data Collection, Data storage, Data transformation, Data Analysis with more focus on Data Engineering.

● Work collaboratively in a team with peers, sharing ideas to solve complex and challenging business problems

● Good understanding of the big data design patterns.

● Manage technical documentation of different components, use cases and architecture changes.

● Good exposure to Agile and scaled agile software development process.

● Good team player with problem solving and trouble-shooting capabilities and will prove a valuable asset to any project.

● Exhibits honest work ethics with the ability to excel in fast paced, time sensitive environment

● Keen to learn new technologies, Team player, Competent and Analytical. TECHNICAL COMPETENCIES

RECOGNITION

● Recognition@ Target for Architecting, Designing and developing the robust data pipelines in timely manner. Programming Languages Java, Python, Scala

Operating Systems Windows 95/98/NT, Linux

Databases Oracle, Postgres, NoSQL(HBase,Druid), Redis, AWS RedShif Application Servers WebSphere, Tomcat, Wildfly

Frameworks Spring, Django Rest

IDE Eclipse, PyCharm, Intellij

Big Data Technologies Hadoop, MapReduce, Hive, Spark, PySpark Cloud Technologies AWS, AWS EMR

Methodology Agile and Waterfall

Hadoop Distribution MapR, Hortonworks

Messaging Queue Kafka

1

● Recognition@ Philips for code refactoring and Software excellence to improve code quality metrics. PROFESSIONAL EXPERIENCE

Nielsen : TAM National - Bengaluru Nov 2024 – present

● Spearheaded the migration of complex, legacy Talend jobs to spark jobs improving speed by 25% and reduce license costs

● Enhanced Data Governance by implementing data quality, observability, fault tolerance to reduce failure rate by 15%

● Ensuring code and data compliance adhering to best security practices.

● Defining the long term technical roadmap aligning with business needs and delivery. Visa : Data platform - Bengaluru Dec 2021 – Oct 2024

● optimized a robust, real-time data pipeline processing 700M to 1 billion transactions achieving 90% under six seconds

● Lead the application migration from hortonworks to visa’s own Hadoop platform to ease development

● Developed a custom Data Validation Tool (DDVT) to compare multi-terabyte datasets involving different file formats/tables.

● Enhanced the robustness and validation of batch data pipelines, decreasing the overall failure rate by 20% Target Corporation : TTS Marketing Maas – Bengaluru Sep 2019 – Dec 2021

● Designed the Maas architecture for digital marketing analytics.

● Designed and developed robust data pipelines using Spark, Kafka and druid.

● Optimized druid for enhanced data ingestion and improved query performance.

● POC for forward cache and drafted lambda architecture for serving real-time and batch ingestion.

● Architected, designed and developed async file delivery solution using Spark, kafka, oozie and spring boot. Philips India : Radiology Solutions – Bengaluru Apr 2016 – Aug 2019

● Consumed data from kafka and perform analytics on data using spark core and streaming.

● Transform and load data into Hbase Tables using Java API.

● Developed python application(backend) using Django framework.

● Running queries, reviewing analytics, and assisted in strategic decision making

● Designed and developed REST APIs in Java with spring MVC framework. Fidelity Investment : Portfolio perform KI - Bangalore Dec 2013 –Mar 2016

● Designed and Developed data transformations using Mapreduce(java) and hive.

● Prototyping and developing POC using Hadoop ecosystem(mapreduce and hive).

● Schema design for hive using partitions and buckets to optimize query performance. CGI : ADVANTAGE ERP - Bangalore July 2010 – Dec 2013 Education

• Master of Technology(WILP) - Data Science and Engineering - BITS Pilani, India – Oct 2020 – Sep 2022 DECLARATION

I hereby declare that the above written particulars are true to the best of my knowledge and belief. 2

Contact this candidate