Mani Shankar Perumalla

Kansas City, MO Email: ad2i2c@r.postjobfree.com Phone: +1-816-***-**** LinkedIn

PROFILE

●Accomplished Data Engineer with expertise in building and optimizing data pipelines, ETL processes, and data warehousing solutions. Skilled in integrating cloud and big data technologies to facilitate efficient data analysis and reporting.

●Proficient in leveraging cloud platforms like Azure, AWS, and GCP, resulting in enhanced data processing capabilities and cost optimization. Demonstrated ability to improve ETL efficiencies by 25% and reduce data processing costs by 15%.

●Experienced in Python, Scala, and Big Data ecosystems, specializing in real-time data processing and large-scale data management.

●Proven track record in project execution and analytics, with an emphasis on improving data accessibility and reliability for strategic decision-making.

SKILLS

●Programming & Scripting: Scala, Python

●Big Data: Hadoop, MapReduce, Hive, Big Query, HDFS

●Cloud: Azure (Data Lake, Data Factory, Databricks, SQL Database, SQL Data Warehouse)

●ETL & Data Processing: SSIS, Informatica, Spark, Kafka

●Data Analysis & Visualization: NumPy, Pandas, SciPy, TensorFlow, Tableau, Power BI, SSRS

●Database Management & Others: MS SQL Server, Oracle DB, MongoDB, MySQL; Agile, Python Scripting, Excel, GitHub

●Certifications: DP-203: Microsoft Certified Azure Data Engineer Associate

EXPERIENCE

Data Engineer MetLife, MO

Jan 2023 – Current

●Led the implementation of Apache Sqoop for efficient data import/export to/from HDFS, facilitating streamlined data availability for advanced processing.

●Integrated and optimized Databricks Spark jobs using PySpark, focusing on sophisticated table-to-table data operations, enhancing overall data handling efficiency.

●Engineered and refined ETL/SSIS, ELT packages, resulting in a 25% improvement in data transformation efficiency and overall process reliability.

●Developed and maintained an effective data pipeline architecture in Azure, employing Data Factory and Azure Databricks, achieving a 15% reduction in data processing costs.

●Crafted advanced Python (PySpark) programs for intricate data wrangling and transformation tasks, enabling more accurate data analysis.

●Managed various features within Azure Data Factory (ADF), including stored procedures and data flows, leading to a significant 25% decrease in data pipeline failures.

●Performed performance tuning of Hive and Spark jobs on MapReduce platforms, optimizing execution times and resource usage.

Data Engineer Dixon Technologies, India

Jan 2019 – Dec 2021

●Generated, maintained, and analyzed detailed Azure monitoring dashboards and reports, achieving a 20% reduction in customer-related issues.

●Implemented ETL and ELT operations using Azure Databricks, connecting to various relational databases with JDBC connectors, thereby streamlining data integration.

●Involved in the creation and execution of Azure pipelines, utilizing PowerShell utility for efficient data movement and transformation.

●Utilized Spark and Scala APIs for comparative performance analysis between Spark, Hive, and SQL, focusing on data frame manipulation and optimization in Scala.

●Managed the creation, monitoring, and restoration of Azure SQL databases, including successful migrations from Microsoft SQL Server to Azure SQL databases.

●Translated complex Hive/SQL queries into efficient Spark transformations using Spark RDDs and Scala, enhancing data processing capabilities.

EDUCATION

Master of Science in Computer Science, University of Missouri Kansas City, MO (May 2023)

Bachelor in Electronics and Communication Engineering, Hindustan Institute of Technology & Science, Tamil Nadu, India (May 2020)

PROJECTS

Twitter Sentiment Analysis of the U.S. Election (Jan 2023 - Apr 2023): Undertook a comprehensive sentiment analysis project on Twitter data concerning the U.S. Presidential election. Utilized advanced data analytics techniques to gauge public sentiment, identify key discussion themes, and derive insights into voter opinions and behavioral trends.

Building a Big Data ETL Pipeline for Real-time Data Processing (Aug 2022 – Dec 2023): Implemented a robust Big Data ETL pipeline for real-time data acquisition, transformation, and loading. This project was instrumental in providing timely, structured data for analytical and reporting purposes, enabling dynamic data-driven decision-making in a high-volume data environment.

Contact this candidate