Data Engineer Big

sidhartha Annamaneni

+1-302-***-**** ************@*****.*** linkedin.com/in/Sidhartha Anna

PROFESSIONAL SUMMARY

Over 5 years as a Big Data Engineer, I have excelled in developing, optimizing, and managing data sets, infrastructure, and machine learning models. My initiatives include enhancing cloud-based data warehouse security, implementing automated data validation processes, and creating high-performance ML models. These efforts have significantly enhanced data integrity, reduced operational costs, and improved system performance. Notably, I achieved a 50% reduction in migration costs for large data sets and a 30% decrease for ML models in cloud-based architectures. TECHNICAL SKILLS

Programming Languages: Python, Scala, SQL, Java.

Big data Technologies: Apache Spark, Apache Kafka, Mapreduce, AWS, S3, EC2, AWS Lambda, EMR, Redshift. Libraries: NumPy, Pandas, Maven, Ant, Scikit-learn. Database: Oracle, MongoDB, Mysql, Postgres.

Tools: Cucumber, Jenkins, Git, Splunk, Docker, Kubernetes, Tableau, PowerBI. PROFESSIONAL EXPERIENCE

Data engineer sept 2022 – present

KL Discovery Minnesota, USA

• Developed an automated AWS pipeline solution that processed over 10TB of data per month while reducing operating costs by 30%.

• Implemented a Amazon Aurora Database and DynamoDB to securely store business insights and operations data increasing storage capability by 50% while reducing latency time by 250%.

• Engineered a comprehensive data analytics framework using Amazon Redshift, Glue, and Lambda; accelerated KPI reporting by 40% and reduced operational costs by $150,000 annually

• Automated on-demand Amazon S3 backups, providing additional layer of data security and reducing manual workload.

• Developed robust Kafka producers and consumers, enabling efficient streaming of millions of events daily, which improved data throughput by 50% and reduced event processing time by 25%. Environment: AWS, DynamoDB, Aurora, AWS Lambda, Glue, Cucumber, Amazon S3, Python, Kafka, Spark. Big data Engineer June 2021 – Aug 2022

CBS Corporation Atlanta, GA

• Developed and implemented data pipelines to improve data quality, resulting in a 30% increase in data accuracy.

• Created detailed data security protocols for data access and data protection, providing layer of enhanced security for company data.

• Implemented data engineering security protocols to protect sensitive customer data and improved customer privacy processes. Automated data extraction processes and reduced manual tasks, providing an overall time savings of 75%.

• Developed Spark-based pipelines using Spark data frame operations to load data to EDL using EMR for jobs execution & AWS S3 as a storage layer.

• Created SQL code to extract and transform data for business requirements, increasing accuracy by 95%.

• Developed Tableau visuals and dashboards to provide insights into key performance trends. Environment: SQL, ETL, AWS, Mapreduce, Tableau, PowerBI, Docker, Redshift. Big data developer Jan 2020 – Mar 2021

Micron Technology India

• Design and develop SSIS packages, store procedures, configuration files, tables, views, and functions, and implement best practices to maintain optimal performance.

• Designed dimensional data models, ETL workflows and SQL queries leveraging a variety of big data technologies.

• Established and maintained technical environment for data analysis, such as databases and data warehouses in cloud environment.

• Built solutions for data collection from diversified sources such as APIs, web logs, and files.

• Collaborated with data engineers to develop data pipelines to improve data quality and accessibility. Environment: ETL, SQL, Python, jenkins, GIT.

Software Engineer June 2018 – Dec 2019

Servizon IT services Hyderabad, India

• Developing spring boot microservice architecture applications and deploy them to AWS EC2 instances using CI/CD Jenkins Pipeline.

• Utilized RESTful APIs to retrieve and update data, ensuring smooth data flow between the front-end and back-end systems.

• Configured Bamboo to handle application deployment on Cloud (PCF)and to integrate with Git Hub version control.

• Developed back-end interfaces using embedded SQL, PL/SQL packages, stored procedures, Functions, Procedures, Exceptions Handling in PL/SQL programs, and Triggers.

• Developed the UI Screens using HTML5, DHTML, XML, Java Scripts, JQuery Custom-tags, JSTL DOM Layout and CSS3. Environment: Java, SQL, HTML, EC2, Jenkins, CI/CD, PL/SQL, Java Script, GIT, Bamboo. EDUCATION

Northwest Missouri State University University Maryville, MO. M.S. in Applied Computer Science, data Science Dec 2022 Lovely Professional University India.

B.S. in Computer Science and Engineering May 2019

Contact this candidate