Senior Python Data Engineer with Spark & AWS Expertise

SAITHANUJA BEERAM

************.*@*****.*** +1-872-***-****

LinkedIn : www.linkedin.com/in/sai-beeram-834ba9357

WORK SUMMARY:

Python Developer with 6+ Years of experience in large-scale data processing, ETL pipelines, and analytics using Spark SQL, DataFrames, and RDDs.

Hands-on experience with Delta Lake, Spark Streaming, and integrating PySpark with cloud services (AWS S3, Glue, Redshift).

Skilled in optimizing PySpark jobs for performance (partitioning, caching, broadcast variables) on AWS EMR, Databricks, or on-prem clusters.

Proficient in data wrangling, transformations, and aggregations using PySpark for batch and real-time processing.

Strong expertise in Python, Django, Flask, Pyramid Framework, and PySpark.

Working experience of Pandas, NumPy, SciPy and NLP. Good working knowledge of Deep Learning Python libraries.

Involved in design of ML web applications to implement business logic.

Experience of software development in Python (libraries used: libraries- Beautiful Soup, NumPy, SciPy, matplotlib, python-twitter, OpenCV, Pandas dataframe, network, urllib, MySQL dB for database connectivity) and IDEs - VScode, Spyder, Jupiter, and PyCharm.

Having experienced in Agile Methodologies, Scrum stories and sprints experience in a Python based environment, along with data analytics, data wrangling and Excel data extracts.

Working experience with AWS cloud: EC2, S3, EFS, EBS, VPC, Lambda, Glue jobs, step functions, batch jobs, CI/CD pipeline, Neptune, ELB, ECS, and code Build.

Working experience with Docker and IAAC platforms cloud formation and Terraform.

Experienced in MVW frameworks like Django, Vue.js, Java Script, jQuery and Node.js.

Proficient in Front end development experience using HTML, XML, CSS, Vue.js, JSON and JavaScript, CSS, Bootstrap.

Expert knowledge of and experience in Object oriented Design and Programming concepts.

Experience object oriented programming (OOP) concepts using Python, JS, Veu.js, CSS and HTML .

Proficient in SQL databases MySQL, Neptune, MongoDB, Cassandra, and SQLite.

Experience in writing Sub Queries, Stored Procedures, Triggers, Cursors, and Functions on MySQL and PostgreSQL database.

Excellent experience with Python development under Linux OS (Debian, Ubuntu, SUSE Linux, RedHat Linux, Fedora).

Good knowledge of server Apache Tomcat, Web logic. Well versed in writing unit test cases and Selenium test cases.

Hands on SVN, Git, JIRA, Bitbucket, Axosoft.

Highly motivated team player with zeal to learn new technologies and always believes in giving the best of efforts in challenging environments.

Good team player and excellent leadership qualities. Hardworking, honest with good interpersonal skills.

CERTIFICATIONS:

IBM Certified: Python for Data Science, AI & Development

Certified Python Programmer

TECHNICAL SKILLS:

Languages

Python, PySpark, Vue.js, JavaScript, Shell Script, Perl

Databases

MongoDB, Cassandra, MySQL, Neptune

Web Technologies

JSP, Servlets, JavaBeans, JDBC, AWT, JSF, XML, DHTML, Node.js, Django, Flask, Pyramid, HTML, CSS, JavaScript/jQuery, PHP

Development Tools

Microsoft SQL Studio, DbVisualizer, Eclipse, NetBeans, MySQL Workbench, Sublime Text.

Methodologies

Agile/Scrum, Waterfall

Other skills

Internet of Things (IoT), Subversion, GIT, Perforce, SVN, CVS, Selenium, Jira, Bugzilla

Operating Systems

Windows variants, Linux, Mac OS

Work experience:

Paychex, NJ Date: June 2024 to Present

Python Developer AWS Cloud Engineer

Designed and implemented PySpark-based ETL pipelines to process terabytes of structured/semi-structured data from DynamoDB, S3, and Neptune, reducing job runtime by 40% through partitioning optimizations.

Developed Spark SQL queries for complex aggregations and joins, enabling real-time analytics dashboards.

Integrated PySpark with AWS Glue for serverless data cataloging and schema evolution.

Optimized Spark jobs by tuning executor memory, parallelism, and leveraging broadcast joins for skewed datasets.

Automated data workflows using PySpark + AWS Step Functions for scheduled batch processing.

Developed business logic-based features and applications using Python and deployed in AWS cloud in test driven Development.

Develop consumer-based features and applications using Python, Flask, HTML and other web technologies.

Wrote Python modules to extract/load asset data from Neptune and DynamoDB databases.

Designed and implemented a dedicated DynamoDB database server to drive the web apps and report on daily progress.

Integrated Redis-Cache with the Flask-rest-framework for reading the data faster.

Implemented RESTful Web-Services for sending and receiving the data between multiple systems.

Extensively worked on code build by installing, configuring and maintaining for Continuous integration (CI) and for End to End automation for all build and deployments.

Developed Docker containers for running multi module python-based pipeline for data transformation and extraction and integrated them and deployed them in AWS using lambda, step functions, batch jobs and CI/CD implementation.

Developed IaC platform using Terraform for CI/CD automation.

Worked on NLP to advance business logic and data driven target generation.

Implemented Bash, Perl, Python scripting to automate many day-to-day tasks.

Implemented REST API's in Python using micro-framework like Flask with SQL Alchemy in the backend for management of data center resources on which OpenStack would be deployed.

Involved in the Design of front end and backend of the application utilizing Python on Flask and Vue.js Web Frameworks.

Worked on front end framework like JS and Vue.js for development of Web applications.

Involved in development of Web Services using REST for sending and getting data from the external interface in JSON format.

Involved in various phases of the project like Analysis, Design, Development, and Testing.

Developed and Tested features of dashboard using CSS, JavaScript, Django, and Bootstrap.

Created deployment groups in one environment for the Workflows, Work lets, Sessions, Mappings, Source Definitions, Target definitions and imported them to other environments.

Tested and debugged the product in controlled and real situations and Implemented tests using pytest.

Tools/Technologies: PySpark, Spark SQL, SPython, Flask, HTML, CSS, AWS EMR, Delta Lake, Terraform, Glue, API Gateway, REST API, Spring framework, Vue.js, Amazon S3, Cloud Watch, Neptune, DynamoDB, EFS, code build, step/batch jobs, lambda functions, MS-SQL Server, BIT, Jira, Jenkins, Confluence.

Scube soft Solutions, Hyd, India Date: Jan-2021 to Nov-2022

Python/PySpark Developer

Designed front end and back end of the application utilizing Python on Django Web Framework.

Developed consumer-based features and applications using Python and Django in test driven Development.

Worked on front end frame works like CSS Bootstrap for development of Web applications.

Develop consumer-based features and applications using Python, Django, pyramid, Flask, Web2py, HTML and other web technologies.

With using of Python OS module on UNIX environment, made job cloning and forking.

Wrote Python modules to extract/load asset data from the MySQL source database.

Designed and implemented a dedicated MySQL database server to drive the web apps and report on daily progress.

Built PySpark pipelines to migrate legacy MySQL data to MongoDB and Cassandra, handling 10M+ records daily.

Used Spark DataFrames and RDDs to cleanse, transform, and aggregate raw data for ML model training (Pandas/Numpy integration).

Implemented Spark Streaming (micro-batching) to process real-time IoT sensor data with Kafka integration.

Collaborated with data engineers to debug and optimize Spark UI/logs for slow-running stages.

Integrated Redis-Cache with the Django-rest-framework for reading the data faster.

Implemented RESTful Web-Services for sending and receiving the data between multiple systems.

Extensively worked on Jenkins by installing, configuring and maintaining for Continuous integration (CI) and for End to End automation for all build and deployments.

Implemented Bash, Perl, Python scripting to automate many day-to-day tasks.

Developed the notification service by posting the JSON request in AWS API Gateway, Validating the response in Lambda by getting the data from Dynamo DB and sending the notification through AWS SNS.

Wrote with object-oriented Python, Flask, SQL, Beautiful Soup, httplib, Jinja, HTML/CSS, Bootstrap, jQuery, Linux, Sublime Text, GIT.

Implemented REST API's in Python using micro-framework like Flask with SQL Alchemy in the backend for management of data center resources on which OpenStack would be deployed.

Involved in development of Web Services using and REST for sending and getting data from the external interface in XML and JSON format.

Involved in various phases of the project like Analysis, Design, Development, and Testing.

Developed and Tested features of dashboard using CSS, JavaScript, Django, and Bootstrap.

Wrote Stored Procedures in SQL and Scripts in Python for data loading.

Involved in data extraction/loading processes with AWS and MySQL.

Tools/Technologies: Python, Django, HTML, PySpark, Spark Streaming, Kafka, CSS, AWS, API Gateway, REST API, Spring framework, AngularJS, Amazon S3, MongoDB, Cloud Watch, eclipse, MS-SQL Server, GIT, Jira, Jenkins, Confluence.

Shan Softech Solutions, Hyd, India Date: June-2018 to Dec-2020

Python Developer

Design, develop, test, deploy and maintain the website.

Designed and developed the UI of the website using HTML, XHTML, AJAX, CSS and JavaScript.

Developed entire front end and backend modules using Python on Django Web Framework.

Designed and developed data management system using MySQL.

Rewrite existing Java, C++ application in Python

Wrote python scripts to parse XML documents and load the data in database.

Generated property list for every application dynamically using python.

Created unit test/regression test framework for working/new code.

Made Django web based apps for Insurance premium calculations.

Designed the architecture of the hardware environments and methodology of the code.

Used Pandas library for statistical Analysis.

Used Subversion version control tool to coordinate team-development.06

Responsible for debugging and troubleshooting the web application.

Involved in entire lifecycle of the projects including Design, Development, and Deployment, Testing and Implementation and support.

Used PySpark for data transformation tasks in data management systems.

Developed application logic using Python, Java script, Java.

Designed and developed the UI of the website using HTML, AJAX, CSS and JavaScript.

Used Java Server Pages for content layout and presentation.

Created Data layer in MYSQL and Oracle Database.

Extracted and loaded data using Python scripts and PL/SQL packages

Debugging and testing of the applications

Managed version control using Subversion.

Tools/Technologies: Python, Django, PySpark, Java, J2EE, HTML, XHTML, CSS, AJAX, JavaScript, MySQL, Oracle, PL/SQL, Unix Shell Scripting, Red Hat Linux, WebLogic Application Server.

EDUCATION:

Master’s in Business Analytics at Trine University, Detroit, Michigan (GPA: 3.7)

Bachelor of Commerce in Computers, Osmania University (GPA: 3.8)

Contact this candidate