Data Java Developer
Resume:
Mani Shankar Ravuri
*****************@*****.***
PROFESSIONAL SUMMARY
·BigData and Java Developer with around 6 years of experience in Requirements Analysis, Design, Development, Implementation, release & deployment and cloud implementation.
·Experience in working with BigData components like HDFS, Apache Spark, Hadoop, Apache Hive
·Involved in designing and deploying applications utilizing almost all of the AWS stack (Including Amazon EC2, Elasticsearch service, S3, RDS, IAM, AWS LAMBDA, AMAZON EMR, AWS Glue, Athena, Cloud watch, VPC, Cloud Formation) focusing on high-availability, fault tolerance, and auto-scaling in AWS CloudFormation
·Experience in working with GOOGLE cloud platform stack (google cloud storage, google cloud composer, Apache airflow, Big Query, cloud build).
·Experience with Cloud formation templates and Terraform for deploying AWS resources.
·Experience in converting business process into RDD transformations using Apache Spark and Python.
·In - depth understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, YARN, RDD and PySpark.
·Hands on experience in various Bigdata application phases like data ingestion, data analytics and data visualization.
·Experience in manipulating the streaming data to clusters through Kafka.
·Experience in developing ETL pipelines in AWS and Google cloud platforms.
·Experienced in writing bash, shell and Python scripts.
·Composed Docker images and maintained docker containers using Kubernetes clusters.
·Experience in developing Java applications with Java versions 1.6, 1.8 and 1.11
·Experience in developing applications in Micro Service Architecture with technology stack as REST, Spring BOOT, Reactive Java programming, Docker and Amazon ECS container service.
·Strong Working experience in Development of several modules of spring like Spring IoC, Spring MVC, Spring JDBC, Spring BOOT, Spring INTEGRATION and Spring DAO.
·Good experience in working with micro service architecture using Spring BOOT and REST services.
·Working experience with various code editor tools like IntelliJ, PyCharm and Visual studio code.
·Experience in creating RESTful Web Services, and Data Transfer using JSON, XML etc.
·Sound RDBMS concepts and extensively worked with Oracle, SQL server and MySQL data bases and good working experience with NoSQL data bases like MongoDB and ElasticSearch.
·Familiar with designing, coding Stored Procedures, Triggers, Cursers, Functions using PL/SQL.
·Developed and deployed web services to interact with partner interfaces, and written client interfaces, and implementation classes to consume the web services.
·Hands on experience on using Control-M and cron schedulers.
·Good Knowledge of using different protocols like FTP, SFTP, FTPS, SSH, HTTP, HTTPS.
·Experience in developing Test Cases using Junit, Mockito, Easy Mock, Power Mock and Cucumber.
·Knowledge in Implementing Continuous Integration using SVN, GIT-BITBUCKET and GITLAB
·Good experience on working with build tool Maven and deployment tools like Jenkins and Bamboo.
·Experience in managing the artifacts using Nexus repository and Virtual Machines using AWS.
·Excellent technical, communication, analytical and problem-solving skills and ability to get on well with people including cross-cultural backgrounds and trouble-shooting capabilities.
SKILLS AND AWARDS:
·Technologies: Java 1.6, 1.8 and 1.11, Python 3.6 and 3.7, SQL, Groovy and Shell scripting
·Cloud services: Amazon Web Service (AWS), Google cloud platform (GCP), EC2, VPC, S3, Cloud watch, IAM, Lambda, Amazon EMR, AWS GLUE, Athena, API GATEWAY, Docker, GCS, Big Query, Google cloud composer, Apache Airflow, Dataflow, Kubernetes
·Big Data tools: Apache Spark, PySpark, Spark SQL, Hadoop, HDFS, Hive
·Code Editors: IntelliJ, Spring tool suite (STS), PyCharm, Visual Studio
·Build & Version control Tools: Maven, Gradle, GitHub, SVN, Jenkins, GitLab
·Operating Systems: Linux, Windows
·Network & FTP protocols: SCP, SSH, SFTP, TCP/IP, DNS, LDAP, HTTP, HTTPS
·Databases: MySQL, SQL Server, Oracle, Elastic search
·Tracking Tools: Agile, JIRA, Kanban, Rally
·Web Services and Frameworks: REST, SOAP, various modules of SPRING, HIBERNATE, JPA
·Code Quality coverage: SonarQube, Crucible
·Test Automation: Junit, Mockito, Power Mock, Selenium, Cucumber, Behave
WORK EXPERIENCE
Sep-2019 – Present – Data Engineer, Cigna, Plano, TX
Tasks performed:
·Design and Develop ETL Processes in AWS Glue to migrate Historical data from external sources like S3, ORC/Parquet/Text Files into AWS Elastic search
·Data Extraction, aggregations and consolidation of Historical data within AWS Glue using PySpark, Spark SQL and Python.
·Create external tables with partitions using AWS Crawler and AWS Athena.
·Create user defined functions UDF for transformations in PySpark.
·Data Ingestion and transformation with extensive use of spark functions and pyspark.
·Performance tune long running AWS Glue and Spark Jobs.
·Imported Data from Amazon s3 into spark dataframe and performed transformations and actions on top of dataframes and written them back to s3 in Parquet and Avro formats.
·Written custom UDF’s in Python for custom data transformation and ETL.
·Worked with various file formats like AVRO and PARQUET and various compression formats like Snappy.
·Develop AWS Lambda functions in Node JS to read data from Elastic search and set up API gateway as trigger for Lambda.
·Used Terraform to deploy AWS stack for creating IAM roles, lambda functions and Glue Scripts
·Used Docker and SAM CLI to develop and test lambda functions
·Developed REST API with Spring boot using Spring reactive programming with Java 11
·Used IntelliJ and PyCharm as code editors and GITLAB and Jenkins for CI/CD
Environment: Java 1.8, Python 3.6, AWS, Amazon EC2, Amazon S3, Amazon IAM, Amazon GLUE, Amazon Cloud watch, Amazon VPC and subnets, HIVE, Apache Spark, Spark SQL, PySpark, Terraform, Kafka, Spring Reactive Programming, REST API, Elastic Search, GITLAB, Jenkins
Apr-2019 – Sep-2019 - Full stack Developer, Dearborn National, Richardson, TX
Tasks performed:
·Involved in the analysis, design, and development and testing phases of Software Development Life Cycle (SDLC).
·Worked in agile framework as an individual contributor Responsibilities include - Interaction with Business team in story grooming, reviewing story/acceptance criteria.
·Implemented REST Microservices using spring boot.
·Deployed, Scaled, Configured, wrote manifest file for various Microservices in PCF.
·All the functionality is implemented using Spring IO / Spring Boot and Hibernate ORM. Implemented Java EE components using Spring MVC, Spring IOC, spring transactions and spring security modules.
·Used JPA as a DAO layer to fetch the data from MySQL database.
·Designed UI screens using AngularJS.
·Developed story/task by following TDD/BDD and Pair programming concepts. Providing daily status in scrum meetings with the client. Mentored new team members on effective usage of blend of Spring Boot /JPA and Java
Environment: Java 1.8, PCF, Spring IoC, AOP, Spring DAO, REST, Groovy, Jenkins, Docker, GITHUB, Kubernetes, Angular JS, Hibernate, Spring Boot, Spring Batch
Sep-2018 – Apr-2019 - Data Engineer, CTP/HPE, Plano, TX
Tasks performed:
·Developed ETL pipeline for getting google advertiser’s data into Google cloud
·Created DAG’s (Directed Acyclic Graph) using Apache airflow and run it on Google composer
·Worked on various airflow operators like Bash operator, Python operator, Big query operator, GCS operator, GCS to BQ operator, Branch Python operator, Python virtual environment operator etc. using Python scripts for creating ETL pipelines and deploying them onto google container engine (Docker)
·Brought up Google cloud composer environment with Kubernetes cluster and deployed DAG’s into the composer environment
·Created Docker images and deployed them on to Google Kubernetes cluster
·Wrote Apache beam scripts using Python for data transformations and integrated them to run on the Google cloud Dataflow environment
·Created CI/CD pipelines using cloud build, GitHub and Jenkins
·Developed SQL queries for loading and reading data from Google Analytical database Big Query
·Used various Airflow hooks (FTP hook, AWS hook, GCP hooks etc.) for integrating with external cloud platforms like Amazon and Azure for developing complex ETL applications
·Monitored logs on the Google Stack Driver and alerting and monitoring
·Developed cloud functions and created triggered them based on stack driver events
·Created service accounts for project level access and OAuth 2 for accessing external platforms
·Developed Python and bash scripts for developing the ETL pipelines
·Implemented REST Microservices using spring boot
·Generated Metrics with method level granularity and Persistence using Spring AOP and Spring Actuator
·Wrote SQL queries to retrieve data from the database using JDBC
·Created unit tests for testing DAG’s using Dag Bag API of airflow and integrating it with PyTests
·Developed integration tests using Behave (Behavior driven development - BDD)
·Monitored the cloud builds for application logs and involved in fixing issues for CI/CD builds
Environment: Google cloud platform, cloud composer, Apache airflow, Big Query, Python 2.7 and 3.6, shell scripting, Behave, Python unitest framework, cloud build, SQL.
July-2017 – Sep-2018 - Data Engineer, The Vanguard Group, Malvern, PA
Tasks performed:
·Involved in the analysis, design, and development and testing phases of Software Development Life Cycle (SDLC).
·Played a vital role to analyze BigData technologies to build Big data platform on Amazon EMR.
·Developed cloud formation template for Data ingestion EMR cluster with IAM roles and s3 Bucket policies and permissions, defined security groups for EC2 instances to communicate with various applications running on EMR cluster like Spark, Hadoop, Hive etc.
·Developed cloud formation template for Data query EMR cluster having autoscaling EC2 instances with Presto installed on them which is used to connect to Tableau for viewing data.
·Monitored health checks of EC2 instances using cloud watch.
·Designed the real-time analytics and ingestion platform using Spark.
·Used Ansible playbooks to install default software on EC2 instances running on EMR cluster.
·Developed shell scripts and python scripts for FTPing files and file conversions.
·Developed LFTP script to move files from on prem to Amazon s3.
·Used Amazon CLI commands for various AWS services in shell scripts.
·Created Tables using HIVEql on top of files stored on amazon s3.
·Written INSERT/READ queries using HIVEql to load data into HIVE tables.
·Logged application logs to the splunk server.
·Used Amazon S3 service to store files and archived them to Glacier.
·Used Tableau, developed some custom screens and published them on to Tableau server.
·Managed GitHub repositories and permissions, including branching and tagging
·Strong understanding of Java/J2EE project structures and developed executable jars for file parsers using various modules of Spring Framework which are executed on EMR cluster through shell scripts.
·Mastery of source code management tools like GIT/BITBUCKET & Subversion/SVN, and strong experience with Branching and Merging in this environment.
·Created Control-M jobs to bring up the data ingestion EMR cluster.
·Used MAVEN to manage dependencies and deploy them to the nexus repository.
·Setup Bamboo tool for continuous integration and deployment.
Environment: Java 1.8, AWS, Amazon EC2, Amazon S3, Amazon IAM, Amazon EMR, Amazon cloud formation, Amazon Cloud watch, Amazon VPC and subnets, Hadoop, HDFS, Apache HIVE, PRESTO, Apache Spark, Tableau, Ansible, Python, Bash, Splunk, HQL, Maven, Bitbucket, Bamboo, Spring IoC, Agility tool.
July-2015 – July-2017 - Java Developer, The Vanguard Group, Malvern, PA
Tasks performed:
·Involved in the analysis, design, and development and testing phases of Software Development Life Cycle (SDLC).
·Use Agile (SCRUM) methodologies for Software Development.
·Used Spring on the server side for IOC, AOP and transaction management.
·Developed RESTful web service for CRUD operations using JAX-RS.
·Written PL/SQL stored procedures and used Spring-Hibernate DAO to fetch data into tables.
·Used Spring BOOT to start spring application and Spring INTEGRATION for file processing.
·Developed JUNIT 4.0 and Mockito test cases for unit testing and cucumber test cases for Integration testing.
·Created Cucumber feature files comprising of various Test Scenarios and Java step definitions for TDD (Test Driven Development).
·Written automated test cases using Selenium and cucumber.
·Monitored the error logs using HVLM and involved in fixing the issues.
·Used MAVEN as build tool for the development of build artifacts on the source code.
·Used AngularJS, HTML5 and CSS3 to design the user interface screens
·Mastery of source code management tools like GIT/BITBUCKET & Subversion/SVN, and strong experience with Branching and Merging in this environment
·Setup Jenkins tool to integrate the JAVA project and maintained Jenkins with continuous integration and deployment.
·Used Oracle11g and DB2 as backend databases and STS as IDE
Environment: Java 1.8, Spring Framework 3.6, REST, AWS, JAX-RS, JAXB, Oracle 11g, DB2, SQL, PL/SQL, SQL-Developer, AngularJS, JIRA, SVN, Bitbucket, Jenkins, Bamboo, SonarQube.
Jan-2013 – Nov-2013 - Java Developer, Data Soft IT Solutions, Visakhapatnam, India
Tasks performed:
Used Agile (SCRUM) methodologies for Software Development
Used JSP MVC, developed front-end screens using AngularJS
Used Spring on the server side for IOC, AOP and transaction management
Used Oracle 10g as the back-end database
Involved in creating the PL/SQL procedures for the database
Developed RESTful web service for CRUD operations using JAX-RS
Involved in writing SQL queries for transactional operations in Oracle Database
Involved in writing DAO classes to handle data from database and inject it to web service
Developed JUNIT 4.0 and Mockito test cases for unit testing
Logged log messages to server using log4j
Used SVN for version control and used Eclipse for development
Performed testing in all non-prod regions and supported production during releases
Environment: Java 1.6, Spring Framework 3.6, REST, JAX-RS, JAXB, Oracle 10g, SQL, PL/SQL, SVN, Jenkins
Education:
Master's degree in Electrical Engineering Technologies/Technicians, University of Missouri-Kansas City
Contact this candidate