Data Developer

VIJAY B

609-***-**** · adbv93@r.postjobfree.com

An aspiring Bigdata Developer with over 5 years of experience in Finance, Retail & Telecommunication domains. A team player attitude with effective communication, geared to increase collaboration and team spirit. A Strong Education & hands on experience with aided technical understanding to support team’s progress. Experience working in distributed environments and culturally diverse team dynamics.

Overall 5+ years of IT experience as Bigdata/Hadoop developer, ETL Developer and Java Developer.

Certified Scrum Master (CSM) and available to polish team on agile frameworks.

Hands on experience in Hadoop ecosystem including HDFS, Spark, MapReduce, Hive, Sqoop, Oozie, Flume, Kafka.

Excellent knowledge on Hadoop Components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, YARN and MapReduce programming paradigm.

Expertise in Java and Scala languages.

Experience in Creating Hive tables and load the tables using Sqoop and processed data using Hive QL.

Hands-on experience on RDD architecture, implementing Spark operations on RDD.

Having knowledge on Spark Streaming to ingest data from multiple data sources into HDFS.

Hands on Experience on Stream Processing including Storm and Spark streaming.

Experience in data cleansing using Spark Functions.

Familiar with Spark Context, Spark SQL, Data Frame and Pair RDD's.

Hands-on experience in using relational databases like Oracle.

Experience in importing and exporting the data using SQOOP from HDFS to Relational Database systems and vice-versa.

Hands on Experience in creating tables, partitions and buckets in Hive.

Extending Hive Core functionality by writing UDF’s for Data Analysis.

Extensive experience in working with various distributions of Hadoop like enterprise versions of Cloudera (CDH5) and Hortonworks.

Good Knowledge in Amazon Web Services.

Extensive programming experience in developing Java applications using Java, J2EE and JDBC.

Expertise in Informatica client tools – Source Analyzer, Mapping designer, Transformations, Informatica Repository Manager.

Experience in Connected and Un-Connected Look up Transformations in the Designer of Informatica PowerCenter.

Well versed with UNIX and Linux command line and shell script.

Experience in collection of JSON data into HDFS using Flume and processed the data using Hive and experienced in using Sequence files, AVRO file, Parquet file formats.

Managed the projects based on waterfall and Agile-Scrum Methods.

Technical Skills:

Professional Experience:

BCBS, Jacksonville, FL Dec’18 – Present

Big Data Developer

Description: The main objective of this project is to retrieve data from different sources and perform required operations using Apache Spark to delete duplicate data and provide it to business users to analyze.

Responsibilities:

Worked on loading structured and semi-structured data into HDFS using Sqoop.

Involved in copying large data from Amazon S3 buckets to HDFS using.

Used big data analytical and processing tools Hive, Spark Core, Spark SQL for batch processing large data sets on Hadoop cluster.

Implemented Spark SQL queries, Hive queries and performed transformations on data frames.

Performed data Aggregation operations using Spark SQL queries.

Implemented Hive Partitioning and bucketing for data analytics.

Used GitHub as code repository and version control system.

Used Maven Build tool for code repository.

Involved in working with Sqoop to export the data from Hive to S3 buckets.

Environment: HDFS, Apache Spark, Apache Hive, Scala, Oozie, Apache Kafka, Apache Sqoop, Agile Methodology, Amazon S3.

Frontier Communications, Allen, TX Sept’ 17 – Nov’ 18 Big Data Developer

Description: The goal of the Project is to Develop an Application to resolve the customer’s credit bureau disputes. It provides an audit system to track how the disputes case was investigated and capture the logic behind how the dispute case resolved. Architecture of this project is developed in Micro services. Application runs in AWS environment and developed using Agile Development Methodology.

Responsibilities:

Involved in the requirements Analysis meetings, grooming sessions and understand the requirements to develop the modules.

Involved in Designing ETL Pipeline in the cloud for Faster Data Processing on Daily and Monthly Basis.

Worked on ingesting data from different sources.

Created hive UDFs for formatting data in SCALA.

Responsible for writing SQL queries for data manipulation.

Working knowledge on batch and Streams.

Developed Spark jobs using Scala and Spark SQL to evaluate Risk Score.

Strong Knowledge on UNIX/LINUX commands.

Implementing the solution using AWS services S3, Redshift, Aurora.

Implemented modules using Core Java APIs, Java Collection, Java 8(Streams) and object - oriented designs.

Development of spring boot Microservices and deploy them in the docker container.

Deploy Services on different Environments cluster using Jenkins CI/CD pipeline.

Working knowledge on running a docker -enable application on an Amazon ECS cluster behind the load balancer.

Checking of AWS logs and docker logs if any issues during deployment.

Used spring core annotations for dependency injection.

Designing and developing Restful API and services using best practices to interact with in the micro services.

Effective tracking of code using Log4j2.

Wrote unit test cases using Junit and Mockito.

Involved in doing Performance Testing using JMeter.

Environment: Hadoop, Spark, Scala, Maven, Docker, EC2, SQS, Redshift, S3, Cloud Watch Logs, Splunk dashboard, GIT, PostgreSQL, JIRA.

EA Sports, Bengaluru, India Oct’ 15 – Aug’ 17

Big Data/ Spark Developer

Description: The aim of this project was to load the huge amount of data from various sources into the data lake then processed using Spark and stored in HDFS for analysis.

Responsibilities:

Worked on Cloudera distribution.

Involved in extracting customer data from various data sources to HDFS data lake which include data from relational RDBMS and csv files.

Loaded and transformed large sets of structured and semi-structured data using Spark.

Involved in working with Sqoop for loading the data from RDBMS to HDFS.

Extensively used Spark Core, Spark SQL.

Developed Spark applications Using Scala as per the Business requirements.

Used Spark Data Frame Operations to perform required validations on the data.

Responsible in performing sort, join, aggregations, filter, and other transformations on the datasets.

Created Hive tables and working on them for data analysis to cope up with the requirements.

Implemented Hive Partitioning and bucketing for data analytics.

Analyzed the data by performing HQL, Spark SQL.

Loaded the Cleaned Data into the hive tables and performed analytical functions based on requirements.

Involved in creating views for the data security.

Involved in the performance tuning of spark applications.

Worked on Performance and Tuning operations in Hive.

Created custom workflows to automate Sqoop jobs monthly.

Involved in Agile methodologies, daily Scrum meetings, Sprint planning.

Experienced in using version control tools like GitHub to share the code snippet among the team members.

Environment: HDFS, Hive, Apache Sqoop, Spark, Scala, YARN, Agile Methodology, Cloudera, MySQL.

ICICI Bank, Hyderabad, India April’ 14 – Sept’ 15 Java Developer

Description: The aim of this project was to load the huge amount of data from various sources into the data lake then processed using Spark and stored in HDFS for analysis. Gained substantial knowledge about the concepts, design advantages, traps and pitfalls of successful object- relational mapping EJB components

Responsibilities:

Involved in requirement collection and analysis.

Worked on developing front-end screens using JSP, Struts and HTML

Involved in implementing persistent data management using JDBC.

Participated in problem analysis and coding

Design and coding of screens involving complex calculations on various data windows accessing different tables on the oracle database.

Developed screens for Patient Registration, Inventory of Medicines, Billing of Services and Asset Modules.

Used JSF framework in developing user interfaces using JSF UI Components, validate Events and Listeners.

Created several pieces of the JSF engine, including value bindings, bean discovery, method bindings, event generation and component binding.

Involved in unit testing, integration testing, SOAP UI testing, smoke testing, system testing and user acceptance testing of the application.

Wrote stored procedures, Database Triggers.

Involved in debugging and troubleshooting related to production and environment issues Performed Unit testing.

Environment: JSP, Servlets, SQL, PL/SQL, WebSphere Application Server, Oracle 9i, JavaScript, windows XP, Unix shell Script, eclipse, MongoDB.

Education:

The University of South Alabama July 2019

Master of Science – Computer Information Systems

Jawaharlal Nehru Technological University, Kakinada March 2014

Bachelor of Technology – Computer Science

Contact this candidate