Data Engineer Sql Server

Swathi Kodoori Ph: 410-***-**** Email: ad2a98@r.postjobfree.com

Senior Data Engineer

Specialize: As a Big Data Developer Implements Huge Data Processing & Provide Best Solutions

3

4

7+ Yrs of IT experience in Data modeling, Data integration, Data Migration in designing, implementation, development, testing and maintaining the ETL components in building Data Warehouse & Data marts across domains like Communications, Health Care, Manufacturing, Retail and Insurance. Experienced working on development projects at all stages from ideation to development and delivering the completed data warehouse. Strong experience in Hadoop ecosystem including its components. Also, expertise in data extraction, Transformation and loading data using ETL tools such as Informatica Power Center and Microsoft SQL Server Integration Services (SSIS). Excellent interpersonal skills, leadership ability, customer service aptitude and business acumen.

Data Warehousing Hadoop Data Lake Data Analysis Dimensional Modeling Hadoop Architecture Agile/Scrum Methodologies Root Cause Analysis Reporting Testing

Technical Proficiencies:

Big Data Ecosystems

Hadoop, Spark2.1.0+, Google Cloud Platform (GCP), Amazon Web Services (AWS), Scala, PySpark, Airflow, HDFS, Hive, YARN, Sqoop, HBase, Kafka 0.10.2, Zookeeper, Hue, Oozie, Jupyter Notebooks, Ambari, Elastic Search, VGrid Yahoo, Hortonworks, Cloudera, Kibana, Maven, Github, Jenkins, Spark streaming, Eclipse, Intellij

Databases

Oracle 11g/10g, MS SQL Server 2008/2012/2014, Teradata, Cassandra, NoSQL, Postgresql

Tools/Utilities

Informatica Power Center 9.x/10.x, SQL Server Integration Services (SSIS) 2012, Google Looker Studio, Tableau, Visual Studio, Putty, MS Visual Studio, Tableau, TOAD, Putty, SQL Loader, Visio, SQL Plus, Query Analyzer, Microsoft office, TFS

Programming

Unix Shell Scripting, Windows Batch Scripting, SQL, PL/SQL, TSQL, NoSQL, Perl Scripting

Job Scheduler

Informatica Scheduler, Maestro, Crontab

Professional Experience:

Sr. Data Engineer - Verizon Communications, Inc, Irving, TX Oct 2019 – Till Date

Involved in the requirement gathering, project documentation, design document, production deployment and support activities in collaboration with Yahoo Team.

Worked on Google Cloud Platform(GCP) and Hadoop VGrid for Yahoo

Implementing Spark Scala & PySpark for programming application in performing ETL using Spark Core, Spark SQL for both batch processing and streaming for interactive analysis requirement by configuring Kafka with Spark Streaming to collect the data from the Kafka.

Worked on migration project for platform change from NDL to VCG Yahoo and utilize VGrid environment

Experience in writing Spark applications using scala for Data validation, cleansing, transformations and custom aggregations.

Involved in converting Hive/SQL queries into Spark transformations using Spark Data Frames, DataSet, RDDs and APIs

Worked with different data sources like Avro data files, XML files, Json files, SQL server and Oracle to load data into Hive tables

Created hive Tables with dynamic and static partitioning including buckets for efficiency.

Worked in NOSQL column oriented databases like HBase Cassandra and its integration with Hadoop cluster

Prioritize daily workflow and demands on quality, time and sources.

Data Engineer - MetLife Inc, Cary, NC Nov 2017 – Oct 2019

Responsible for handling business requirements, technical requirements and all the necessary details from source team such as nature of data, ingestion type to enable us in smooth ingestion of the systems.

Been part of Design Reviews, Sprint Planning’s & Daily Project Scrums and also worked with offshore teams.

Involved in creating a framework which enables us to handle all types of data including unstructured and structured (XML, CSV, FAC, JSON, XML) by coding in Scala.

Used SQL, PL/SQL and PySpark for development of Procedures, Functions, Packages and Triggers for ETL data loads pipelines

Worked in designing, enhancing and scheduling SSIS Packages for transferring data from multiple data sources to SQL server 2014.

Worked on POC for informatica connectors with Bigdata Hadoop Horton works.

Created change tickets and RITM's for making changes in the higher environments. Worked in deployment and postproduction deployment verifications activities support.

Experienced in production support to monitor daily jobs for data ingestion to publish the files to the downstream. Also worked as a L2 Support by communicating with AMS team for the changes in production.

Data Engineer - Carefirst Blue Cross Blue Shield, Baltimore, MD June 2016 – Mar 2017

Involved in all phases of SDLC from requirement gathering, design, development, testing, Production, user training and support for production environment.

Gained understanding Hadoop Architecture and its related components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node programming paradigm

Involved in importing data from MySQL to HDFS using SQOOP.

Implemented the workflows using Apache Oozie framework to automate tasks.

Creating Hive tables, loading with data and writing hive queries which will run using spark.

Implemented Spark using Scala and utilizing Data frames and Spark SQL API for faster processing of data.

Initialized utilization of Data Warehouse ETL software during conversion of data to Oracle DB by using pyspark as coding.

Experience in UNIX Shell scripting & Windows Batch Scripting for Parsing Files & automation of batch ETL jobs.

Participated in weekly status meetings and conducting internal and external reviews as well as formal walk through among various teams and documenting the proceedings.

ETL Informatica Developer - Talen Energy, Allentown, PA Feb 2016 – June 2016

Worked with Data Architects, Business Analysts and Independent Testing Team.

Responsible for analyzing, programming and implementing modifications to existing systems required by changes in the business environment using Informatica Power Center

Converted Business Specification Document to Technical Specification documentation for developing Source and Target mappings

Worked on data warehousing techniques for Slowly Changing Dimension phenomenon, surrogate key assignment, Normalization & De-normalization, Cleansing, Performance Optimization along with the CDC (Change Data Capture)

Involved in the creation of Oracle Tables, Views, Materialized views and PL/SQL stored procedures and functions.

Written SQL Queries and PLSQL stored procedures to define identify and validate the code

Uploading the Performance Test Plan, Test Scripts, Scenarios and Final Reports in the Quality Center for every application.

ETL Informatica Developer - Benjamin Moore & Co., Montvale, New Jersey July 2015 – Nov 2015

Formulate and define best practice programming standards that meet regulatory compliance requirements for Implementation, Support and Upgrade Projects.

Working closely with Functional Leads, Business Analysts to determine optimal programming solutions to functional requirements in O2C, R2R and P2P process areas

Worked on all stages of Developing, Configuring, Fine-tuning ETL workflows, Code Reviewed, fixed bugs, testing and deployment

Uploading the Performance Test Plan, Test Scripts, Scenarios and Final Reports in the Quality Center for every application.

Prepared Traceability Matrices to track the requirements with the test cases and make sure none of them have been missed. Involved in Unit testing Interacted with QA team for system/integration testing

Worked on Informatica Power Center Designer - Source Analyzer, Warehouse designer, Mapping Designer & Mapplet Designer and Transformation Developer.

Have experience in developing mappings according to business rule, migrating to QA AND production, naming conventions, mapping design standards and good knowledge in Datawarehouse, PL/SQL, ODBC connections etc.

Informatica Developer - Nationwide Insurance, Columbus, OH Oct 2014 – Feb 2015

Worked with Business analysts and the DBA for requirements gathering, business analysis and designing of the data marts.

Preparation of technical specification document for the development of Informatica Extraction, Transformation and Loading (ETL) mappings to load data into various tables in Data Marts and defining ETL standards using Informatica Power Center 9.5.1

Processes to generate Daily, Weekly and Monthly data extracts were developed and the data files were sent across to the downstream applications.

Created and scheduled Sessions, Jobs based on demand, run on time and run only once

Monitored Workflows and Sessions using Workflow Monitor.

Performed Unit testing, Integration testing and System testing of Informatica mappings

Experience in Performance tuning of SQL queries.

Writing Perl script to load data from sources to staging tables, to create indirect file list, generate parm files for respective paths.

Educational Details:

Master of Science, Electrical Engineering – Texas A & M University, Kingsville Aug 2012 – Dec 2013

Contact this candidate