Post Job Free

Resume

Sign in

Data Engineer

Location:
Brooklyn, NY
Posted:
March 21, 2019

Contact this candidate

Resume:

SURAJ DEVAL

612-***-****

Professional Summary:

Over 5 years of IT Experience in analysis, design, development, implementation and troubleshooting of Data warehouse applications.

Expertise in building Enterprise Data Warehouses (EDW), Operational Data Store (ODS), Data Marts, and Decision Support Systems (DSS) using Multidimensional and Dimensional modeling (Star and Snowflake schema) Concepts.

Significant Multi-dimensional and Relational data modeling experience, Data Flow Diagrams, Process Models, ER diagrams with modeling tools like ERWIN & VISIO

Involved in all phases of data warehouse project life cycle. Designed and developed ETL Architecture to load data from various sources like Oracle, Flat files, XML files and MS SQL Server into Oracle, Teradata, XML, and SQL server targets.

Experience in writing complex SQL queries involving multiple table inner joins and outer joins, temporary tables and table variables.

Created views to facilitate easy viewing implementation.

Developed procedures for database monitoring, maintenance, back up, recovery and automated alerts.

Review specifications of new modules or reports.

Design; implement modules in the Data warehouse using Informatica and IBM Infosphere Data Stage .

Write the design documentation.

Ensure data consistency and integrity. Maintain the software.

Provide Support on daily jobs (monitoring, troubleshooting, root cause analysis, preventive & corrective actions)

Understanding user requirements and functional specifications and develops solutions which meet these needs and specifications.

Experience in creating complex stored procedures, effective triggers, and user defined functions to facilitate efficient data manipulation and consistent data storage.

Self Motivated with Excellent communication skills with a strong focus on Client oriented service and ability to work excellently as an individual and also as part of a team with minimum amount of supervision.

Demonstrated expertise in utilizing ETL tool Informatica power center 10.x/9.x/8.x/7.x/6.x/ for developing the Data warehouse loads as per client requirement.

Highly experienced In data modelling to support business requirement and Informatica MDM velocity recommendations and best practices.

Expertise in MDM Project implementation .

Expertise in SAP S4 HANA data extraction through the XML files.

Extensive experience in handling various XSD’s with multiple nodes.

Extensive experience in developing mappings using various transformations like Source Qualifier, Expression, Lookup, Aggregator, Router, Rank, Filter and Sequence Generator transformations and various reusable components like Mapplets and reusable transformations.

Excellent knowledge of Informatica Administration. Involved in grid management, creation and up gradation of repository contents, creation of folders and users and their permissions.

Extensive knowledge in developing Teradata, Fast Export, Fast Load, Multi load and BTEQ scripts. Coded complex scripts and finely tuned the queries to enhance performance.

Created profiles, rules, scorecards for data profiling and quality using IDQ

Used Infa Data Quality for addresses and names clean-ups and developed error handling & data quality checks to pull out the right data

Used IDQ to cleanse and accuracy check the project data, check for duplicate or redundant records.

Profound knowledge about the architecture of the Teradata database.

Experience in writing PL/SQL procedures for processing business logic in the database. Tuning of SQL queries for better performance.

Extensive experience in implementation of Data Cleanup procedures, transformations Scripts, Triggers, Stored Procedures and execution of test plans for loading the data successfully into the targets.

Good experience in designing and developing audit,error identification and reconcile process to ensure the Data Quality of Data warehouse.

Good experience in working with cross-functional teams to provide practical solutions.

Good experience in root cause analysis, testing, reporting and logical design.

Supported data governance in handling the data quality and integrity of their respective projects.

Excellent knowledge in identifying performance bottlenecks and tuning the Informatica Load for better performance and efficiency.

Experience in UNIX shell scripting, CRON, FTP and file management in various UNIX environments.

Strong understanding of Data warehouse project development life cycle. Expertise in documenting all the phases of DWH projects.

Extensive experience in communicating with business stakeholders to understand business needs requirements

Education:

Bachelor of Technology Computer Science, JNTU, Hyderabad, India. - 3.5 gpa

Masters in Computer Science, Western Michigan University, Kalamazoo MI -3.2 gpa

Technical Skills:

Databases

Oracle 7.x/8.x/9i/10g/11g, SQL Server 2008/2005/2003/2000, Teradata V2R4,SQL Assistant, My SQL 5.0/4.1Editors (SQL Navigator, Toad)

ETL Tools

Informatica (Power Center 10.2/ 9.1/8.6/8.5/8.1.2/8.1.1/7.1.1/ 6.2 / 5.1

And Power Mart 6.2 / 6.0 / 5.1 IBM Infosphere Data Stage 9.1/10.2/11.7/SSIS/SSRS

Data Modeling Tools

ERWIN r 7/r4, Visio 2007/2010, ER/Studio

Programming Skills

Shell Scripting (K-Shell, C-Shell), PL/SQL, SQL,JAVA (Eclipse IDE and Net Beans IDE), HTML, JAVA Script,CSS

Methodologies

Scheduling Tools

Data Modeling – Logical, Physical

Dimensional Modeling - Star / Snowflake

Autosys, Informatica Scheduler

Reporting Tools

Cognos, Business Objects XI, MS SQL Server Reporting services 2005(SSRS), Developer 2000(Forms 5.0, Reports 6i), Crystal Reports 10, Crystal Reports 2008

Operating Systems

UNIX (Sun-Solaris, HP/UX), Windows 95/98/00/NT/XP

PROFESSIONAL EXPERIENCE:

Department of Finance – NYC State July’18 – Present

Senior Informatica Developer

In an effort to maximize efficiency, lower cost and improve the overall experience with the City, the Department of Finance (DOF) is undertaking a transformative initiative, known as the Citywide Payments Services & Standards (CPSS) Program, to make the entire payment process easier for any individual or entity making payment to the City and optimize the City’s management of the full life cycle of accounts receivable, from origination through to settlement. The Citywide Services & Standards Data Repository is where all current receivables will be consolidated in a way that makes it easy for them to be passed to one or more third-party service provider(s) and for the payment information to be returned to the proper City systems for reconciliation purposes. From a City Agency System point of view, this project will require the integration of the agency system with the CPSS, in order for the agency to be able to send receivables to the CPSS and for the agency to receive payments through the CPSS against those receivables, as well as against retail payments.

Worked on Agile methodology

Designed complex mapping for the property tax system consisting of sensitive credit card data.

Users used to check their account balance via NYCServ, NYCe-File where the file was being generated through informatica .

Developed rules and reusable mapplets to use in different mappings .

Migrated all the informatica code from 9.6.1 to 10.2.0.

Created new mappings and enhancements to the old mappings according to changes or additions to the Business logic.

Implemented SCD1, SCD2, SCD3 type maps to capture new changes and to maintain the historic data.

Collaborate with the MDM Service team, SOA Team, Data Governance, and Business Partners to define functional and technical requirements for MDM technology

Define/Design MDM solution/architecture.

Create the MDM implementation roadmap

Handled the MDM application and technical architecture analysis, design, and implementation, and ensure that the product fulfills the requirements.

Worked on developing FTP shell scripts to move the files and manipulate them.

Worked on few python shell scripting to migrate from perl scripting .

Worked with the functional team on gathering the data and worked closely with the business team to analyze the data and plan for the design.

Used Oracle Database for loading and extracting data.

Automated workflows through Unix shell scripts.

Performance tuning for the traditional jobs that has been running on the CPSS database for property taxes related jobs.

Environment: Informatica Power Center 9.6.1/10.2.0 PL/SQL,, MS SQL Server 2010, Oracle11g,Flat Files, Windows XP, UNIX, PL/SQL, XML,XSD .

Tapestry (Coach, Kate Spade, Stuart Weitzman) Dec’17 –July’18

Software Engineer / ETL Informatica Developer

Tapestry business units creates offers to help sell products, reduce inventory of goods, or to help promote a new product line. These offers allow for discounts and special deals when the product is purchased via various selling channels that include the Point of Sale in stores and e-commerce platforms. In order to support the different offers that the Tapestry business units create, the SAP Promotion module has been configured. One type of offer includes the creation of discounts at the subclass, class, or department level which is handle via the Promotion Discount interface. Also, The interface requirement is to provide Micros POS systems Article Assortment Data and Pricing from SAP S4HANA system, Article Assortment data and Pricing is required at the Micros POS system to sell the product.

Worked in Agile methodology

Designed mappings as per the company standards, Unit testing, debug and provide the test cases for the code developed.

Analyzed the data before starting the development and in monitoring the quality of the data, Worked on the Data Analytics.

Designed and developed mapping using various complex transformations like XMLParser, XML generator transformations

Worked on complex XML structures with at least minimum of 25 nodes.

Used Idocs for the implementation of SAP interfaces for transferring the data through informatica .

Experience designing and developing dashboard applications with full Reporting and Analytics functionality

Used complex XML sources and targets for extracting and loading data into flat files.

Worked with the functional team modifying the XML’s, XSD structures.

Used Netezza Database for loading and extracting data.

Using transaction WESOUT in SAP Retail, An application log indicates the status of the outbound message processing

Automated workflows through Unix shell scripts using Tidal

Unix Shell scripts for triggering the jobs.

Performance tuning has been done to improve the runtime of the jobs for a better performance.

Environment: Informatica Power Center 10.2.0, power exchange, Netezza,SAP, Agility Workbench, PL/SQL,, MS SQL Server 2010, Flat Files, Windows XP, UNIX, PL/SQL, Teradata,XML,XSD .

Verizon Wireless July’15- Dec’17

Software Engineer / ETL Informatica Developer

In this project we have developed various applications for Verizon Point of Sales where the Verizon store employees could use them for different criteria like the traffic, sales and many more metrics. Used Kronos application for scheduling. Was involved in various projects and used power center to transform the data from various data sources. As part of this project we also migrated the data from SQL server to Oracle. The data would be fed into oracle from Teradata via Informatica ETL. Various applications were used by representatives and customer to view the overages, average calls, promotions, plans information pertain to an account.

The Migration project we accomplished was to move the legacy application from SQL Server stack to Verizon Standard Oracle stack.

Worked in both Waterfall and Agile methodology

Involved in writing PL/SQL code in stored procedures, functions and packages to support applications back end.

Optimized and Tuned SQL queries

Designs and codes from specifications, analyzes, evaluates, tests, debugs, documents, and implements moderately complex software applications

Designed solutions after being involved with the analytics team as a Data Analytic developer by providing constructive solutions to the business users.

Designed and developed mapping using various transformations like XML, XML generator, Expression, Lookup, Aggregator, Router transformations

Designed and developed mapping using various transformations like Web Service, Http transformations for calling web services through Informatica .

Experience designing and developing dashboard applications with full Reporting and Analytics functionality

Automated workflows through Unix shell scripts using autosys.

Design reference data and data quality rules using IDQ and involved in cleaning the data using IDQ in informatica Data Quality 9.6 environment

Used IDQ’s standardized plans for addresses and names clean ups

Worked on IDQ file configuration at user's machines and resolved the issues

Used IDQ to complete initial data profiling and removing duplicate data

Shell scripts extensively developed for all the jobs

Performance tuning achieved for better performance and improved customer experience from desktop and tablets

Worked with Verizon non-technical employees in gathering the feedback of the application and improvising the app.

Environment: Informatica Power Center 9.6.1, power exchange, Oracle 11g, SQL, PL/SQL, DB2 8.0, MS SQL Server 2010, Flat Files, Windows XP, UNIX, PL/SQL, Teradata .

Stryker Corporation Inc Aug’14-June’15

Data warehouse Engineer/ETL Informatica Developer

In this project we mainly used Informatica to loop through the folders which searches for a file in a specific format picks it up and it process it. We generate the flat files into Informatica, It will be scheduled to retrieve a file in this expected format which would apply logic as defined in the MD-50 Functional requirements document, processing each file by uploading on the Power center. Informatica will generate output in the target ODS.

After each file is processed successfully, it is archived into the /archives folder. If file is not processed successfully (because of any compatibility errors, missing columns, data types conversion errors, etc) the file will be moved to the /errors folder.

Responsibilities:

Actively involved in gathering requirements and acquiring application knowledge from Business Managers.

Involved in analyzing the requirements for designing the process flow for extracting the data across various source systems.

Designed efficient and optimized ETL Strategy by identifying, analyzing sources and destinations.

Designed Informatica Mappings to transfer data from various sources like Text Files, XML Files, Excel, Flat files to SQL Server

Optimized the performance of various SQL scripts by using Stored Procedures, Indexes, Views, User defined functions, Indexed Views for querying data faster.

Designed and developed mapping using various transformations like Source Qualifier, Expression, Lookup, Aggregator, Router, Rank, Filter and Sequence Generator transformations.

Used Mapping Variables, Mapping Parameters in the Informatica Mappings to filter the daily data from the source systems.

Proficient in implementing complex business rules through Informatica transformations, Workflows/Worklets and Mappings/Mapplet.

Made use of Post-Session success and Post-Session failure commands in the Session task to execute scripts needed for cleanup and update purposes.

Tuned performance on Sources, targets and mappings and SQL queries in the mappings.

Implemented Informatica push down optimization for utilizing the data base resources for better performance.

Automated Unix shell scripts to verify the count of records added everyday due to incremental data load for few of the base tables in order to check for the data consistency.

Involved in writing PL/SQL code in Oracle stored procedures, functions and packages to support applications front end and back end.

Developed PL/SQL procedures, functions to facilitate specific requirement.

Optimized and Tuned SQL queries and PL/SQL blocks to eliminate Full Table scans to reduce Disk I/O and Sorts.

Automated the batch jobs using Unix shell scripts.

Scheduled jobs using Informatica and monitored automated weekly jobs.

Prepared Detail design documentation thoroughly for production support department to use as hand guide for future production runs before code gets migrated.

Also involved in conducting and leading the team meetings and providing status report to project manager

Resolved and closed the Production tickets generated due to failure of daily incremental production jobs.

Member of core ETL team involved in gathering requirements, performing source system analysis and

Development of ETL jobs to migrate the data from the source to the target DW

Used data quality transformations like address validator, Parser, Match,Exception,Association,Standardizer and other significant transformations.

Extensively worked on Infa Data Quality and Infa powercenter throughout complete data quality and mdm projects.

Worked in Water Fall Methodology.

Used Erwin to reverse-engineer and refine business data models.

Prepared Data Architect document, Mapping Specification and Unit testing documents for ease of future maintenance and support.

Developed forms based on views, tables and procedures in tabular and form layouts.

Experience in creating master detail forms and adding alerts.

Extensive experience in UNIX Shell Scripting.

Experience in Informatica Power Center, Repository Manager, Designer, Workflow Manager and Workflow monitor.

Provided technical support in designing the workflows and processes across various systems.

Provide technical support for production issues by identifying the issue and provide root cause analysis.

Experience in loading data, troubleshooting, Debugging mappings, performance tuning of Informatica (Sources, Targets, Mappings and Sessions) and fine-tuned transformations to make them more efficient in terms of session performance.

Environment: Informatica Power Center 9.1, power exchange, Oracle 11g, SQL, PL/SQL, DB2 8.0, MS SQL Server 2008, Flat Files, Windows XP, UNIX, PL/SQL

Kelly Blue Book Irvine CA May’14-Aug’14

ETL Informatica Developer/Data Warehouse Engineer

The project mainly involves in designing, Developing and maintaining the data in the data warehouse and delivering the data to the various departments in the Web Team according the client requirements. The ETL process mainly involves extracting the data from Netezza, Flat files, SQL server and mainframes with the target data warehouse as Netezza and SQL Server according to the client requirements.

Responsibilities:

Member of core ETL team involved in gathering requirements, performing source system analysis and development of ETL jobs to migrate the data from the source to the target DW

Analyzed the business requirement document and created functional requirement document mapping all the business requirements.

Worked in Agile Methodology.

Created logical and physical data models for the star schema using ERWIN r7

Used Erwin to reverse-engineer and refine business data models.

Involved in designing the process flow for extracting the data across various source systems.

Prepared Data Architect document, Mapping Specification and Unit testing documents for ease of future maintenance and support

Extracted data from various Relational Databases like SQL Server, Netezza, Flat Files using Informatica mappings.

Designed, Built and deployed effective SSIS packages using various control flow data flow items to transform and load the data from various databases using SSIS .

Experience in maintaining Batch Logging, Error logging with event handlers and configuring connection managers using SSIS.

Subject Matter Expert for conversion and data mapping to Netezza

Used HR Data knowledge, heavily participated in data conversion to Oracle Data Warehouse

Installed and configured Informatica Power Exchange CDC and for Oracle on UNIX platform.

Designed and developed mapping using various transformations like Source Qualifier, Expression, Lookup, Aggregator, Router, Rank, Filter and Sequence Generator transformations.

Created Workflows and used various tasks like Email, Event-wait and Event-raise, Timer, Scheduler, Control, Decision, Session in the workflow manager.

Used Mapping Variables, Mapping Parameters in the Informatica Mappings to filter the daily data from the source systems.

Made use of Post-Session success and Post-Session failure commands in the Session task to execute scripts needed for cleanup and update purposes.

Worked with Netezza database Developed stored procedures on Netezza and SQL server for data manipulation and data warehouse population

Involved in writing PL/SQL code in Oracle stored procedures, functions and packages to support applications front end and back end.

Developed PL/SQL procedures, functions to facilitate specific requirement.

Worked on SQL Loader for bulk load of data and used SQL tuner for tuning SQL.

Optimized and Tuned SQL queries and PL/SQL blocks to eliminate Full Table scans to reduce Disk I/O and Sorts.

Automated the batch jobs using UNIX shell scripts.

Tuned performance on Sources, targets and mappings and SQL queries in the mappings.

Implemented Informatica push down optimization for utilizing the data base resources for better performance.

Automated UNIX shell scripts to verify the count of records added everyday due to incremental data load for few of the base tables in order to check for the data consistency.

Involved in writing windows batch scripting.

Scheduled jobs using Autosys and monitored automated weekly jobs.

Prepared Detail design documentation thoroughly for production support and release management department to use as hand guide for future production runs before code gets migrated.

Also involved in conducting and leading the team meetings and providing status report to project manager.

Environment: Informatica Power Center 9.1/8.6,Power exchange, Netezza 7.0.2, Erwin r7, Oracle 11g, SQL, PL/SQL, DB2 8.0, MS SQL Server 2008, Flat Files, Autosys, Windows XP, UNIX, PL/SQL, SQL*Loader, TOAD, ANSI SQL, IBM Data Stage 9.1, SSIS

Western Michigan University

Independent Research (Mar’13 –August ’13)

This was an independent research under my Prof.Lillien, This is a consequence for network service providers (NSPs) striving to offer improved security features to their customers as a value-adding feature is to devise a security framework in which detection devices are placed within the network. Before doing so, the NSP must take into account that it is not desirable to make frequent changes or experiment

With various security feature deployments in the network infrastructure of a production

System. For this reason, network operators can greatly profit from a network simulation tool in which various features of the security architectures can be tested in order to ensure maximum attack detection efficiency before the actual physical deployment.The advantage over conventional testbeds is the low cost and ease at which tests can be carried out. The Network Security Simulator NeSSi2 allows

NSPs to experiment with different network-centric security framework setups and algorithms in order to evaluate and compare intrusion detection efficiency and operational costs. In this fashion, a devised security framework can be tested in a simulation environment before the actual detection units are physically deployed in the network.



Contact this candidate