Post Job Free

Resume

Sign in

ETL&Terradata Developer

Location:
Hillsboro, OR
Posted:
August 15, 2019

Contact this candidate

Resume:

Uday Kumar

ETL Developer

Employer Details

ac93c4@r.postjobfree.com

503-***-**** X 416

SUMMARY

Having 6 years of IT experience in Data warehousing with emphasis on Business Requirements Analysis, Application Design, Development, coding, testing, implementation and maintenance of client/server Data Warehouse and Data Mart systems.

Worked with Informatica Power Center 10.1/9.5/9.1/8.6, Informatica Data Quality (IDQ) 9.5/9.1 as ETL tool for extracting, transforming and loading data from various source data inputs to various targets.

Strong Data Warehousing ETL experience of using Informatica Power Center Client tools - Mapping Designer, Repository manager, Workflow Manager/Monitor and Server tools - Informatica Server, Repository Server manager.

Worked extensively with Dimensional modeling, Data migration, Data cleansing, ETL Processes for data warehouses.

Developed the DataStage jobs Deals with binary data as well as fixed length data from mainframe systems.

Worked extensively with Teradata versions 15/14/13, strong knowledge on teradata utilities - Fast load, Multi load, Tpump and Teradata Parallel Transporter (TPT) to load huge amounts of data from flat files into Teradata database and PDCR utility.

Experience in integration of various data sources definitions like SQL Server, Oracle, Teradata SQL Assistant, MYSQL, Flat Files, XML and XSDs.

Generated BTEQ scripts to invoke various load utilities transform the data and query against Teradata database.

Build a Python program to export the data from Teradata to AWS-S3 bucket. Created SNS topics, SQS queues, Lambda triggers while Exporting from Teradata to S3.

Designed and deployed reports with Drill Down, Drill Through and Drop-down menu option and Parameterized and Linked reports using Tableau Server and desktop.

Extensive experience in integrating data from flat files - fixed width, delimited, XML, Web Services by using various transformations available in Informatica such as - Source qualifier, XML parser, and Web services consumer transformation.

Strong expertise in designing and developing Business Intelligence solutions in staging, populating Operational Data Store (ODS), Enterprise Data Warehouse (EDW), Data Marts / Decision Support Systems using Informatica Power Center 9.x/8.x/7.x/6.x ETL tool.

Good Understanding of BigData/Hadoop environment using CDH with Spark.

Used Spark-SQL to process the data and to run on Spark engine.

Worked on Spark for improving performance and optimization of existing algorithms in Hadoop using Spark-SQL and Scala.

Used Different Spark Modules like Spark Dataframe and Spark SQL.

Expertise in creating databases, users, tables, triggers, macros, views, stored procedures, functions, Packages, join and hash indexes in Teradata database.

Expertise in Master Data Management concepts, Methodologies and ability to apply this knowledge in building MDM solutions.

Good knowledge on Alteryx workflows.

Proficient in performance analysis, monitoring and SQL query tuning using EXPLAIN PLAN, Collect Statistics, Hints and PL/SQL Trace both in Teradata as well as Oracle.

Extensive knowledge on scheduling tools - Control-M, Autosys, Tivoli (TWS), ESP and CRON.

Worked in Agile and Waterfall Methodology.

Nike (Sr. Tech Engineer - Informatica &Teradata Developer)

Hillsboro, OR Oct 2018 –Till Today

Responsibilities:

Develop and communicate a deep understanding of NIKE product metrics.

Able to identify any data integrity issues in generating any of metrics.

Work with product stakeholders (product analytics, PMs, engineers) for identifying the root cause of any data integrity issues and suggested changes to generate product insights.

Perform analytical deep-dives to identify problems, opportunities and specific actions required.

Leverage data driven insights in the product brainstorming, road-mapping, and trade-off discussions.

Work together with product management, engineering, design, policy, and senior executives to rapidly execute, learn and iterate.

Worked with Informatica power center Designer, Workflow Manager, Workflow Monitor and Repository Manager.

Used Informatica as ETL tool to pull data from source systems/ files, cleanse, transform and load data into the Teradata.

Updated and maintained ETL (Extract, Transformation and Loading) mappings to extract the data from multiple source systems like Oracle, SQL server and Flat files and loaded into Oracle.

Excellent Knowledge on Slowly Changing Dimensions (SCD Type1, SCD Type 2, SCD Type 3), Change Data Capture, Dimensional Data Modeling, Ralph Kimball Approach, Star/Snowflake Modeling, Data Marts, OLAP and FACT and Dimensions tables, Physical and Logical data modeling.

Experience in Dimensional Data Modeling, Star/Snowflake schema, FACT & Dimension tables.

Experience in developing solutions in Snowflake Cloud Computing.

Used Sqoop to extract data from Oracle SQL server and MySQL databases to HDFS.

Replaced existing MR jobs and Hive scripts with Spark SQL, Spark data transformations for efficient data processing.

Involved in creating new table structures and modifying existing tables and fit into the existing Data Model.

Extracted data from different databases like Oracle and external source systems like flat files using ETL tool.

Used SQL to query the databases and do as much crunching as possible in Teradata, using very complicated SQL Query optimization (explains plans, collect statistics, data distribution across AMPS, primary and secondary indexes, locking, etc.) to achieve better performance.

Monitoring database space, identifying tables with high skew, working with data modeling team to change the Primary Index on tables with High skew.

Flat files are loaded into databases using Fast Load and then used in the queries to do joins.

Created proper PI taking into consideration both planned access and even distribution of data across all the available AMPS.

Used Teradata utilities (Fast Load, Multiload) to load data into Target Data Warehouse and used Teradata Sql Workbench to Query data in the target Teradata data warehouse.

Involved in migration of ETL processes from Oracle to Teradata to test the easy data manipulation.

Handled data transformation and ingestion from Amazon S3 into Snowflake and performed data validation.

Involved in OLAP model based on Dimension and FACTS for efficient loads of data based on Star Schema structure on levels of reports using multi-dimensional models such as Star Schemas and Snowflake Schema.

Worked on GitHub to manage the Parameter files, SQL code (Structure of Tables, Views, Stored Procedure, and PLSQL Packages) and Python code in respective repositories.

Migrate the code from Data Base to DEV/QA/Master Repo’s using Intellij as IDE tool and also used to GIT BASH to push the code.

Worked on AWS services such as SNS, SQS, Lambda, ECS, ECR, S3 and Athena.

Created a Docker image that to run ECS task through lambda.

Developed a python script to extract data from Teradata and exported to S3 AWS

Terraform is used to export the data from Teradata to AWS using a Jenkins file.

Created few SNS topics to get the notifications in case of any failures while exporting the data to S3 buckets to flat files.

Used Athena to validate the huge data in easier way that present in S3. Developed SQL queries in Athena that compares quantities, count, sum and any changes of that data.

Managed log files, backups and capacity.

ENVIRONMENT: Teradata, Informatica, SQL Developer, AWS, Teradata SQL Assistant, Python, SQL, PL/SQL, Spark SQL, Snowflake, GitHub, MS Project, MS Visio, Unix.

Thermofisher Scientific (Sr. Informatica / Teradata Developer)

Carlsbad, CA Apr 2017 – Sep 2018

Responsibilities:

Co-ordination from various business users' stakeholders and SME to get Functional expertise, design and business test scenarios review, UAT participation and validation of data from multiple source.

Defined and developed brand new standard design patterned, ETL frameworks, Data Model standards guidelines and ETL best practices.

Worked with Informatica power center Designer, Workflow Manager, Workflow Monitor and Repository Manager.

Extensively worked on Informatica Power center transformations as well like Expression, Joiner, Sorter, Filter, Router and other transformations as required.

Performed detailed data investigation and analysis of known data quality issues in related databases through SQL.

Actively involved in Analysis phase of the business requirement and design of the Informatica mappings.

Performed data validation, data profiling, data auditing and data cleansing activities to ensure high quality Cognos report deliveries.

Developed Informatica mappings for TYPE 2 Slowly Changing Dimensions.

Teradata utilizes like BTEQ, FASTLOAD, MLOAD, and FASTEXPORT.

Created sessions and work-flows for the Informatica mappings.

Heavily used Informatica Cloud integration using Amazon Redshift connector and integrated data form various sources.

Configured sessions for different situations including incremental aggregation, pipe-line partitioning etc.

Created mappings with different look-ups like connected look-up, unconnected look-up, Dynamic look-up with different caches such as persistent cache etc.

Created various Mapplets as part of mapping design.

Involved in writing Oracle stored procedures and functions for calling during the execution of Informatica mapping or as Pre-or Post-session execution.

Created effective Test Cases and performed Unit and Integration Testing to ensure the successful execution of data loading process.

Documented Mappings, Transformations and Informatica sessions.

Analyzed Session Log files in case the session failed to resolve errors in mapping or session configurations.

Involved in designing the ETL testing strategies for functional, integration and system testing for Data warehouse implementation.

Extensively involved in testing by writing some QA procedures, for testing the target data against source data.

Used Unix Commands and Unix Shell Scripting to interact with the server to move flat files and load the files in server.

Worked on Data Profiling, Data standardization, error handling and exception handling.

Worked on Merge, match and reference table to find duplicate and generate report.

Written UNIX shell scripts for file manipulation, ftp and to schedule workflows.

Co-ordinated offshore team on daily basis to leverage faster development.

ENVIRONMENT: Informatica Power Center 9.6.1, Informatica Power Exchange 9.6.1, Amazon Redshift, Cognos 9.0, Sun Solaris, SQL, PL/SQL, Oracle 11g, TOAD, SQL Server 2012, Shell Scripting, Teradata 14, Autosys, GitHub, Cognos 9

Real Power Vision (Informatica / SQL Developer)

BANGALORE, India. Sep 2013– June 2016

Responsibilities:

PRPD (Partial Redistribution Partial Duplication) by developing test cases for functional & performance testing and coding modules across different levels of the Teradata Database in 2013.

Tuning of Teradata SQL statements using Explain analyzing the data distribution among AMPs and index usage, collect statistics, definition of indexes, revision of correlated sub queries, usage of Hash functions, etc.

Teradata Columnar (Column Partitioning) project by developing test cases for functional & performance testing and coding of modules across different levels of the Teradata Database.

Designed and implemented appropriate ETL mappings to extract and transform data from various sources to meet requirements with Informatica.

Designed and developed Informatica ETL mappings to extract master and transactional data from heterogeneous data feeds and load.

Experienced in Repository Configuration/using Transformations, creating Informatica Mappings, Mapplets, Sessions, Worklets, Workflows, Processing tasks using Informatica Designer / Workflow Manager to move data from multiple source systems into targets.

Analyzed business process workflows and assisted in the development of ETL procedures for moving data from source to target systems.

Used workflow manager for session management, database connection management and scheduling of jobs.

As a member of dbs frontline, provided support to the various teams in identifying the database issues and solving the issues.

Provided the support for Customer Support Team and handled the escalated issues.

Enhanced the testing validation procedure by developing the Stored Procedure's using XML Plan Developed Perl scripts to automate the quantum plan comparison testing procedure

Developed plan comparison tool using C language to compare the query plans and print the plan in tree format

I Suite Solutions (SQL Developer)

Hyderabad, India. July 2012 –May 2013

Responsibilities:

Expertise in SQL, PL/SQL (Triggers, Records, Function, Procedure, Packages, Collections, Analytical Functions, Partitioning, Materialized Views, Oracle DBMS_JOB, DBMS_SCHEDULER)

Create new users, set up roles, user access and permissions on Oracle DB. Monitor table space.

Proactively worked on daily system checks to monitor application activities and server’s status

Write complex SQL queries.

Managing production and development MS SQL Servers.

Integrate MS SQL with Oracle. Apply DDM (data masking) to sensitive data.

Created packages, procedures, custom functions, triggers, tables, table/index partitions, collections and other schema objects to support interactive applications and reporting needs.

Troubleshoot slow running reports and code units to find the bottleneck and apply my knowledge in performance tuning to normalize, optimize, partition (tables, mv, indexes) and improve the overall performance.

Identified application bugs and working with vendors for further enhancements to fix those issues.

Develop and manage reporting infrastructure that allows for the real-time generation of reports as well as viewing dashboards of key metrics.

Designed and developed various reports on daily outages of VoIP, Cable and other Network devices for various divisions by using Oracle PL/SQL with BULK COLLECT.

Automate data collection from the data warehouse by writing shell scripts and load data into Oracle DataMart via SQL Loader.

Test ETL process with new IPs. Document the changes through multiple environments (development, test, production).

Leading a team of developers to streamline and redesign the reporting process to be fully automated.

For the sales cycle, created standard API's which would extract all the Residential and Commercial Sales Orders, and Bill of Materials from the Legacy System to the current Oracle's Order Fulfillment Module.

Based on the Purchase Order and the associated acquisition costs, built logic which dynamically picks all the associated accounting units and generates PO Distributions.

Managing WebLogic application and Oracle BI reporting tool and serve as the first point of contact for advanced or high complexity report requests.

Produced numerous network performance reports using OBI.

Maintain and open new firewall request as needed to access an external data source.

Managing SFTP server users and directory structure.



Contact this candidate