Post Job Free

Resume

Sign in

AWS,Snowflake,Hadoop,Teradata,ETL Informatica,Datastage,Talend,Java

Location:
Peoria, IL
Salary:
120000
Posted:
April 30, 2020

Contact this candidate

Resume:

Narayana Kasireddy

704-***-****(Cell) / adc1df@r.postjobfree.com

SUMMARY

Result-driven, self-motivated IT professional with 16+ years of total IT experience in Analysis, Design, Development, Testing, Administration, Implementation and Support for Data Warehousing projects.

Experience in Caterpillar working with AWS(S3,Lambda,Fargate,DynamoDB,SQS,SNS etc..),Microsoft Azure, Snowflake associated technologies to build Telemetry BI Store to have all telemetry data made available in one common place to support end user needs.

Experience in developing and designing products/solutions using a wide array of tools and technologies that help organizations use their data strategically to innovate their business models as well as reduce cost, improve efficiency, and comply with regulations. Highly energetic with relentless approach to solving problems and a very strong sense of accountability and ownership.

Worked as Technical Lead, Support Lead to work onsite and offshore complex technical issues and resolutions during to off-shift and weekend support meetings to meet the deliverables without any slippages.

Infrastructure as Code (YAML and JSON Templates) using AWS Cloud Formation, Azure Resource Manager.

Amazon Web Services and Microsoft Azure Cloud Services, Azure DevOps / Visual Studio Team Services (VSTS)

Automated Deployments and Release Management. Continuous Integration and Continuous Delivery

Involved in:

Project timeline estimates and leading team to follow SDLC best practices such continuous integration, automated unit test and regression testing, etc. and focus on end to end quality of the delivery.

Work with Business Analyst Team to understand functional and non-functional requirements and translate into technical designs to build conceptual, logical and physical data models for data warehouse.

Lead Creating data flow diagrams, and mapping documents, technical designs, code reviews, test strategies and implementation plans.

Mentoring technical development team on optimal utilization of Emerging ETL and Big Data solutions.

Leveraging DevOps techniques and practices like Continuous Integration, Continuous Deployment, Test Automation, Build Automation and Test Driven Development to enable the rapid delivery of end user capabilities.

Works independently on complex process modules and customized solutions to address business problems.

Created an ETL Job/Custom Data pipeline to migrate bulk data from on-premise legacy systems to cloud to suite end user need.

Created and End to End flow to process the TSF data by using stream analytics/event hubs/topics to load SQL DB.

CAB meetings, Migration Process followed defined change management process as per company standards prior production deployments.

Good Experience in leading a team of 5-10 developers in all phases of SDLC from Requirements, Analysis and Design, development, testing and deployment.

Strong Analytical and Problem Solving Skills with ability to work independently and in team environments, simultaneously on multiple projects with competing priorities.

TECHNICAL SKILLS

ETL Tools : DataStage 8.1, 8.7,11.5, Informatica 7.1, Data integration, Data ingestion

Data Modeling Tools : ER Win 7.1,9.7

Databases : Oracle 9i, 10g, 11g, DB2 UDB 8.1, Teradata V2R15, Hadoop and Impala

Cloud Technologies : Microsoft Azure Data Lake/Data Factory,AWS and Snowflake,SnapLogic

NoSQL : DynamoDB,HBase

Programming Language : SQL, Java 8.0,Python,Scala, Hive, Spark, Sqoop, XML, Json

Operating Systems : Unix, Linux, AIX, Sun Solaris, Windows NT, Windows Server 2008 R2

Scheduling Tools : Tidal, UC4,Autosys

EDUCATION

Master in Computer Applications(MCA) from Periyar University, Tamilnadu, India, 2002

Sr AWS Data Engineer/Sr ETL Developer May 2012 to Till Date

Caterpillar Inc., Peoria, IL

At Caterpillar, I got exposure to multiple projects with different technologies and performed diverse roles starting from developer to Designer,Tech Lead and Support Lead . Also, involved in Cloud Technologies (Microsoft Azure,AWS, Snowflake). Below are some of the projects and brief engagements that I took out during my tenure.

Cat Digital Data Warehouse & Telematics datahub:

Developed an ETL process to pull dealer data from snowflake to Oracle for Drive Train Consumer needs.

The data flow for our TSF pipeline is as follows. Unenriched TSF messages are placed on a Kinesis stream from the IoT Gateway. Messages are then processed by the Message Steward service, where they are validated and enriched by multiple APIs and caches. Enriched messages (those that successfully exit Message Steward) are ready to be persisted.

In addition to the main TSF pipeline, the Telematics Data Hub has two data pipelines capable of funneling TSF V0 and VIMS Productivity data into Snowflake data tables. These files are data formats used in the legacy CCDS system built in Azure. Because many assets still send the Data Hub this data, TDH processes and stores these messages as well.

Created Data Factory job to pull JSON messages from TSF/Event Hub to SQL Data warehouse.

Primarily involved in Data Migration using SQL, SQL Azure, Azure storage, and Azure Data Factory.

Strong development skills with Azure Data Lake, Azure Data Factory, SQL Data Warehouse Azure Blob, Azure Storage Explorer.

PostgreSQL widely using for open source RDBMS while snowflake is multi-structured data handler.

Two stream extractor Fargate services carry the data away from the respective Azure Event Hubs and onto Kinesis streams. Firehoses batch save the files to separate folders (tsf-v0 and vims-productivity-v0) in the same S3 bucket as TSF, where the data is then stored in Snowflake by means of SQS queue triggers. This Advanced Productivity VIMS data(Location & segments) & Telematics Data(Location) Availability in Snowflake to enable the algorithm, Business Intelligence and Advanced Productivity report for CD&T Team.

Created ETL mapping document and ETL design templates for the development team.

Worked on Hive optimization techniques to improve the performance of long running jobs.

Migrated Hive scripts and workflows to Spark data frames, datasets as part of performance tuning.

Converted Map reduce XML PARSER programs into Scala API to process XML files using XSD’s and XSLT’s as per the client requirement and used to process the data into Hive tables

Worked on different file formats like Avro, Parquet and Json formats.

Involved in migration from On prem to Cloud AWS migration.

Led a migration project from Oracle to Snowflake warehouse to meet the SLA of customer needs

Used Analytical function in hive for extracting the required data from complex datasets.

Prepare the Oozie workflows and schedule the Workflows using Coordinators.

Created ETL pipelines using Stream Analytics and Data Factory to ingest data from Event Hubs and Topics into SQL Data Warehouse.

Responsible for Migration of key systems from on-premises hosting to Azure Cloud Services.

TIDAL Scheduling tool is used to Schedule all the Data Stage jobs, Unix Scripts to execute in production environment.

Created the Error handling, and audit process common modules to use across the project.

Resolving the Business critical Issues in Production environment and helping the production team.

Environment: IBM Data Stage 8.5/11.3, Teradata V2R14, Oracle, PL/SQL, AWS EMR,EC2,S3,Cloudwatch,Lambda functions,Stepfunctions,AWS CLI, CDH 5.8.2, Hadoop 2.5.0Microsoft Azure, Tidal and Windows XP

Sr ETL Developer January 2012 to April 2012

TJX Enterprise Data Warehouse (EDW),Framingham, MA

Incorporated data from systems all over the enterprise, including point-of-sales, human resources, merchandise planning, distribution and PO management. Data ranged from flat file extracts to direct querying of databases.

Ensured that this disparate data was imported quickly and accurately, while establishing relationships between the different types of data that loaded the groundwork for new kinds of business analysis.

Used reliable hardware infrastructure that was scalable and powerful enough to accommodate the information needs of rapidly growing business.

Standardized process to minimize development and testing costs.

Outlined ETL strategy in document to address the design in extracting, transforming and loading process to meet business requirements.

Environment: IBM Information Server 8.7 (DataStage and QualityStage, FastTrack, Business Glossary, Information Analyzer), Netezza 4.x, Cognos Query Studio v10, Windows XP

ETL Lead Data Warehouse Developer September 2011 to December 2011

Anheuser Busch InBev (ABI), St. Louis, MO

Managed performance of packaging lines for the different breweries.

Provided KPI reports that were used for allocation of resources and measuring of targets.

Supported number of change requests to avoid manual intervention and implemented the automation process without scope or schedule changes.

Extracted data from variable format sequential files, SAP and Oracle using various stages in DataStage designer to build jobs and load data into Teradata area.

Environment: IBM Information Server 8.0.1/7.5 (DataStage and QualityStage, FastTrack, Business Glossary, Information Analyzer), Teradata V2R9, Oracle 10g, SAP Business Objects XI R3, ERWin 4.1.4, Tivoli Scheduling, Windows XP.

ETL Lead Data Warehouse Developer February 2010 to August 2011

Advance Auto Parts, Roanoke, VA

Provided analysis, design, development, testing, UAT, implementation and post-implementation support activities in full SDLC life cycle.

Used operational and production fixes to deliver as part of the EDW Nightly Batch Cycle with high productivity.

Maintained change requests to avoid manual intervention and implemented automation process without scope or schedule changes.

Assisted new developers to build skills in DS and DB2 and bring them up to speed.

Performed coding, testing and code review of ETL changes for the enhancements and defects and ensured on-time delivery.

Developed parallel jobs using stages which included join, transformer, sort, merge, filter, lookup and copy.

Supported unit, system and integration testing. Developed test scripts, test plan and test data.

Prepared the job document and collaborated with scheduling team in scheduling the jobs in QA and production using UC4 scheduling tool.

Used the Remedy Tool to track the tickets and project based on priority given by the client team.

Environment: IBM Information Server 8.5 / 8.0.1 (DataStage and QualityStage, FastTrack, Business Glossary, Information Analyzer), IBM DB2 9.1/9.7, Oracle 10g.11g, OBIEE 11g, SAP Business Object XI R3, ERWin 4.1.4, AIX 5.3, UC4 Scheduling, Windows XP

ETL Lead Data Warehouse Developer January 2007 to January 2010

St. Paul Travelers, Hartford, CT

Maintained change requests to avoid manual intervention and implemented automation process without scope or schedule changes.

Assisted new developers to build skills in DS and Teradata and bring them up to speed.

Performed coding, testing and code review of ETL changes for the enhancements and defects and ensured on-time delivery.

Used DataStage Designer to develop jobs for extracting, cleansing, transforming, integrating and loading of data into warehouse.

Extracted data from variable format sequential files, mainframes and teradata using various stages in DataStage designer.

Environment: Ascential DataStage 7.5, Teradata V2R5/V2R7, ERWin 4.1.4, Oracle 10g, OBIEE 9i,Cognos 8, ERWin 4.1.4, AIX 5.3, Maestro/Autosys Scheduling, Windows XP

ETL Data Warehouse Developer November 2003 to December 2006

BellSouth, Atlanta, GA

Performed full SDLC life cycle in analysis, design, development, testing, UAT, implementation and post implementation support activities.

Documented the Component Test and Assembly Test results in common Share Point.

Used Informatica PowerCenter 7.1/6.0/5.1 to load extract data from Flat Files, Oracle, and Sybase databases and load to Sybase, Oracle, TeraData database and Flat Files.

Performance tuned mappings and sessions to achieve best possible performance.

Created tasks, worklets and workflows and scheduled workflows to run the jobs at required frequency using Workflow Manager.

Involved in Deployment Activities and Hypercare activities.

Extracted data from multiple data sources, performed multiple complex transformations and loaded data in to SQL Server Tables.

Environment: Informatica Power Center 7.1, Oracle 8.0/7.x,SQL*Plus, SecureCRT 4.1, WinSCP, Rapid SQL 7.1.0, PL/SQL, Solaris 8.0, Windows NT 4.0



Contact this candidate