Big Data Engineer

Ashok Veludandi

Status: US Citizen

248-***-****; *********@*****.***

Big Data Engineer

Designer, Development, Test and Production Support of Big Data Applications

A collaborative engineering professional with 17 years of substantial experience in designing, developing, executing and supporting applications for complex business problems involving big data solutions, data warehousing, near to real-time Analytics and BI solutions. Known for using the right tools when and where they make sense and created an intuitive architecture that helped organizations effectively analyze data and create Machine Learning (ML)/Artificial Intelligence (AI) solutions to enhance business opportunities.

Core Competency

Data Lake (CDP):

Proven history of building data processing systems and serving as an expert in data warehousing solutions while working with a variety of database technologies, structure and unstructured files (social media, campaigns & inventory), vsam files, flat files designing and optimizing while loading into data lake/data warehouses/data marts and HIPAA X12 EDI transaction sets

Experience architecting data models of highly scalable, distributed systems using different tools as well as designing and optimizing down streams. Able to integrate and convert to Big Data technologies as team-member and also lead Team.

Data warehousing:

BI/Analytics:

Tools and Technologies:

Database and Tools: HDFS, NoSQL, Hue, Jupyter Notebook, Python, Pypark, Teradata Studio, HDFS, Apache Hive, Kudu, Impala, Informatica (Power Center, BDM Developer, Power Exchange, IDQ), DataStage, Alteryx, Talend, HQL, Sqoop, Cloudera, Collibra, Data Mapping tool, Erwin & Power Designer, Power BI, Cognos (certified), Alteryx, Tableau, Oracle, MS SQL Server, Teradata, DB2, Netezza,AWS Cloud,Redshift,EMR,EC2.Jupyter Notebook.

Data Analysis: Consulted with business partners and data architects and made recommendations to improve the effectiveness of Big Data systems for descriptive analytics systems. Integrated new tools and developed technology frameworks/prototypes to accelerate the data integration process and empower the deployment into Hadoop data lake. Identify gap analysis in the existing data models in different environments in order to integrate into data lake. Data visualizing of the data with goal of discovering useful information, informing and supporting decision making.

Tools:

Data Transformation: Experience hands-on designing high level & low design documents Mapping documents, design data flow diagram and process improvements for the applications by implementing and optimizing data transformation processes in the Hadoop and Informatica ecosystems. Able to consolidate, validate and cleanse data from a vast range of sources- from applications and databases to files and validate the target data once loaded applying Audit Base Control (Reconciliation) and process restart ability.

Data Collection/Catalog/Mapping: Capable of extracting the data from existing different databases/files. Experience in designing and implementing fast and efficient data acquisition using Big Data processing techniques and tools and quickly discover data that matters so that can generate impactful insights that drive business value. Using Mapping Tools to help business and technology data mappers to identify the upstream and downstream attributes in order to identify if any transformation logic applied. Using the catalog tool identifying the fully managed and highly scalable data discovery and metadata management service.

Data Model: Designed the Logical and physical data mart models to organize the elements of data and standardize how they relate to other entities.

Business Intelligence: Designed and Developed reports for data analysis and

Data Visualization for different strategies, data analysis, reports and dashboard

development to help organizations to make more data-driven decisions.

Scrum: Played as team member focus on creating stories that meet the Definition of Done, causing the removal of impediments to the team process ensuring all the

events take place and are positive, productive and kept within the time box sprint

completion. Backlog grooming, Retrospective and Informing the stakeholders for

upcoming production deployment and feedback after deployment.

Tools: Jira, Microsoft Team.

●Strong communication skills, including the ability to convey analytic insights effectively to both IT and business team.

●Past experience leading the team onsite and offshore to deliver against the goals in the data environment.

●Exposure to Cloud env’s (AWS & Microsoft azure) and hands on NoSQL databases

●Past experience leading as Team Lead and training/mentoring less experienced personnel

●Documentation for data solutions (data models, data dictionaries, data catalog, data process flows and architecture diagrams)

EDUCATION:

●Bachelor of Engineering – Mechanical -1990

Kavikulguru Institute of Technology and Science, Ramtek, Maharashtra, India

●Post Graduate Diploma in Personal Management & Industrial Relations.1992

Kakatiya University, Warangal, Telangana, India.

TRAINING AND CERTIFICATES

Training Informatica certificate on Cloud Mapping Designer basic transformation.

Training Informatica certificate on Data Quality Running DQ Exception in Hadoop.

Training Informatica certificate on Cloud Data warehouse and cloud data lake.

Informatica Certification on Cloud Integration for Application Modernization

Informatica Certification on Cloud Lakehouse Data Management Foundation.

Training Certificate on Power Center Developer II from Informatica

Training Cloudera certificate on Data Analyst, & Developer training on Spark & Hadoop.

Training Talend certificate on Talend Studio for Data Integration- ELT.

IBM Cognos Certified –BI Authors & Training IBM certificate on DataStage

Training certificate on ERP Systems (Baan -Logistics) trained from NITIE, Mumbai, India.

Training workshop on Scrum Alliance from http://3back.com

Workshops on Data marts by Ralph Kimball and AWS Products

Python Boot Camp & Cassandra Workshop- 8 weeks programmed hands on experience for data analysis.

Big Data University Courses & Certifications Awarded.

(i)Hadoop & HBase for Real-time Access to Big Data certification. (ii)Accessing Hadoop Data Using Hive certification (iii) Moving Data into Hadoop certification. (iv)Big Data, MapReduce and Yarn certification (v) Training and pass certification in python 101.

GM Learning & training certifications

(i) Installation of Hadoop clusters and cloudera Manager(ii) Ecosystem for Hadoop (iii)Apache Hadoop (iv)Data Factory with Hive, Pig, Ozzie & Hue (v) Data Flow for the Hadoop Ecosystem (vi) Data Repository with Sqoop.

Agile Project Management with Scrum.

Training Netezza certification of Completion – NPS Appliance Usage 5.0

Training MapR certificate on Essentials & Data Analysts courses.

Training certificate from Pluralsight on Snowflake and AWS Products

Training certificate from plural sight in Microsoft Azure fundamental course AZ-900

AWS Data bricks cloud data lake Development workshop.

RECOGNITION

Recognized as Best Employee from IT Department -- BBVA Compass -2009.

Professional Experience

Availity.com. April 2021-Present

Data Service Providers facilitates billions of clinical, administrative, and financial transactions annually with tools for patient eligibility and benefits, claims status and authorization to facilitate the business side of health care for providers, health insurance and Members.

Responsibilities:

--Data Analysis, Daily handling Data Reconciliation process, loading the History data and patching missing data in different domains (Claims, Claim status, HL7, Eligibility Benefits, Transactions, ERA and HCSR)

--DLQ (Dead Letter Queue data – error handling data analysis) and Patching scheduling on DAG Airflow schedular.

---Loaded the history data into different stages in AWS S3 buckets and finally into Redshift database

---Handling the Billing transactions to bill to the customer’s monthly for all the Domains.

---Production Support Jobs on AWS Cloud -Apache DAG Airflow, Informatica Power Center -Batch and Realtime Applications.

---Development ETL Informatica mappings/sessions/Worklets/Workflows and scheduling on Jams schedular tool.

Highlights:

●Production support, Design, Development, ETL Mapping document, Data analysis, Bug fixing issues, Data Quality and Data patching process.

Using Informatica PowerCenter, RDBMS (Oracle, SQL Server), AWS Cloud (Redshift, EMR, EC2,S3,DynamoDB,CloudShell)

Jupyter Notebook), Python, pyspark and Scala (Knowledge)

●Knowledges on Health HIPAA X12 EDI transaction sets.

M&T Bank: Technical Engineer: October 2019- April 2021

Credit Risk System (CRS): In-progress

Credit Risk System for Commercial Loans that currently exists in Microsoft SQL Server database. Business business wanted to migrate into Big Data Platform for Machine Learning and Artificial Intelligence solutions.

Highlights:

●Involved in current (understand /design/development/load process/), hands on development and reviewed code and support.

●Core bdm mappings (take recent and gm projects in nice way),

●Social media, customer experience,OEM,Inventory,

●Data Gap analysis on existing data mart and data lake data to identify the entities, attributes, analyzing and profiling, analysis on the raw data for important decisions by identifying various facts and trends for downstream.

●Creating Data Model in different Layers (raw/stage/int/consumer and extract)

●Design and develop Big Data applications for processing commercial loans data.

●Integrated Hadoop into traditional ETL, accelerating the extraction, transformation, and loading into different layers and aggregate data creating the Files for stakeholders.

●Scheduling and creating workflows and objects.

Customer Profitability: CDP/BDM Upgrade

Existing application integrated different sources: commercial loans, small business, direct deposits, dealer floor plan and Mortgage loans in Hadoop ecosystem built on Cloudera platform with BDM informatica for downstream stakeholders reports and for OFSAA application

Highlights:

●Coordinating with the Production team for source daily ebcdic files into downstream env’s (dev/test/cert/) to test Jobs.

●Coordinating with the Production team for source daily ebcdic files into downstream env’s (dev/test/cert/) to test Jobs.

●Hands on executing BDM mappings, validating the data at each individual stages with production w.r.t specific business date.

●Optimized ETL Jobs to improve the performance

●Data validation coordinating with the Data Analysts team for approval to deploy.

●Created change control process for production deployment

● Support Level 1 & 2 proactive monitoring daily batch jobs, resolving the tickets raised meeting the SLA’s and informing if any issues to stakeholders.

●Accountable to deliver the applications in Agile environment on time delivery through agile methodology principles.

●Problem solving mindset working in an agile environment.

●Knowledge sharing among peers for process improvement.

Credit Lifecycle Management (CLM): Operational Data Store (ODS):

This application is integrated up streams daily batch files processing into Oracle, Teradata databases and CRM Sales force with all the financial credit information (commercial, retail and construction loans) in order to track and monitor downstream reporting in order to improve the business.

Highlights:

●Identified gaps in the data processes and drove the improvements via continuous improvement loop (Data process) for performance and scalability.

●Worked on the transformation logic (bug fixes), in the existing jobs.

●Data analysis for new source to integrate, providing the details to Business owner

●Designed the Data Models at different stages.

●Data flow diagram, Design document and ETL mapping specification coordinating with Data Analyst team.

●Designed, development, test and deploying into production.

●Change control process for production deployment.

GENERAL MOTORS: April 2016- September 2019

IT Data Engineer/Data Intelligence Analyst

Market Investment Optimization (MIO):

Migrating into the Hadoop ecosystem from existing Microsoft access and Excel. based on the external vendors landing the files for Marketing strategy analysis for the stakeholders reporting based on running the quarterly campaigns.

Highlights:

●Integrated into Hadoop ecosystem using Apache HiveQL into traditional ETL, accelerating the extraction, transformation, and loading of the massive structured and unstructured Google, JD Edwards & Adobe data (social media and campaign’s -impression, clicks and execution) into HDFS using Hive Data warehouse tables into different layers Raw & Staging.

●Data analysis by improving data quality, reliability and efficiency of individual source components for downstream processes.

●Created Models in different layers and ensured architecture meets the business requirements in HDFS and Hive tables.

●Loaded the aggregate data into a reporting Layer for reporting and ad-hoc analysis, which to improve processes for low latency and operating costs and offset the rising cost of existing programming.

●High- and Low-Level design frameworks, data flow diagram and ETL Mapping Specification, Unit Testing Templates.

●Worked closely with data architects, solution architect’s business and data analysts

●Weekly Meetings with Stakeholders and followed Agile process for any changes

Global Business Service (GBS):

Ad-hoc projects for Analytics.

Highlights:

Design, development, Testing and production support with Design documents (ETL Mapping specifications) for the below projects worked in Agile Methodology.

●Indirect material Inventory: Ingested and integrated from all 54 for plants North America Region process loading into Teradata/Aster Database and HDFS to Hive and query using BigSQL and also using IBM Big insights web console for import and export Files into HDFS for Analytical forecasting patterns.

●Dealer Optimization analysis (DrMo) monthly North America sales analysis & CRM Maven

●Recall- Effectiveness: Reporting the vehicle warranty claims information as per campaigns, type of service labor operation occurred on vehicle at dealer shop in order gauge the different level of campaigns.

●Dealer Risk Management and Optimization: Identifying incentive and warranty program, sales and service provided by the dealers for cars.

GENERAL MOTORS: May 2013- March 2016

Global Warranty Management (GWM):

Upgrade project for Dealers claim charges for warranty vehicles for any vehicle related repairs/maintenance, integrating with upstream Files, SAP-IDOC and Oracle databases using the DataStage ETL daily batch process.

Highlights: Technical Lead

Implemented migration for several applications from See-beyond to DataStage 11.3 by providing guidance to team members on all aspects of software development compliance with design technical frameworks and enabling the coding standards.

●Provided accurate estimates, identify dependencies and risks and own all development activities.

●Support cross function activities involving data governance and analysis, mapping document specifications, functional and test cases templates

●Install Operating systems (Solaris, Linux), data stage software installation, databases, setup SAP integration and exciting hands on ETL jobs by comparing with existing and new platforms for downstream deployment into test/pre-prod and production.

●Collaboration with stakeholders, report progress and escalate any issues to their attention

●Engage SME for upstream and downstream interfaces to complete UAT for deployment into the Prod environment.

ITAC SOLUTIONS, Birmingham, AL: March’2011-May’2013

Client: BBVA Compass Bank, Birmingham, AL

Highlights: ETL Consultant: Individual and Team Member contributor in Setting up the dev principles and delivery standards. Designed, build ETL Mapping spec, Testing & supporting the ETL Deployed jobs for below projects:

(i)Enterprise Information System (EIS) – DM (ii)Basel 3-New sources Integration into EDW. (iii) CRM-DM (iv)Non-Consumer Revenue Portfolio-DM (v) E3 (Mortgage & Insurance) -DM (vi)Conversion Project Windows to AIX ETL Jobs.

ROSE INTERNATIONAL, Jefferson City, MO: Sept 2010–March 2011

Client: Bank of America, Charlotte, NC

Highlights: Technical Analyst

Technology Conversion Project: Enterprise Credit Risk Project for POC using tool: Talend, Data Quality Mpx (ELT), Netezza (Database), Informatica (Power Center), Subversion, Perforce Tools.

BBVA COMPASS BANK, Birmingham, AL: March 2008 – Sept 2010

Highlights: Sr. Lead Programmer Analyst /ETL Architect

●Designed, Development and Administration tasks for different applications.

●Analysis on the source system and created etl mapping specifications and if any enhancements for the existing source systems.

●Requirements review, design of ETL process flow designs/architecture/framework, track progress, code/design documents to ensure data quality to meet standards

●Track of production ETL loads, identify SLA issues and inefficient jobs and metadata daily/weekly/monthly reports.

●Administrative activities for Informatica (Power Center) integration and Repository Services, Informatica IDQ support on AIX, Solaris and windows-support installation and applying hotfixes.

●New Development/Support/Maintenance for Data Marts for Credit Risk (CRDB), Collections Data Mart, Credit Origination and Navigation, Retail Incentive (RTLINC), Anti Money Laundering (AML-DIS Files to Mantas), Visa Payroll source into EDW, Treasury Management System (TMS), SBA and CPS.

DAIMLERCHRYSLER FINANCIAL SERVICES, Detroit, MI

March 2003 – March 2008

Highlights: ETL Consultant

SPOT (Single Point of Truth) – Data Warehouse

●Design, Development, testing using ETL in order to consolidate reports from data stored in over 50 different operations systems for DCS business leaders and key decision-makers throughout North and South America.

●Production Support for daily batch Jobs.

MBC Commercial Vehicle MORIS –

(Management Operational Reporting Intranet Solution) Data Mart Project

●Created data flow diagram & ETL Mapping specification document.

●Designed, development, testing data validation and support the application taking care of complete ETL Process

●Created change request process for deployment to Dev to test to Prod env.,

●Worked closely with stakeholders, BI Team, solution architect and Data Architect.

Dealer Profitability – Truck Finance – Data Mart Project

●Developed a process for operations management to better analyze the profitability of their Portfolios use to enhance management’s ability to price business profitability from a different perspective, allowing management to identify top sales performers.

Opti load Conversion – Data Warehouse:

Since its evolution in DCS organization, Opti load replacement to replace the current Production Opti load jobs with Informatica Power Center / Power Connect (Striva) tools.

Highlights:

●Requirements gathering and designing Logical and Physical Model using Power Designer 9.0

●Designed Star Schema Model for Dealer ROE Project and comprehensive Data Dictionary Documents. ETL Mapping Specification Documents for Development.

●Developed hands on mappings (@200+) / mapplets using Informatica – Power center (Designer) sessions & workflow using Workflow Manager.

●Involved in testing of mappings, which developed extensively using Debugger.

●Informatica Administrative (users, Groups and Privileges, folder organization and backups), Documented the Operational Manuals for the developed

●Involved in Installation / Upgrade of software Installation PowerCenter / Power Exchange 5.1.2.

Contact this candidate