VAISHALI SHUKLA

ad4l2v@r.postjobfree.com

913-***-****

PROFESSIONAL EXPERIENCE

Experienced Azure Data Engineer and ETL Developer with 8+ Years of Expertise in Data Solution, Design and Maintenance.

Hands-on experience in Azure Cloud Services (PaaS & IaaS), Azure Synapse Analytics, SQL Azure, Data Factory, Azure Analysis services, Application Insights, Azure Monitoring, Key Vault, Azure Data Lake.

Expertise in designing and implementing ETL pipelines using Azure Data Factory, Data bricks, and Azure SQL Database for real-time and batch data processing.

Proficient in Data Modeling, Data Integration, and Data Transformation using technologies like Azure Synapse Analytics, Data Lake Storage, and HDInsight.

Strong experience in writing complex SQL queries for Data Analysis, optimization, and performance tuning.

Hands-on experience in designing and implementing scalable and secure data architectures using Azure services like Azure Cosmos DB, Azure Stream Analytics, and Azure Event Hubs.

Designed and developed ETL pipelines in Azure cloud to process customer data from API and load it into Azure SQLDB.

Experience in using Azure Key Vault as a central repository for maintaining secrets and referencing the secrets in Azure Data Factory and Databricks notebooks.

Proficient in maintaining and supporting Teradata architectural environments for EDW Applications.

Proficient in logical modeling, physical database design, data sourcing, data transformation, data loading, SQL, and performance tuning.

Proficient in creating proper Teradata Primary Indexes (PI) taking into consideration both planned access of data and even distribution of data across all the available AMPS.

Hands-on experience in developing Data Extraction, Transformation and Loading jobs from flat files, Oracle, SAP, and Teradata Sources into Teradata using BTEQ, FastLoad, FastExport, Multi-Load and stored procedure.

Skilled in developing custom data solutions using Python and Spark, and automating data workflows using Azure Logic Apps and Azure Functions.

Proficient in implementing data security and compliance measures using Azure Key Vault, Azure Active Directory, and Azure Security Center.

Experience in working on Informatica advanced concepts and implementation of Informatica Pushdown Optimization technology and pipeline partitioning.

Experience in performing bulk data load from multiple data sources (ORACLE 8i, legacy systems) to TERADATA RDBMS using BTEQ, MultiLoad, and FastLoad.

Used Python and Shell scripts to automate Teradata ELT and admin activities.

Proven ability to work collaboratively with cross-functional teams including business stakeholders, data scientists, and developers to deliver high-quality data solutions on time and within budget.

Excellent communication and presentation skills, with a proven ability to explain complex technical concepts to non-technical stakeholders in a clear and concise manner.

TECHNICAL SKILLS

Azure Data Services

Azure Data Factory, Azure Databricks, Azure Stream Analytics, Azure HDInsight, Azure Event Hubs, Azure Cosmos DB, and Azure SQL Database.

Data Warehousing

Azure Synapse Analytics, Azure SQL Data Warehouse.

ETL TOOLS

Azure Data Factory, Azure Databricks, Informatica, Talend.

CLOUD TECHNOLOGIES

Microsoft Azure Cloud, AWS, or Google Cloud.

DATA VISUALIZATION & REPORTING TOOL

Power BI & Tableau.

PROGRAMMING LANGUAGES

Python, Scala, SQL, Perl, Java.

MICROSOFT TOOLS

SSIS, SSAS, SSRS, Microsoft Visio, Microsoft Excel.

PROFESSIONAL EXPERIENCE

WireCo, Overland Park, KS April’22 - Present

Sr. Data Engineer

Responsibilities:

Develop, design data models, data structures and ETL jobs for data acquisition and manipulation purposes.

Develop deep understanding of the data sources, implement data standards, maintain data quality and master data management.

Expert in developing JSON Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data.

Expert in using Databricks with Azure Data Factory (ADF) to compute large volumes of data.

Performed ETL operations in Azure Databricks by connecting to different relational database source systems using JDBC connectors.

Developed Python scripts to do file validations in Databricks and automated the process using ADF.

Developed an automated process in Azure cloud which can ingest data daily from web service and load in to Azure SQL DB.

Developed Streaming pipelines using Azure Event Hubs and Stream Analytics to analyze data for dealer efficiency and open table counts for data coming in from IOT enabled poker and other pit tables.

Analyzed data where it lives by Mounting Azure Data Lake and Blob to Databricks.

Used Logic App to take decisional actions based on the workflow.

Developed custom alerts using Azure Data Factory, SQLDB and Logic App.

Developed Databricks ETL pipelines using notebooks, Spark Dataframes, SPARK SQL and python scripting.

Developed complex SQL queries using stored procedures, common table expressions (CTEs), temporary table to support Power BI reports.

Implemented complex business logic through T - SQL stored procedures, Functions, Views and advance query concepts.

Worked with enterprise Data Modeling team on creation of Logical models.

Development level experience in Microsoft Azure providing data movement and scheduling functionality to cloud-based technologies such as Azure Blob Storage and Azure SQL Database.

Independently manage development of ETL processes - development to delivery.

Environment: ETL, Azure Data Factory (ADF), Azure Databricks, Azure Event Hubs, Stream Analytics, Azure Data Lake, Azure Blob Storage, and Azure SQL Database, T-SQL etc.

Morgan Stanley, Overland Park, KS July’2019 – March’2022

Data Engineer/ETL Developer

Responsibilities:

Involved in Requirement gathering, business Analysis, Design and Development, testing and implementation of business rules.

Understand business use cases, integration business, write business & technical requirements documents, logic diagrams, process flow charts, and other application related documents.

Used Pandas in Python for Data Cleansing and validating the source data.

Designed and developed ETL pipeline in Azure cloud which gets customer data from API and process it to Azure SQLDB.

Orchestrated all Data pipelines using Azure Data Factory and built a custom alerts platform for monitoring.

Created custom alerts queries in Log Analytics and used Web hook actions to automate custom alerts.

Created Databricks Job workflows which extracts data from SQL server and upload the files to SFTP using Pyspark and python.

Used Azure Key vault as central repository for maintaining secrets and referenced the secrets in Azure Data Factory and also in Databricks notebooks.

Built Teradata ELT frameworks which ingests data from different sources using Teradata Legacy load utilities.

Built a common SFTP download or upload framework using Azure Data Factory and Data bricks.

Maintain and support Teradata architectural environment for EDW Applications.

Involved in full lifecycle of projects, including requirement gathering, system designing, application development, enhancement, deployment, maintenance and support

Involved in logical modeling, physical database design, data sourcing and data transformation, data loading, SQL and performance tuning.

Project development estimations to business and upon agreement with business delivered project accordingly

Created proper Teradata Primary Indexes (PI) taking into consideration of both planned access of data and even distribution of data across all the available AMPS.

Considering both the business requirements and factors, created appropriate Teradata NUSI for smooth (fast and easy) access of data.

Developing Data Extraction, Transformation and Loading jobs from flat files, Oracle, SAP, and Teradata Sources into Teradata using BTEQ, FastLoad, FastExport, MultiLoad and stored procedure.

Design of process oriented UNIX script and ETL processes for loading data into data warehouse

Developed mappings in Informatica to load the data from various sources into the Data Warehouse, using different transformations like Source Qualifier, Expression, Lookup, aggregate, Update Strategy, and Joiner

Worked on Informatica Advanced concepts & also Implementation of Informatica Push down Optimization technology and pipeline partitioning.

Performed bulk data load from multiple data source (ORACLE 8i, legacy systems) to TERADATA RDBMS using BTEQ, MultiLoad and FastLoad.

Used various transformations like Source qualifier, Aggregators, lookups, Filters, Sequence generators, Routers, Update Strategy, Expression, Sorter, Normalizer, Stored Procedure, Union etc.

Used Informatica Power Exchange to handle the change data capture (CDC) data from the source and load into Data Mart by following slowly changing dimensions (SCD) type II process.

Used Power Center Workflow Manager to create workflows, sessions, and also used various tasks like command, event wait, event raise, email.

Designed, created and tuned physical database objects (tables, views, indexes, PPI, UPI, NUPI, and USI) to support normalized and dimensional models.

Created a cleanup process for removing all the Intermediate temp files that were used prior to the loading process.

Used volatile table and derived queries for breaking up complex queries into simpler queries.

Responsible for performance monitoring, resource and priority management, space management, user management, index management, access control, execute disaster recovery procedures.

Used Python and Shell scripts to Automate Teradata ELT and Admin activities.

Performed Application level DBA activities creating tables, indexes, and monitored and tuned Teradata BETQ scripts using Teradata Visual Explain utility.

Performance tuning, monitoring, UNIX shell scripting, and physical and logical database design.

Developed UNIX scripts to automate different tasks involved as part of loading process.

Worked on Tableau software for the reporting needs.

Worked on creating few Tableau dashboard reports, Heat map charts and supported numerous dashboards, pie charts and heat map charts that were built on Teradata database.

Environment: Azure Cloud, Azure Data Factory, Databricks, Azure SQLDB, Teradata RDBMS, Tableau, Pandas, Log Analytics, Azure Key Vault, Informatica, Python, Pyspark, SQL, Shell Scripts etc.

CareFirst, Ashburn, VA Jan’2018 – June’2019

Data Analyst/Engineer

Roles & Responsibilities

Designed and Developed Tableau dashboards from scratch to generate actionable insights and solutions for client services.

Unified dashboard which displayed Implementation and Opportunity stages; Opportunities won by sales rep, features most used, Live customer count, breakdown by Persona of users etc. which helped in increasing product’s sales by 4 % and improving the overall user experience.

Created databases and schema objects including tables, indexes, and applied constraints, connected various applications to the database and written functions, stored procedures, and triggers.

Build and published customized interactive reports and dashboards, report scheduling using Tableau server.

Created action filters, parameters, and calculated sets for preparing dashboards and worksheets in Tableau.

Designed, developed, and implemented BI solutions for Sales, Product and Customer KPIs.

Involved in re-designing of database to create de-normalized reporting schema for better throughput.

Worked in both Waterfall and Agile Methodology.

Experience in Troubleshooting Report Processing, Subscription, Delivery and Configuration Problems.

HR Predictive Modeling: Fetched employee data from database. Pre-processed data using Python/Java and identified trends in attrition and retention rate over the years using Tableau. Created Dashboards and Story to convey the analysis & predictions.

Created Stored Procedures, User-defined Functions, designing and implementing of Database Triggers, Views, and Indexes.

Excellent report creation skills using Microsoft SQL Server Reporting Services (SSRS) with proficiency in using Report Designer as well as Report Builder.

Experience in designing dashboards and reports, parameterized reports, predictive analysis in Tableau.

Performed day-to-day Database Maintenance tasks including Database Monitoring, Backups, Space and Resource Utilization.

Identified, documented, and created linking diagrams to Excel using Microsoft Visio.

Environment: Matplot, Oracle 11G, SQL Server Reporting Service, MS Access SQL, Tableau, Power BI, Microsoft Visio.

HighRadius, India June’2015 – Dec’2017

Data Analyst

Roles & Responsibilities

Worked through all the phases of Software Development Life Cycle (SDLC) including Requirements Gathering, Analysis, Design, Development, Testing, Production and Post-Production Support.

Developed Stored Procedures and User Defined Functions for providing input feed for front end applications.

Created complex Stored Procedures, Triggers, Functions, Indexes, Tables, Views, and joins.

Designed and worked on Extraction, Transformation and Loading (ETL) process by pulling up large volume of data from various data sources using SSIS.

Developed a 360-business dashboard in Tableau with multiple panels and parameters for Salesforce team.

Map sources to targets using Excel Macro functions, SQL scripts, including BusinessObjects Data Services/BODI.

Design and develop ETL processes. Establish coding standards, perform Peer review, and automate the health of the platform.

Develop mapping/sessions/workflows.

Developed complex SQL queries using stored procedures, common table expressions (CTEs), temporary table to support SSRS reports.

Actively involved in developing Complex SSRS Reports involving Sub Reports, Matrix/Tabular Reports, Charts and Graphs.

Worked on the data warehouse design and maintained different dimension and fact tables.

Installed/configured SQL Server in Virtual Environments using VMWare.

Reduced the timeline of individual migration projects by several months through optimization and documentation.

Environment: Microsoft Server Integration Services, MS Excel Macros, SSRS, VMware.

--

Thank you.

Contact this candidate