Sql Server Azure Data

Gopal Udayagiri

Azure Databricks and Azure Data Engineer

Email: *********@*****.***

Phone: +1-571-***-****

SUMMARY

●13+ years of experience in Business Intelligence Using Microsoft SQL Server with 6 plus years exposure to Cloud Architecture Design, Modelling, Development, Testing, Maintenance using Microsoft Azure Cloud Tools like Databricks, Azure Dat Factory, Azure Data Lake Storage, Azure Cosmos NO SQL DB and Azure HD Insight.

●Hands on Experience on Unified Data Analytics with Data Bricks, Databricks workspace user interface, Managing Databricks Notebooks, Delta Lake with Python, Delta Lake with Spark SQL.

●Experience working in reading Continuous Json data from different source system using Apache Kafka into Databricks Delta and processing the files using Apache Structured streaming, Pyspark and creating the files in parquet format.

●In-Depth understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames and Spark Streaming.

●Strong Knowledge on Power BI to import data from various sources such as SQL Server, Azure SQL DB, SQL Server Analysis Services (Tabular Model), MS Excel etc.,

●Experience in Extracting, Transforming and Loading (ETL) data from Excel, Flat file to MS SQL Server using SSIS.

●Experience in managing and automating control flow, data flow, events and logging programmatically using Microsoft .NET framework for SSIS packages.

●Experienced working on OLAP database development including KPIs, Data Mining.

●Expertise in designing complex reports like Dashboard Reports, Drill-Down Reports, Parameterized Reports, Cascaded Reports and Sub Reports using SQL Server Reporting Services based on client requirement.

●Expert in creating complex Stored Procedures, User Defined Functions, DDL/DML Triggers, Views, Cursors and Indexes to facilitate efficient data manipulation and data integrity in Teradata.

●Experience in Azure SQL, Azure data lake, Azure data factory, Azure Blob storage and azure pipelines.

●Good Hands-on Experience in Azure Data Factory Pipelines and building ETL Data Flows in ADF.

●Logged all types of objects in JIRA/Remedy from the requirement specification issues, mapping documentation defect and database issues.

EDUCATION

Master’s in science and engineering, Oklahoma Christian University, USA

Bachelor of Engineering from JNTU, INDIA

TECHNICAL SKILLS

●Database: MS SQL Server, MS Access, PostgreSQL, Oracle 10g, 11,12.

●Programming Languages: T-SQL, SQL, PL/SQL, C, HTML, DHTML, XML, VB.NET, C#.Net, SharePoint.

●ETL Tools: SQL Server Integration Services (SSIS), Data Transformation Service (DTS), Tableau, SQL Server Business Intelligence Development studio, Enterprise Manager, SQL Profiler, Query Analyzer, Import & Export (DTS), SSAS.

●Reporting Tools: SQL Server Reporting Services (SSRS), Crystal Reports, Report Studio, Power BI, Tableau.

●Cloud Computing: Apache Kafka, Apache Spark, Azure SQL, Azure data Lake, Azure Data Bricks, Power BI, Data Factory, Azure DevOps, Spark Data Frame API, Spark Programming, Manage Cluster Databricks

PROJECT DETAILS

Ahold Delhaize, Salisbury, NC, USA Mar 2021 – Till Date

Senior Data Engineer (Azure Databricks and Data Factory).

Responsibilities:

●Design Logical Data Model using notebooks in Azure Databricks.

●Orchestrated comprehensive data processing workflows leveraging Azure Databricks and Apache Spark to enhance large-scale data transformations and advanced analytics, achieving a notable 14% improvement in data processing speed.

●Extract Transform and Load data from Sources system to Azure Data Storage Services using a combination of Azure Data Factory, T-SQL, Spark SQL.

●Worked on migration and conversion of data using Pyspark and Spark SQL for data extraction, transformation and aggregation from multiple file formats for analyzing and transforming from Databricks Notebooks using Python.

●Responsible for estimating the cluster size, monitoring and troubleshooting of the Spark Databricks Cluster.

●Analyzed SQL scripts and designed it by using PySpark SQL for faster performance.

●Migrated on -Prem ETL’s from MS SQL Server to Azure Cloud using Azure Data Factory and Data Bricks.

●Worked on migration of data from On-prem SQL server to Cloud databases (Azure Synapse Analytics (DW) & Azure SQL DB).

●Conversion of Data using Data Factory from On-Prem to Azure SQL Databases.

●Performed ETL Transformation activities in Data Factory and built several pipelines and loaded data to Azure SQL Databases.

●Led the migration of Hive metastore tables from Databricks and other point-of-sale tables to Unity Catalog, streamlining data management processes and ensuring centralized access for enhanced efficiency Scheduled Jobs in Azure Data Factory and monitor pipeline runs and Debug pipeline failures.

●Design and Develop Data Visualizations in Power BI.

●Involved in Development & Deployment of Azure Data Factory Pipelines via IDE, Portal and CI/CD pipelines Development & Deployment of Azure Databricks codebase in PySpark via IDE and CI/CD pipelines.

Environment: Databricks, Azure Data Lake & BLOB, Azure SQL, Azure data factory, Data Bricks, Management Studio (SSMS), ETL, Integration Services (SSIS), Azure Data Bricks, Spark 3.1.2 Cluster, Git Repository Management

U.S. Department Of Veteran Affairs (Booz Allen Hamilton), VA April 2020 – Feb 2022

Senior Integration Engineer (SSIS and Azure Data Factory)

Responsibilities:

●Design and implement end-to-end data solutions (storage, integration, processing, visualization) in Azure.

●Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL.

●Migrate data from traditional database systems to Azure databases.

●Build Complex distributed systems involving huge amount data handling, collecting metrics building data pipeline, and Analytics.

●Health Checks on VA ICAMP Page Dashboard and Debugged Azure Data Factory pipeline Failures.

●Worked with Meta-Data Driven SSIS Packages to pull the data from different Sources and load to Data mart.

●Manual Executions on OIG (Office of Inspector General) Data Loads by migration Data from Nessus Scans to OIG Audit and Process SSIS Packages and produce IG Findings Reports and Validate.

Environment: Azure SQL, Azure data factory, Management Studio (SSMS), ETL, Integration Services (SSIS), DevOps, Git

Unilever, Englewood Cliffs, NJ Jan 2018 – Feb 2020

Data Engineer

Responsibilities:

●Build Complex distributed systems involving huge amount data handling, collecting metrics building data pipeline, and Analytics.

●Worked on Azure DataBricks workspace, mounting, Delta DB setup and stored Schema on

External SQL Server and used cluster management for decreasing runtime of the pipeline by

17%.

●Worked on NiFi to ingest data from multiple sources, transform enrich and load data into Kafka.

●Experience managing Azure Data Lakes (ADLS) and Data Lake Analytics and an understanding of how to integrate with other Azure Services like Synapse and Azure Data Factory.

●Developed and created pipelines, jobs, scheduling triggers, Mapping data flows using Azure Data Factory(V2) and using Key Vaults to store credentials.

●Analyzed the sql scripts and designed it by using PySpark SQL for faster performance.

●Worked on Azure BLOB and Data Lake storage and loading data into Azure SQL Synapse analytics (DW).

●Had used T-SQL to write stored procedures, triggers, functions, tables, views, indexes and relational database models.

●SSIS performance tuning using counters, error handling, event handling, re-running of failed SSIS packages using checkpoints and scripting with Active-X and VB.NET in SSIS.

●Development using SSIS script task, look up transformations and data flow tasks using T- SQL and Visual Basic (VB) scripts.

●Transferred the data (ETL) to data warehouse by SSIS and processed SSAS cubes to store data to OLAP databases.

●Performance Monitoring with SQL Profiler Windows System Monitor.

●Contributed to the project's overall understanding of Indiana Medicaid Management Information System Core MMIS.

Environment: Databricks, Azure Data Lake & BLOB, Azure SQL, Kafka, Azure data factory, MS SQL Server 2018, Management Studio (SSMS), ETL, Integration Services (SSIS), DevOps.

Wells Fargo, Charlotte, NC, USA Apr 2017 – Dec 2017

SSRS Developer

Responsibilities:

●Maintained Microsoft Team Foundation Server TFS to allow for secure code, Reports and use as a repository to compare modified and unmodified code.

●Creating reports using SQL Reporting Services (SSRS) for customized and ad-hoc Queries.

●Wrote complex queries for the drill down reports and used conditional formatting using SSRS.

●Scheduled and monitored all maintenance activities of SQL Server 2008 including database consistency checks and index defragmentation.

●Actively supported business users for changes in reports as and when required using SSRS.

●Analyzed reports and fixed bugs in stored procedures using SSRS.

●Designed and optimized indexes, views, stored procedures and functions using T-SQL

●Helped designing and implementing processes for deploying, upgrading, managing, archiving and extracting data for reporting.

●Performed maintenance duties like performance tuning and optimization of queries, functions and stored procedures.

●Designed high level SSIS architecture for overall data transfer from the source server to the Enterprise Services Warehouse.

Environment: MS SQL Server2005/2008, SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), AWS, MS Visual Studio.NET# 2008, Microsoft TFS (Team Foundation Server), Visual Basic6.0/VB.net, VB Script.

PwC (PriceWaterhouseCoopers), Dallas, TX Apr 2016 – Mar 2017

POWER BI/SQL Developer

Responsibilities:

●Developed dashboard reports using Reporting Services, Report Model and ad-hoc reporting using Report Builder.

●Experience in creating Parameterized reports and Linked reports with thorough knowledge of report serving architecture. (Table, chart and matrix report).

●Experience in using tools like index Tuning Wizard, SQL Profiler, and Windows Performance Monitor for Monitoring and Tuning MS SQL Server Performance.

●Deployed SSIS Package into Production and used Package configuration to export various package properties to make package environment independent.

●Designed SSRS reports with sub reports, dynamic sorting, defining data source and subtotals for the report.

●Created and Migrated Partially Contained Databases within Always On Availability Groups.

●Used Power BI, Power Pivot to develop data analysis prototype, and used Power View and Power Map to visualize reports.

●Published Power BI Reports in the required originations and Made Power BI Dashboards available in Web clients and mobile apps.

●Used Power BI Gateways to keep the dashboards and reports up to date.

Environment: SQL Server 2012 Enterprise, SQL Server 2016, Microsoft Reporting Service (SSRS), SQL Server Integration Services (SSIS), Clustering, Always On, Power BI, Mirroring, Replication.

KPMG, Montvale, NJ Aug 2014 – Mar 2016

SQL BI Developer [SSIS/SSRS]

Responsibilities:

●Implemented Event Handlers and Error Handling in SSIS packages.

●Converted Data Transformation Services (DTS) application to SQL Server Integrated Services (SSIS) as assigned.

●Created packages using SSIS for data extraction from Flat Files, Excel Files, and OLEDB to SQL Server.

●Developed Tabular Reports, ad-hoc reports using SSRS Report Designer.

●Running DBCC consistency checks and fixing data corruption in application databases.

●Configured SSIS packages using Package configuration wizard to allow packages run on different environments.

●Migrated SSIS Packages from SQL Server 2005 to SQL Server 2008.

●Developed complex SSRS reports using multiple data providers, Global Variables, Expressions, user defined objects, aggregate aware objects, charts, and synchronized queries.

Environment: SQL Server 2012 Enterprise, BIDS, Microsoft Reporting Service (SSRS), SQL Server Integration Services (SSIS).

Verizon, Irving, TX Jan 2012 – Aug 2014

MS BI Developer SSAS/Teradata

Responsibilities:

●solved complex business problems by designing, implementing, maintaining and monitoring multi-dimensional cubes using SQL Server Analysis Services (SSAS).

●Designed OLAP cubes making use of Star and Snowflake Schemas.

●Interacted with end-users in requirement gathering sessions, use case analysis, report layout design.

●Used Calculated Member Builder to create custom measures.

●Deploying cubes and reporting services reports to Production servers generated by the offshore team.

●Writing SQL Server Stored procedures/complex queries to retrieve data from Teradata.

●Writing MDX queries to generate OLAP reports.

●Writing DAX queries to generate Excel and Drill to detail reports. Also created power pivot reporting from Analysis Services tabular model.

●Created various complex reports like Dashboard, Drill Through, parameterized reports, Matrix and chart reports according to user specifications.

●Scheduled the Reports to run on daily and weekly basis in Report Manager and also emailing them to end users and analysts to be reviewed in Excel Sheets.

●Writing SSIS packages to load data into Facts and confirmed dim tables to build cubes.

●Created Database Objects – Schemas, Tables, Indexes, Views, User defined functions, Triggers, Stored Procedure, and Constraints.

●Involved in optimizing code and improving efficiency in databases including re-indexing, updating statistics, recompiling stored procedures and performing other maintenance tasks.

Environment: SQL Server 2012 SSIS, SSAS, SSRS, Teradata, Oracle, Visual Studio 2005, Windows Enterprise Server 2003, ADO.NET, Visual Source Safe, XML, HTML, Erwin.

Contact this candidate