Data engineer

Priya Ethirajan

phone: (***) *** – ****

Email: *****.*******@*****.***

Professional Summary

•Results oriented, resourceful, and problem-solving Data engineer on Azure Cloud.

•Experience with Azure Big Data and Data transformation implementations

•Experience on Migrating SQL database to Azure data Lake, Azure data lake Analytics, Azure SQL Database, Data Bricks and Azure SQL Data warehouse and controlling and granting database access and Migrating On premise databases to Azure Data Lake store using Azure Data factory.

•Manage SQL Server databases through multiple project lifecycle environments, from development to mission-critical production systems.

•Extensive experience with Migrating Databases residing on servers in one data center to another and Upgrading Database platforms to newer versions.

•Experience with Azure Data Factory (ADF), Integration Run Time (IR), File System Data Ingestion, Relational Data Ingestion.

•Good client communication skills with excellent work ethics and a proactive team player

Technical Skills

•Azure Data Lake, Azure Data factory

•Azure Databricks, Azure SQL database, Azure SQL Datawarehouse

•SQL, Python

•Data Visualization (Tableau, Power BI)

•Data Migration

•Postgres SQL

Certifications:

Azure - AZ 900

Udemy – PostgreSQL

Intellipath - Tableau

Work Experience

Apr 2021 – Present Cognizant Technology Solutions

Analyze, design, and build Modern data solutions using Azure PaaS service to support visualization of data. Understand current Production state of application and determine the impact of new implementation on existing business processes.

Experience with Extraction Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and Azure Data Lake Analytics. Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks.

Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse.

Built dynamic pipelines to migrate data from Source Systems to Azure

Experience using Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics.

Experience with processing streaming data with SQL query in Stream Analytics.

Sample of few Projects executed

Built ETL pipelines with Databricks

Developed spark Applications using Spark SQL in Databricks for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data.

Built and Monitored Databricks pipelines. Changed code logics according to the Business needs.

Stored data in Cosmos DB from Databricks to Show the data in the Front-end application for Customer usage.

Good understanding of spark Architecture including spark SQL, Data Frame, Driver Node, Worker Node.

Created data pipeline in ADF using Linked services/datasets/pipelines to extract, Transform and load data from different sources such as Azure SQL, Azure data lake, Blob storage.

Oracle to SQL Migration using Azure Data Factory:

Use ADF to Migrate the data from Oracle to Azure SQL DB

Use Databox to Migrate TBs of data from oracle to Azure.

Worked with internal teams, partners, and customers to identify and define data conversion requirements

Validate, test, and maintain all developed data conversion models to ensure successful upload of data from Oracle to Azure

Analyzed existing conversion processes to identify and address problems impacting the quality of the data conversions

Documented and created best practices for developing, executing, and maintaining all aspects of the Data migration

Participated in and occasionally lead cross-functional internal process improvement projects

DB2 to Azure Migration using Azure Data Factory:

Created Database objects like stored procedures, sequences

Create ADF pipelines to migrate data from db2 to Azure

Migrated database schemas using SQL server migration Assistant

Transformed and stored streaming data in Azure Data Lake and processed it through Databricks and storing it in SQL DB.

Process streaming data using stream Analytics and storing the data in Azure Data Lake

Mar 2018 – Apr 2021 GG Tech Solutions

Optimized data collection procedures and generated reports on a weekly, monthly and quarterly basis using PostgreSQL.

Presented findings and data to team to improve strategies and operations.

Collected and analyzed data on new loyalty members, marketing results and trends.

Develop reports using SQL to provide insights into route optimization.

Developed JSON Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the SQL Activity

Education & Training

April 2021 – June 2021 Cognizant Bootcamp on Azure Services

Jan 2006 – Dec 2010 PhD in Applied Psychology & Statistics

Jul 1999 – Apr 2004 BS & MS in Applied Psychology & Statistics.

Contact this candidate