Azure Data Engineer (5yrs)- joiner

Designed Azure ADF pipelines to move data from Open hubs to Azure Data Lake Gen2 and then to Azure Data Warehouse.

Developing notebooks and applying transformation using spark SQL in Azure data bricks.

Testing the notebook and validating the scripts using SQL.

Implemented activities Copy activity, Execute Pipeline, Get Meta data, If Condition, Lookup, Set Variable, Filter, For Each pipeline Activities for On-cloud ETL processing.

Load the files to stage tables and then to data warehouse tables in dedicated SQL pool in synapse. Writing views, stored procedures for transformations.

Converting views to external tables for direct query for faster retrieval.

Preparing views data, data models for PBI reports Designed dashboards visuals with slicers, filters, drill through options.

Assure that data is cleansed, mapped, transformed, and otherwise optimised for storage and use according to business and technical requirements Develop and maintain innovative Azure solutions Solution design using Microsoft Azure services and other tools The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.)

Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications Build data pipelines to collectively bring together data Other responsibilities include extracting data, troubleshooting and maintaining the data warehouse

Role: Data Engineer (4-8 year relevant exp)

Primary skills: Advance SQL/DWH, Strong is SQL hands on experience in writing complex queries/stored procedure, Azure Data Factory, Azure Data Bricks, ADLS)