Azure Data Engineer

Our client is looking for an experienced Azure Data Engineer to join their team and help build and maintain the modern data platform on Azure. As an Azure Data Engineer, you will be expected to contribute to all stages of the data lifecycle, including data ingestion, modeling, profiling, quality, transformation, movement, and curation.

Responsibilities:

Design, implement, and support on-premise and cloud-based data infrastructure that is resilient to disruptions and failures.

Design and implement data acquisition, curation and consumption pipelines using Azure technologies including Azure Synapse, Azure ADLS, and Azure SQL.

Migrate on-premise data stores to the Azure cloud.

Optimize and expand data pipelines for performance and scalability.

Monitor and troubleshoot data pipelines within Azure Synapse.

Participate in designing and building Azure data platforms supporting both batch and real-time (event-based) architecture.

Collaborate with data scientists to design and implement machine learning models using Azure Synapse and Azure Machine Learning.

Participate in driving best practices around data engineering and software development processes.

Have familiarity with the technology stack available in the industry for metadata management, including Data Governance, Data Quality, MDM, Lineage, and Data Catalog.

Required Qualifications:

Bachelor’s degree in Computer Science or a related discipline preferred, or equivalent education/professional work experience.

Technical training and certifications related to the role are highly preferred.

Proven experience as a Data Engineer, with a strong background in data infrastructure design and implementation.

Hands on experience with implementing analytics solutions using Azure cloud technologies, including Azure ADLS Gen 2, Azure Data Factory, Azure Synapse Analytics, and Azure SQL Database.

Experience designing, building, and maintaining ETL/ELT data pipelines.

Experience with leveraging, designing, and building database schemas on SQL Server dbms.

Experience with designing data lakes and data warehouses using cloud technologies.

Experience working with large datasets, data lakes, and big data technologies.

Experience using Git and Azure DevOps pipelines for deployment automation of data solutions.

Experience working in an agile development environment.

Excellent problem-solving skills and attention to detail.

Ability to work independently and as part of a team.

Demonstrates strong communication and interpersonal skills. Demonstrates the ability, desire, and focus to meet deadlines.

Demonstrates organizational skills while working on multiple projects, and communicates effectively within the team.

Desired Qualifications:

Minimum of 4 year’s hands on experience

Familiarity with metadata management tools and practices.

Understanding of data and query optimization, and query performance monitoring tools and techniques.

Experience with Python and Spark, Azure Logic Apps, Machine Learning, and Data Bricks.

Experience with cloud infrastructure services such as Azure AD, Azure Kubernetes, Azure Networking, Azure Virtual Machines, and Azure Security.