Data Engineer

Primary Job Functions

Responsible for full development cycle of Data Engineering & Warehousing concepts, including requirements gathering, architecture, design, implementation, and maintenance.

Identify and resolve data issues to ensure the quality and consistency of data; determine and implement opportunities for automation.

Continuously develops expertise in the data and data models within the data warehouse and an understanding of supported business domains.

Communicate effectively, both orally and written, to varied levels of the organization to include technical personnel, business managers, and senior leadership, regarding data inquiries and request.

Conducts and may facilitate internal testing & user acceptance testing.

Develop technical specifications and design documents.

Ensure that quality processes are carried out through all phases of the Development Lifecycle.

Design and implement efficient and effective functional design solutions.

Work closely with the Business and Business Intelligence team to ensure requirements are met.

Build robust data pipelines and integrate with multiple components and data sources.

Build & maintain highly scalable and secure data environments.

Design and architect new product features, champion the use of cutting-edge technologies and mentor the team in the adoption of these new technologies.

Collaborate with the Data Science team on complex machine learning models to ensure that data processing, structure and accessibility maximizes model performance.

Applies domain technical expertise when providing solutions to the business and its operations.

Required Qualifications

Bachelor's Degree in computer science, Information Technology, Management Information Systems, or related field, Required. Technical MS Preferred.

5 Years of professional experience, in data engineering/data warehousing enterprise or related experience, Required.

Advanced skills with data warehouse architectures, dimensional models, star schema designs and/or in-memory/columnar databases

SQL & T-SQL skills

Proficient in programming languages such as Spark, Spark SQL, R, Python, Java, and Scala

Knowledge of Delta Lake

Proficient in: SQL Server syntax, Microsoft/Azure services like Data Factory, & Azure Analysis Services (Tabular). Familiarity with associated syntaxes (DAX, MDX)

Proficient with Microsoft Azure Cloud services in design and management, monitoring, security, and privacy of data using the full stack of Azure data services

Strong experience with Database Management Systems (DBMS), specifically SQL and NoSQL.

Demonstrated success working within Azure Synapse Analytics and Azure Blob Storage

Working experience with API / RESTful data services

Proficient with big data technologies such as Spark, Databricks, Hadoop, Hive, etc.

Knowledge of popular data discovery, analytics and BI (Business Intelligence) software tools like Power BI for semantic-layer-based data discovery

Knowledge with containerization techniques such as Docker, Kubernetes

Kindly share your profiles to