Data Engineer

Job Description

The Enterprise Data Engineer position on the Enterprise Data Solutions team is part of our Corporate Systems, Enterprise Architecture and Information Management department within Enterprise Technology. The role is responsible for establishing, expanding and optimizing our Enterprise Analytical Data Platform and associated data pipeline architecture, as well as optimizing data flow for cross-functional analytical teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys building and optimizing data systems for maximum business benefit. The Data Engineer will work with our software developers, database architects, data analysts and data scientists on initiatives and will ensure the data delivery architecture is efficient and consistent. The Enterprise Data Engineer works in a self-directed fashion, concurrently supporting the data needs of multiple teams, systems, and products. This role understands complex source system data structures and implements models, patterns, and solutions in both ‘big data’ and traditional data warehouse platforms.

Principal Duties & Responsibilities: Consultative solution development in collaboration with business and IT stakeholders Assemble large, complex data sets that meet functional / non-functional business requirements Develop and maintain an optimized Enterprise Data Pipeline architecture Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics Develop, maintain and support Data Solutions using both Spark and SQL Design, configures/code, test, debug and document software/solution according to Industry best practices Work with Enterprise Architects and Data Governance to help in data ingestion, staging, data quality monitoring Preparation of design/technical/functional/operational specification documentation Work with stakeholders and design teams to assist with data-related technical issues and support their data platform needs Develop platform strategies and patterns for Data Integration, Delivery, Partitioning and Access Control Identify, design, and implement internal process improvements, such as: Automating manual processes Optimizing data delivery Re-designing or aligning infrastructure for greater scalability Mentor and provide guidance on data solutions and patterns5+ Years in IT - Software development and Engineering 5+ Years in Big Data and Analytics (Spark, Azure Analytics) 5 Years’ experience with advanced Structured Query Language (SQL) and experience working with relational databases, query authoring as well as working familiarity with a variety of databases, primarily Databricks but also including Microsoft SQL Server, IBM DB2 and Oracle 5 Years’ experience working with online cloud providers (e.g., Azure, AWS, Google) 3 Years working with unstructured datasets 3 years’ experience using programming languages Python or Scala Strong project delivery and organizational skills Experience supporting and working with cross-functional teams in a dynamic environment

Full-time

Fully remote