Data Engineer - Big Data Warehouse

Job Description

Job Summary

We are seeking a hands-on Data Engineer to design, code, and deliver Big Data Warehouse solutions. The ideal candidate is passionate about technology, thrives under pressure, and excels in collaborative environments. This role involves working closely with product owners, technical stakeholders, and cross-functional teams to deliver high-impact data solutions.

Key Responsibilities

Design and develop scalable Big Data Warehouse solutions across the data supply chain

Implement metadata management solutions

Create and maintain technical and user documentation (data models, dictionaries, glossaries, process/data flows, architecture diagrams)

Extend and enhance the enterprise Data Lake

Solve complex data integration challenges across multiple systems

Design and implement real-time data analysis and decisioning strategies

Collaborate with stakeholders to support data quality initiatives

Partner with Data Science teams to enhance actionable insights

Continuously learn and adopt new technologies

Required Skills

Databricks

PySpark

Spark

Spark Cluster Configuration

Experience & Qualifications

Strong background in data management and access (Big Data, Data Marts, Data Warehousing)

Proficient in SQL, Spark SQL, and DataFrames

Familiarity with Redshift, Spark, Hadoop, and web services for data-driven decisioning

Experience in data architecture, governance, and security

Hands-on experience with data integration tools (e.g., Talend preferred)

Skilled in scripting for data manipulation

Exposure to Business Intelligence, MDM, XML, SOA/WebServices

Familiarity with Data Science tools and technologies

Bachelor’s or Master’s degree in Computer Science, Data Processing, or related field

Proven experience in Data Warehousing or similar analytics environments

Java programming and framework development experience

Experience with Hadoop, Spark, and AWS (EMR, EC2, Aurora, Athena, Redshift, S3)

2+ years of Python experience

Proficient with Bitbucket and Git

Comfortable working in Linux environments

Familiarity with Jenkins and CI/CD pipelines

Strong understanding of core computer science concepts

Experience with Postgres and MySQL

Excellent organizational and project management skills

Outstanding communication skills