Senior Data Engineer Databricks

Job Description

ComResource is looking for a Senior Data Engineer – Databricks.

We need someone to assist in building and optimizing enterprise data pipelines while expanding scalable data architecture and supporting advanced analytics platforms.

Responsibilities:

Build and maintain scalable ETL and ELT pipelines using Databricks technologies.

Ingest structured, semi-structured, and streaming data into lakehouse architecture layers.

Develop optimized data transformations and reusable pipeline frameworks.

Implement job orchestration, monitoring, alerting, and automation processes.

Design dimensional data models supporting analytical workloads.

Apply data modeling strategies to improve performance and usability.

Implement data quality controls and governance policies.

Collaborate with cross-functional teams to support analytics and operational data needs.

Optimize data processing performance and cost efficiency in cloud environments.

Essentials:

7–10+ years of experience in data engineering.

Strong hands-on experience with Databricks, Spark, and Delta Lake.

Experience building ETL or ELT pipelines using PySpark and SQL.

Strong understanding of Medallion architecture (Bronze, Silver, Gold).

Experience with distributed data processing and lakehouse architectures.

Experience with CI/CD pipelines, Git, and job orchestration tools.

Strong understanding of data modeling for analytics and data warehousing.

Desired:

Experience with streaming technologies such as Auto Loader or Structured Streaming.

Experience with metadata management and data governance tools such as Unity Catalog.

Experience with orchestration tools such as Airflow or Databricks Workflows.

Experience optimizing cloud-based data environments for performance and cost.

Req ID: AM99591312

Full-time