Lead Data Engineer

Job Description

Lead Data Engineer (Los Angeles, CA)

Experience: 9–12 years

Strong data engineering background

Databricks is a must, along with experience in data modeling, building pipelines, orchestration, and supporting reporting teams

Job Functions:

Collaborate with client stakeholders to gather requirements, structure solutions, and ensure high quality, timely delivery.

Experience working in the Databricks tech stack with strong proficiency in SQL, Python, and PySpark

Design and optimize data models, data marts, and Lakehouse/warehouse layers with strong focus on medallion architecture, query optimization, and performance engineering.

Build, orchestrate, and monitor scalable data pipelines on Databricks, ensuring reliable ingestion, transformation, CDC handling, and incremental load strategies.

Manage end to end pipeline operations including performance tuning, data quality monitoring, alerting, and issue resolution across production workloads.

Lead a project team of data engineers supporting multiple workstreams and provide technical leadership through code reviews, best practice guidance, reusable pattern creation, and mentorship to engineering team members.

Prepare and maintain project documentation to support project execution and delivery.

Expected work split

50% Technical – Data modeling, hands-on coding, orchestration, and pipeline monitoring.

50% Management– Client Collaboration, requirements gathering, designing technical solutions, presentations, global team management and mentoring.

Qualifications (Required):

8-12 years' experience in data engineering and analytics roles

Bachelor's or Master's degree in analytics, computer science/engineering, economics, mathematics, or related areas.

Experience building and maintaining ETL/ELT pipelines

Solid understanding of data warehousing concepts and dimensional data modeling

Familiarity with workflow orchestration tools such as Airflow or similar

Experience working with cloud data platforms or modern data infrastructure

Entrepreneurial hands-on approach to work. Demonstrated leadership ability and willingness to take initiative

Superior analytical and problem solving skills

Outstanding written and verbal communication skills

Effective time management and attention to detail

Hands on experience in using SQL, Python and Workflow Schedulers (Apache Airflow, Cron)

Experience in leading team and coordinating with internal / external stakeholders

Qualifications (Preferred):

Experience in using Cloud Platforms (AWS / GCP / Azure)

Experience in using Visualization tools (Tableau / Power BI)

Experience with Big Data Technologies (Hadoop, Hive, Hbase, Pig, Spark, etc.)

Full-time