Lead Data Engineer with Java, Python, Spark & AWS - Onsite - W2 only

Title:Lead Data Engineer with Java, Python, Spark & AWS - Onsite - W2 only

Location:Columbus,OH

Length: Long term

Restriction:w2 only

Description:

Webcam interview; *** Long term project with the possible contract to hire ***Linkedin Must*** Onsite

Job Description

Seeking an experienced Lead Data Engineer with strong expertise in Java, Python, AWS, ETL/ELT pipelines, and Apache Spark.

Key Responsibilities

Lead architecture and development of large-scale ETL/ELT data pipelines.

Design data ingestion, transformation, validation, and reconciliation workflows.

Develop distributed data processing solutions using Apache Spark.

Design and implement data lake and lakehouse architectures.

Optimize Spark workloads, including joins, caching, partitioning, and performance tuning.

Develop Java-based services and REST APIs.

Implement automated testing, CI/CD, monitoring, and production support.

Drive engineering best practices and mentor development teams.

Promote enterprise-approved AI-assisted engineering practices.

Required Skills

10+ years of software engineering experience.

Strong hands-on Java development.

Python proficiency.

ETL/ELT pipeline development.

Apache Spark and distributed processing.

AWS cloud experience.

Parquet, Avro, and data lake/lakehouse architecture.

Snowflake or equivalent cloud data warehouse experience.

AI-assisted development tools and responsible AI practices.

Preferred Skills

Airflow, Dagster, or Control-M.

Kafka and Spark Structured Streaming.

Delta Lake, Apache Iceberg, or Hudi.

Data governance and data quality.

Financial services or regulated industry experience.

Agentic AI and Generative AI exposure.