Hadoop Developer

Job Title: Big Data Engineer / Developer

Location: Austin, TX (On-site)

Employment Type: Full-Time

About the Role:

We are looking for a highly skilled Big Data Engineer / Developer to join our data engineering team in Austin, Texas. In this role, you’ll design and develop scalable data processing pipelines using Apache Spark, Scala, and Python to support real-time and batch data workflows. You’ll work closely with data scientists, analysts, and backend engineers to deliver high-quality, high-performance data infrastructure.

Key Responsibilities:

Design, develop, and optimize large-scale distributed data processing pipelines.

Build real-time and batch data solutions using Apache Spark.

Write clean, efficient, and reusable code in Scala and Python.

Collaborate with cross-functional teams to understand data needs and deliver solutions.

Implement data quality, monitoring, and governance solutions.

Perform root cause analysis on internal and external data and processes to answer specific business questions.

Document systems, workflows, and processes.

Required Qualifications:

Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

8+ years of experience in Big Data engineering or backend development roles.

Strong hands-on experience with Apache Spark (batch and streaming).

Proficiency in Scala and Python for data engineering.

Solid understanding of distributed systems and data processing frameworks.

Experience working with data lakes, data warehousing, and large-scale ETL pipelines.

Proficient with Linux/Unix systems and shell scripting.

Familiarity with Git, CI/CD tools, and Agile development practices.

Preferred Qualifications:

Experience with Hadoop ecosystem (HDFS, Hive, etc.)

Familiarity with Kafka, Airflow, or other data orchestration tools.

Knowledge of containerization tools like Docker and orchestration via Kubernetes is a plus.

Experience with cloud platforms other than Azure (e.g., AWS, GCP) is acceptable.