Azure Databricks engineer - W2 Only

Title: Azure Databricks engineer

Location: Atlanta, GA, USA

Length: Long term

Restriction: W2 Only

Description:

Webcam interview; *** Long term project *** Initial remote due to covid then onsite

Databricks certified (or ready to do certification before joining)

Responsibilities:

Big Data Engineer with expert level experience in Azure ADLSGEN2, Databricks -Delta Lake, Azure Data factory PySpark/Scala-Spark/Hive/Hadoop CLI/MapReduce/ Storm/Kafka/Nifi/Lambda Architecture

expert with using the larger Hadoop eco system

Familiar with job scheduling challenges in Hadoop

Experienced in creating and submitting Spark jobs

Experienced with Kafka and real-time analytics

Experience in high performance tuning and scalabilit

Core Java and Python/Spark background and their related libraries and frameworks

Experienced with Spring Framework and Spring Boot

Unix/Linux expertise; comfortable with Linux operating system and Shell Scripting

PL/SQL, RDBMS background with Oracle/MySQL

Familiarity with ORMs a plus

Design, Development, Unit and Integration testing of complex data pipelines and to handle data volumes to derive insights

Ability to optimize code to be able to run efficiently with stipulated SLA

Excellent problem-solving skills, with attention to detail, focus on quality and timely delivery of assigned tasks