Azure Databricks

Responsibilities

Design, develop, and deploy data processing solutions using Azure Databricks.

Collaborate with data engineers, architects, and business analysts to gather and analyze requirements.

Develop, optimize, and manage complex data pipelines and workflows.

Ensure data quality, integrity, and security across various data sources.

Troubleshoot and resolve performance issues related to Databricks solutions.

Document technical specifications and maintain comprehensive project documentation.

Provide technical guidance and mentorship to junior developers.

Qualifications

Bachelor's degree in Computer Science, Information Technology, or related field.

Minimum of 5 years of experience in data engineering or similar roles.

Proven expertise with Azure Databricks and data processing frameworks.

Strong understanding of data warehousing, ETL processes, and data pipeline design.

Experience with SQL, Python, and Spark.

Excellent problem-solving and analytical skills.

Effective communication and teamwork abilities.