Data Engineer

Key Responsibilities

Design, build, and sustain efficient, scalable and performant Data Engineering Pipelines to ingest, sanitize, transform (ETL/ELT), and deliver high-volume, high-velocity data from diverse sources.

Ensure reliable and consistent processing of versatile workloads of granularity such as Real Time, Near Real Time, Mini-batch, Batch and On-demand.

Translate business requirements into technical specifications, design and implement solutions with clear documentation.

Develop, optimize, and support production data workflows to ensure comprehensive and accurate datasets.

With Software Engineering mindset/discipline, adopt best practices in writing modular code that is maintainable, scalable and performant.

Use orchestration and scheduling tools (e.g., Airflow, GitLab Runners) to streamline workflows.

Automate deployment and monitoring of data workflows using CI/CD best practices. Use DevOps best practices to instrument retries and self-healing to minimize, if not avoid manual intervention.

Use AI/Gen AI tools and technologies to build/generate reusable code.

Collaborate with Architects, Data scientists, BI engineers, Analysts, Product/ Program Mgmt and other stakeholders to deliver end-to-end solutions.

Promote strategies to improve our data modelling, quality and architecture

Mentor junior engineers, and contribute to team knowledge sharing.

Have an analytics mindset to explore and identify opportunities for improved metrics, insights contributing to business growth.