Developer Programmer
Description:
Job duties:
• Build robust, efficient, and reliable data pipelines consisting of diverse data sources
to ingest and process data into cloud-based data platforms using Python, Scala,
bash or Java as programming language
• Design and develop batch and streaming data ingestion pipelines from various
source systems to cloud-based data and analytics platform
• Transform data using data mapping and distributed data processing capabilities like
Databricks, Apache beam, Apache Spark etc.
• Work with stakeholders including the Product Owner and data analyst teams to
assist with data-related technical issues and support their data infrastructure needs
• Share knowledge with immediate peers and build communities and connections
that promote better data engineering practices across the organisation
• Lead/mentor junior data engineers in various methodologies, standards, and best
practices
• Collaborate with Architects to validate the architecture and technology selection
Requirements
• Proven working experience as data engineer for 4+ years preferably in building data
lake solution by ingesting and processing data from various source systems
• Minimum two years of experience with cloud-based data components (Azure
Databricks, AWS or GCP) from one of the leading public cloud-based providers
and/or Snowflake
• Understanding of SDLC processes, experience of working in a large-scale program,
data warehousing and ETL development
• Experience in DevOps to build automated pipelines for the deployment of new
features on enterprise data platforms
• Experience in “peer review” for Source control
• Understanding of Data Engineering, Data design patterns and Data Management
practices in the cloud
• Experience in building various Data Ingestion, Data Processing and Data Quality
frameworks for an enterprise data lake is highly desirable
• Certification from one of the leading public cloud-based providers
• Data Engineering certification is desirable but not imperative
• Bachelors degree in IT or equivalent