Data Engineer

Purpose of the Role

The Data Engineer role will sit within the new Data Engineering team, and report to the Head of Research. The Data Engineering team will provide infrastructure, tools and support to analysts in Trading Analytics, and quants in the Trading Research team.The role will be shaped depending on experience - senior candidates will be able to take a leading role in designing and shaping the data infrastructure.

Responsibilities and Duties

Your responsibilities and duties will include, but will not be limited to:

● Building, maintaining and improving Gambit’s data infrastructure

● Understanding the fast-evolving data engineering landscape, suggesting and supporting the adoption of appropriate new tools

● Building services for real time data preparation, conversion, and storage, using Python,C++, Bash, and Apache Parquet

● Implementing data transformation pipelines using Python, Apache Spark, Postgres, and Apache Airflow

● Extending the data platform by integrating AWS (Amazon S3, EMR) into the workflow

● Supporting Gambit’s notebook infrastructure (Jupyter, Apache Zeppelin) used by traders and quants for data analysis

● Supporting trading data analysis applications that run on the data platform (in-house tools which help with data analysis)

● Collaborating with our Data Scientists to provide data support for machine learning models

● Collaborating with the Sysadmin team to ensure the data infrastructure is properly resourced

Skills and Competencies

You’ll have:

● Solid experience of working in Data Engineering with technologies like Spark, Parquet,Airflow, Kafka

● Built a stable and scalable data pipeline processing tons of data every day

● Familiarity with cloud services like AWS

● A proven ability to learn and willingness to work with unfamiliar new technologies

● A love of data. You’ll enjoy digging into stuff, and working on a problem until it’s solved

● Familiarity with Spark, Python, SQL and Git

Any of these other skills would be good to have:

● In-depth knowledge of PostgreSQL

● Knowledge of one or more of the other programming languages in use at Gambit: Erlang, C++, and JavaScript (React)

What you get in return

● Help to build the next-generation data pipeline

● A direct connection to the success of Gambit, receiving in return a generous bonus linked to your salary

● Health insurance, and life insurance, on passing your probation

● Cycle to work scheme

● Travel card loan

● Flexible hours and no dress code

● An annual training budget, to help with your personal development

● The chance to choose your own equipment and be set up for success.

● As much coffee, drinks, snacks, and fruits as you can handle, plus a chef prepared office lunch twice a week