Senior Lead Data Engineer

Key responsibilities and Accountabilities

* Contribute using our standard tech stack, which is Python, Linux (via WSL), PostgreSQL, SQL Server, Databricks, and spans both Azure and AWS cloud infrastructure.

* Participate in the design and implementation of advanced mechanisms for data ingestion and transformation, and mass parallel orchestration of network IO.

* Maintaining Infrastructure as Code and working with infrastructure teams to develop valuable new features for Client's Engineering Platform.

* Model data for a variety of purposes (operational and analytical) in collaboration with your team, other teams, and analytical data customers.

* Contribute to CLIENT's data platform in a way that promotes the development of a data mesh architecture for the entire enterprise.

* Work on indicators of data quality across our estate and improve the quality of our data along key metrics.

* Evaluate the quality (completeness and accuracy) of potential new data sources for specific applications.

* Contribute to quality automation in our CI/CD process, optimising for bug-free release cadence and developer delight.

* Maintain good quality documentation (starting with your code) for technical and non-technical audiences.

* Participate in a pairing-first development team, learning skills from your colleagues and giving back to the community by imparting hard-won wisdom and skills learned from past experiences.

* Mentoring and coaching of less experienced team members through design and code reviews

* Participate in a safe and honest workplace culture by participating in open conversations about technical design, implementation and interrogating decisions when appropriate.

* Contribution to test automation through either enhancement to the platform or additional tests

* Assessment of application security to ensure it complies with CLIENT Application Security Standards

* Evaluation of APIs to ensure that they comply with the relevant CLIENT API Standards

Skills & Ability

* You are able to grasp, articulate, and advocate for technological solutions in both technical and non-technical settings and intelligently apply agile principles as you do so.

* You are able to interrogate and present data and data analysis and have a willingness to present results of analysis undertaken as a part of your team's work.

* You have demonstrable experience as a senior software developer or engineer with a focus on Python.

* SQL (Postgres, SQL Server, Databricks)

* comfortable working without an ORM.

* familiar with the challenges of working with data *at scale*.

*prefer the command line use of git, bash/zsh, and (optionally) vim.

* An undergraduate degree or demonstrable knowledge and experience of concepts in Computer Science, Software Engineering, Logic, Applied Statistics, Actuarial Science, Data Science, or any related discipline would be advantageous.

Schedule: Full-time

Travel: No