Spark/Big Data Developer

The data engineer role is a technical person who is involved with architecting, building, testing, and maintaining the data platform.

Data engineers will implement infrastructure for data processing, analysis, reporting, integrations, and machine learning model deployment.

RESPONSIBILITIES:

Technical expertise with distributed Spark or other distributed data processing technologies.

Work with large, complex data sets and high throughput data pipelines that meet business requirements.

Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources.

Build data and analytics tools that utilize the data pipeline to provide actionable insights to operational efficiency and other key business performance metrics.

Work with internal and external stakeholders to assist with data-related technical issues and support data infrastructure needs.

Collaborate with data scientists and architects on several projects.

Solve various complex problems.

Designing processes to integrate data from multiple sources to facilitate client centric advanced analytics.

Developing efficient, scalable and repeatable processes to transform data into insight on regular basis.

QUALIFICATIONS:

Degree in an analytical field such as Data Science, Machine Learning, Analytics, Statistics, Computer Science, or highly quantitative engineering.

Previous experience as a data engineer or in a similar role.

5+ years of Python development experience is necessary.

Hands-on experience with database technologies (e.g. SQL and NoSQL)

Experience building high throughput data pipelines.

Technical expertise with distributed Spark or other distributed data processing technologies.

Experience with machine learning techniques.

Great numerical and analytical skills.

Ability to write reusable code components.

Open-minded to the new technologies, frameworks.

Thorough business analysis skills.

Understanding Blockchain system mechanism.

Knowledge of data preparation techniques to aid statistical analysis.

Results-oriented self-starter who is confident in defending his/her critical thinking abilities.

ABOUT DTCC: With 50 years of experience, DTCC is the premier post-trade market infrastructure for the global financial services industry. From 20 locations around the world, DTCC, through its subsidiaries, automates, centralizes, and standardizes the processing of financial transactions, mitigating risk, increasing transparency, enhancing performance, and driving efficiency for thousands of broker/dealers, custodian banks and asset managers. Industry owned and governed, the firm innovates purposefully, simplifying the complexities of clearing, settlement, asset servicing, transaction processing, trade reporting and data services across asset classes and bringing increased security, enhanced resilience, and soundness to financial markets. In 2022, DTCC’s subsidiaries processed securities transactions valued at U.S. $2.5 quadrillion and its depository subsidiary provided custody and asset servicing for securities issues from over 150 countries and territories valued at U.S. $72 trillion. DTCC’s Global Trade Repository service, through locally registered, licensed, or approved trade repositories, processes more than 17.5 billion messages annually. Skills : Spark, Big Data, BigData, Data Science, Machine Learning, ML, Analytics, Statistics, Python, SQL, NoSQL, Data Pipelines, Distributed Data Processing, Blockchain, Block Chain, Large, Complex Data Sets, Optimal Extraction, Transformation, Loading Of Data.

Direct hire