Senior Data Engineer

Data Engineer

US Citizenship required; preference will be given to those with an EOD (Entry on Duty/Suitability) Clearance.

Description

We are currently seeking a Data Engineer to work within a team providing Data Warehouse and Business Intelligence services to our government customer using Agile processes. As a Data Engineer, you will work with varying huge data sources with different schemas and data elements to produce an effective and efficient Data Warehouse. You have an eye for spotting data correlations and a desire to dig into large datasets to find technical solutions and deliver business value.

Program Details

The program you will be supporting has a mission to provide development, security, and operations (DevSecOps) support to U.S. Citizenship and Immigration Services (USCIS) with a focus on development, operations, and modernization of the Agency’s Enterprise Data Warehouse/Data Lake. The team utilizes open-source, AWS Cloud, and Big Data technologies, agile project management practices, and modern DevSecOps delivery to provide the business intelligence support systems to meet the reporting, data analytics, and machine learning/artificial intelligence needs critical to USCIS leadership, data/business analysts, data scientists, and other decision-makers.

Requirements:

Interact with designated product owners, system owners, and source system business owners to understand transactional system data models and elicit requirements and logic for ETLs

Develop ETL workflows/data pipelines to ingest data using AWS Data Migration Service (DMS), Scala, Kafka, Restful APIs, and other technologies as determined by the client from multiple transactional systems to the target (including ODS, data marts, and data lake) according to documented logic and source-to-target mappings

Troubleshoot data discrepancy and missing data issues resulting from daily ETL loads

Work with operations team to deploy ETL jobs in integration and production environment and debug / troubleshoot critical production issues.

Actively participate in Agile release development activities and ceremonies, including sprint planning, sprint grooming, artifact creation, sprint testing, demonstrations and retrospectives and solution releases.

Document ETL logic, mappings, etc. in a concise and traceable manner to be used as a reference for future development and maintenance

Execute other activities related to development work, such as participate in meetings, provide briefings, presentations and other support materials that will promote the program, assist in achieving user buy-in, and explain technical concepts to non-technical audiences.

Required Skills:

5+ years of experience with ETL development ingesting data from diverse and huge data sources

5+ years of experience with programming languages such as Java, Scala, Python, R, JSON Schema

2+ years of experience producing and consuming Rest APIs.

5+ years of experience with relational databases used to support BI analytics

Experience with Databricks/Apache Spark structured streaming, and/or Kafka experience

Demonstrated experience in a Data Warehouse/Data Lake and Business Intelligence environment

Ability to write complex SQL queries and scripts

Strong teamwork, co-ordination, planning and influencing skills

Self-driven with the ability to adapt quickly, work in a challenging and fast paced environment within cross-functional teams, and to promote creative problem solving within their team

Experience with Agile development practices, including Scrum and Kanban, and management tools (e.g., Jira, Confluence)

Experience with GIT and branching strategies

Experience with engineering/DevOps tools (i.e., Jenkins)

Excellent analytical, communication and organizational skills

Experience working in AWS Cloud environment

Experience with Microsoft Office Suite including Excel, PowerPoint, and Visio

Desired Skills:

Experience with AWS Database Migration Service (DMS)

Experience with Postgres and Oracle

Experience with Scaled Agile Framework SAFe

Education: Bachelor’s degree in a technical discipline preferred – Computer Science, Mathematics, or equivalent technical degree, or the equivalent combination of education, professional training, and work experience.

Location: Reston, VA – Currently fully remote due to COVID-19.

Clearance: Must be a US Citizen and be able to obtain a government agency Suitability Clearance. USCIS Entry on Duty (EOD) preferred.