Data Engineer-Washington, D.C.
We have partnered with a large consulting firm in the Washington, DC area to provide them with a Data Engineer.
Please review the below description and let us know if you are interested.
Responsibilities of the Data Engineer:
Designing and developing ongoing data ingestion and cleaning using Python and Spark.
Support Requirements gathering activities, especially from the data migration perspective.
Using programming skills to create big data pipelines that can deal with both structured and unstructured data.
Support other teams in effective use of big data tools and provide guidance in writing efficient code in Python and/or R.
Analyzes and loads data for use by a larger audience.
Analyzes databases to improve speed of data access.
Writes views for consultants to simplify data access.
Diagnoses and corrects database performance bottlenecks.
Analyzes data provided by project consultants and makes it available in our database; provides access to the database to those that have a need.
Develops strategies for data acquisitions, archive, recovery, and implementation of a database.
Maintains the database and cleans data as required.
Designs and develops databases, data warehouses, and multidimensional databases.
Leads and directs data management work of others as applicable.
Reports to office on a regular basis to allow for in-person interactions including providing oversight and mentorship to team, attending meetings with other employees, candidates, and vendors, participating in performance conversations, attending firm meetings, or as otherwise requested by direct supervisor.
Requirements of the Data Engineer:
Bachelor's degree required.
5+ years of experience working with big data.
3+ years of experience with software development lifecycle.
3+ years of developing data ingestion and transformation processes associated with big data Technologies (e.g. Hadoop, Spark, etc.)
5+ years of hands-on Python coding experience.
Experience with data modeling, integration, and warehousing.
Hands-on experience with CI/CD automation is preferred.
Working knowledge with UNIX variants preferred.
Previous experience working with Cloud (e.g. AWS) is a plus.
Experience with Databricks is a plus
Experience with performance tuning of ETL jobs.
Ability to communicate technical issues to non-technical staff.
Ability to perform essential functions with or without a reasonable accommodation.
May require more than 40.0 hours per week to perform the essential duties of the position.
Benefits of the Data Engineer:
Profit Sharing Program