Databricks Data Engineer

Job Description

Databricks Data EngineerWho we are

Horizon Industries Limited (Horizon) is a dynamic IT and Management Consulting firm based in the Washington, DC area. Founded in 1996, Horizon Industries Limited (Horizon) has evolved into a company of highly qualified professionals that provides full-cycle IT Consultant and Management support in both the private and public sectors.

Horizon's culture is based on our core values, and we PRIDE ourselves on a diverse, employee, and family-centric approach.

Who we are looking for

We are seeking a Databricks Data Engineer for a full-time position supporting our various clients

We are committed to your growth and success in the IT software development industry. We are looking for a Databricks Data Engineer ready to assume leadership responsibilities in a supportive team environment focused on your success as a function of client success. What your day-to-day will look like / Responsibilities:

Build end-to-end implementation of multiple ETL/ELT pipelines, demonstrating efficient data transformation and ingest patterns to move raw data from data producers to an enterprise data ecosystem, with a focus on performance and reliability

Assess and understand the ETL jobs, workflows, BI tools, and reports

Address technical inquiries concerning customization, integration, enterprise architecture and general feature / functionality of data products

Experience in crafting database / data warehouse solutions in the cloud (Preferably AWS, Azure, Alternatively GCP)

Key must have skill sets – Python, SQL, Databricks, AWS Data Services

Experience with message queuing, stream processing, and highly scalable 'big data' data stores

Experience manipulating, processing, and extracting value from large, disconnected datasets

Experience manipulating structured and unstructured data for analysis

Experience with data modeling tools and processes

Experience aggregating and transforming data from multiple datasets to create data products

Support an Agile software development lifecycleWhat we would like to see on your resume/ Required Skills

Ability to hold a position of public trust with the US government.

S. in Computer Science or equivalent

4+ years' experience in data engineering and big data. At least 2 years' of professional services experience interacting directly with clients.

Big data tools: Hadoop, Spark, Kafka, etc.

Relational SQL and NoSQL databases and experience working with relational databases

AWS cloud services: EC2, S3, RDS, Glue, Step Functions, Lamda, EMR, DynamoDB, DocumentDB, Redshift, Aurora, Athena

Data Platforms: Databricks

Data streaming systems: Batch,Kafka, Storm, Spark-Streaming, etc.

Languages: Python, R, Scala, Go

Ability to inspect existing data pipelines, discern their purpose and functionality, and re-implement them efficiently in Databricks.

Extensive knowledge of data warehousing concepts and hands-on experience deploying pipelines using Databricks **a must

Data modeling and database design skills and knowledge of version control

Excellent verbal and written communication skills

Experience architecting scalable and fault-tolerant data solutions across Azure, AWS, and Databricks

Databricks Data Engineer Professional certification a plus.

Preference for candidates with Databricks Professional certificationsWhat specifically does this role require

Education: Bachelor's degree in a technical field such as software engineering, engineering, computer science, etc.

Location : Horizon is headquartered in the heart of Tyson's Corner, VA. Occasional travel may be required.

Why you will love working with us/ Perks

A comprehensive benefits package including healthcare (medical, dental, vision and disability)

a 401k program where you are 100% vested from day one with an employer match after 90 days.

an Educational Assistance program.

a Student Loan Repayment Program

Gym Reimbursement Program.

Paid Time off

Dynamics, passionate, multi-disciplinary team of creative minds to work with, and many more.

Horizon is an Equal Employment Opportunity employer, and it is our policy to consider all applicants for employment. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, or national origin.

EOE/Vet/Disabled

Full-time