Data Engineer/ Python Developer

Job Description

3155892

Contract to Hire

Top Three Skills:

1. Data Engineer who is comfortable and capable of developing with python as well as SQL

2. Experience working with millions of rows of data this application pushed more then 19 million rows of data in a given day as well as 30,000 - 40,000 views a week

3. Data Engineer who is comfortable working in large databases both which have structured and unstructured data

Job Description: For this role as a Fullstack/Data engineer you will be part of a cross-functional Reporting & Analytics web application team ensuring successful, high quality product deployments. We are looking for an energetic, highly motivated, and detail-oriented data engineer to fill this role. This application is in the middle of a modernization, and we are looking for a well-rounded data engineer who "gets it", who thrives in an agile environment, and who doesn't recognize role boundaries, always swarming to get things done.

As a member of this team, you will be:

• Responsible for expanding and optimizing our data and future data pipeline architecture, as well as optimizing data flows and collections for cross-functional teams.

• An experienced data wrangler who enjoys optimizing data systems and building them from the ground up.

• Self-directed and comfortable supporting the data needs of multiple teams, systems, and products.

• Excited by the prospect of optimizing or even re-designing our data architecture to support our next generation of data initiatives.

Fundamental Responsibilities:

• Ability to recommend and implement ways to improve data reliability, efficiency, and quality.

• Able to employ a variety of languages and tools to marry systems together or try to hunt down opportunities to acquire new data from disparate systems

• Ensure current (and future) architecture supports the needs and requirements of all stakeholders

• Develop data set processes for data modeling, mining, and production

• Assemble large, complex data sets that meet functional / non-functional business requirements.

• Build & maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and various other technologies.

• Create and maintain optimal data pipeline architecture,

• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

• Work with stakeholders including executive/product/data/design teams to assist with data-related technical issues and support their data infrastructure needs.

Required Qualifications

• 4+ years experience in a Data Engineering role, preferably within a web app environment

• Strong understanding of both relational data structures and writing complex data analysis queries, as well as non-relational structures and concepts related to BI

• Advanced working SQL knowledge and experience working with relational/non-relational databases, query authoring (SQL) as well as working familiarity with a variety of databases (eg, MS SQL, MySQL, Oracle, MongoDB).

• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.

• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

• Strong analytic skills related to working with unstructured datasets.

• Build processes supporting data transformation, data structures, metadata, dependency and workload management.

• A successful history of manipulating, processing and extracting value from large disconnected datasets.

• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.

• Strong project management and organizational skills.

• Experience supporting and working with cross-functional teams in a dynamic environment.

SW/Tool Experience:

• Experience with relational & non-relational databases, including MS SQL & MongoDB

• Expert in working with JavaScript and REST APIs

• Experience with data visualization tools: Domo (preferred), Tableau

• Experience with big data tools: Hadoop, Spark, Kafka, etc

• Experience with scripting languages: Python, R, SQL, Scala, etc

• Experience with tooling: Jenkins Pipeline, Git, Docker, OpenShift

• Experience with unit and functional-level automation: Jasmine/Mocha, Selenium, Spock

• Experience in agile scrum/DevOps/DataOps