Data Engineer

Job Description

Role: Data Engineer

About Rigil:

Rigil is an award-winning, woman-owned, small business that specializes in technology consulting, strategy consulting and product development. We value teamwork and strive to build strong leaders.

Location: San Francisco, CA

Job Type: Full Time

Data Engineer is integral to evolving Vision and its underlying data and AI platform, which forms the backbone of our business. Vision is much more than a tool; it's the operational core and a vital integration layer that connects our installed hardware, connectivity network, and customers. This platform streamlines the marketing, delivery, installation, and maintenance of our energy products globally.

Your work will be pivotal in enabling our platform to scale, be intelligent and provide secure and easy access to data that customers can trust. By enhancing Vision, you will empower our international partners, simplifying and improving their ability to connect with customers and support our energy appliances. Your efforts will ensure that our data and analytics platform is not only agile and responsive but also anticipates and adapts to the evolving needs of diverse markets and stakeholders.

In this foundational phase of our platform, your role is central to adopting an AI-centric strategy. You will be at the forefront of building the next stage of Vision, focusing on new capabilities that embed conversational interfaces that dynamically learn and adapt to customer needs. These interfaces will not only enhance user experiences but also glean insightful data about our customers and business operations.

This position demands a unique combination of technical skill, pragmatic orientation, data- centric thinking, and an intuitive sense for how Large Language Models are poised to change how we approach software, both how it works, and how we develop it.

Responsibilities

Create and maintain optimal data pipeline architecture.

Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, NoSQL and AWS big data technologies.

Build analytics (AI & ML) tools that utilize the data pipeline to provide actionable insights into customer acquisition, device operations and other key business performance metrics.

Keep our data separated and secure across tenants and geographical boundaries through data segmentation and infrastructure.

Manage databases effectively using PostgreSQL, ensuring data integrity and performance.

Utilize AWS for continuous deployment and cloud services management.

Work within Agile development frameworks, contributing to all phases of the software development lifecycle.

Innovate and contribute to a technology roadmap that aligns with our mission of providing sustainable energy solutions.Qualifications

Bachelors degree in computer science or related field; Advanced degree preferred.

At least 5 years of data engineering experience.

Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

Experience building and optimizing big data data pipelines, architectures, and data sets.

Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.

Experience with AWS cloud services: EC2, EMR, RDS, Redshift.

Working knowledge of message queuing, stream processing, and highly scalable big data data stores.

Working knowledge of MLOps.

Familiarity with IoT platforms and integrating AI tools in data applications.