Senior Data Engineer

** Sr. Data Engineer **

** Direct Hire – no C2C ** eligible to work without sponsorship

** onsite in West Houston **

We have the following Senior Data Engineer role that plays a crucial role in architecting, implementing, and managing robust, scalable data infrastructures.

This position demands a blend of systems engineering, data integration, and data analytics skills to enhance a company’s data capabilities, supporting advanced analytics, machine learning projects, and real-time data processing needs.

ESSENTIAL FUNCTIONS

Design and implement scalable and reliable data pipelines to ingest, process, and store diverse data at scale, using technologies such as Apache Spark, Hadoop, and Kafka.

Work within cloud environments like AWS or Azure to leverage services including but not limited to EC2, RDS, S3, Lambda, and Azure Data Lake for efficient data handling and processing.

Develop and optimize data models and storage solutions (SQL, NoSQL, Data Lakes) to support operational and analytical applications, ensuring data quality and accessibility.

Utilize ETL tools and frameworks (e.g., Apache Airflow, Talend) to automate data workflows, ensuring efficient data integration and timely availability of data for analytics.

Collaborate closely with data scientists, providing the data infrastructure and tools needed for complex analytical models, leveraging Python or R for data processing scripts.

Ensure compliance with data governance and security policies, implementing best practices in data encryption, masking, and access controls within a cloud environment. QUALIFICATIONS

Bachelor's degree in computer science, MIS, or other business discipline or equivalent combination of education and/or experience.

8+ years of experience in data engineering, with a proven track record in designing and operating large-scale data pipelines and architectures.

Strong experience in AWS and Azure cloud services, with hands-on experience in integrating cloud storage and compute services with Databricks.

Expertise in developing ETL/ELT workflows.

Comprehensive knowledge of platforms and services like Databricks, Dataiku, and AWS native

Hands on RDBMS experience (data modeling, analysis, programming, stored procedures)