Data Engineer-W2

Job Description

Overview:

The Data Engineer plays a crucial role in our organization, responsible for designing, developing, and maintaining scalable data pipelines and infrastructure. This role will have a significant impact on our ability to efficiently and effectively process and analyze large volumes of data, enabling data-driven decision-making across the organization.Key Responsibilities:

Design, build, and maintain an efficient and reliable ETL pipeline for large-scale data processing.

Develop and optimize data models for use in data science and analytics applications.

Implement data quality and validation processes to ensure accuracy and reliability of data.

Collaborate with data scientists and analysts to understand data requirements and implement solutions.

Manage and optimize data storage and retrieval systems.

Create and maintain documentation for data infrastructure and processes.

Monitor and troubleshoot performance issues with data pipelines.

Implement security and privacy measures to protect sensitive data.

Explore and evaluate new data technologies and tools to drive innovation.

Participate in cross-functional teams to support data-related initiatives.Required Qualifications:

Bachelor's or Master's degree in Computer Science, Engineering, or related field.

Proven experience in big data technologies and platforms.

Proficiency in programming languages like Python, Java, or Scala.

Strong SQL and database management skills.

Experience with ETL tools and processes.

Expertise in data modeling and data warehousing concepts.

Familiarity with cloud-based data solutions such as AWS or Azure.

Ability to work with distributed computing and parallel processing.

Understanding of data governance and compliance requirements.

Excellent problem-solving and analytical abilities.

Strong communication and collaboration skills.

Proven ability to manage multiple projects and priorities effectively.

Experience with version control systems like Git.

Knowledge of Agile and DevOps methodologies.

Ability to thrive in a fast-paced, dynamic environment.