Python PySpark Developer - Willington, DE

Python PySpark Developer

Location: Willington, DE

Job Type: Contract

No third party C2C

Key Responsibilities:

Develop, implement, and optimize large-scale data processing pipelines using Python and PySpark.

Collaborate with data scientists, data engineers, and other stakeholders to design and enhance data solutions.

Perform data analysis and develop algorithms to process and analyze large datasets efficiently.

Maintain and improve existing PySpark codebase, ensuring high performance and scalability.

Integrate data from various sources, ensuring data quality and consistency.

Write and maintain comprehensive documentation for data processing workflows and algorithms.

Troubleshoot and resolve issues related to data processing and workflow automation.

Stay updated with the latest industry trends and technologies to continuously improve our data processing capabilities. Qualifications:

Bachelor's or Master's degree in Computer Science, Information Technology, or related field.

Proven experience as a Python developer, with a strong focus on PySpark.

In-depth knowledge of Spark architecture and the Hadoop ecosystem.

Experience with big data technologies such as Hive, HDFS, and Kafka is a plus.

Strong SQL skills and experience with relational databases.

Proficient in writing efficient, scalable, and maintainable code.

Experience with cloud platforms (AWS, Azure, GCP) is highly desirable.

Excellent problem-solving skills and attention to detail.

Strong communication and teamwork abilities.