Big Data Developer (Spark/Scala Python)

Job Summary

Synechron is seeking an experienced Big Data Developer with strong expertise in Spark, Scala, and Python to lead and contribute to large-scale data projects. The role involves designing, developing, and implementing robust data solutions that leverage emerging technologies to enhance business insights and operational efficiency. The successful candidate will play a key role in driving innovation, mentoring team members, and ensuring the delivery of high-quality data products aligned with organizational objectives.

Software Requirements

Required:

Apache Spark (latest stable version)

Scala (version 2.12 or higher)

Python (version 3.6 or higher)

Big Data tools and frameworks supporting Spark and Scala

Preferred:

Cloud platforms such as AWS, Azure, or GCP for data deployment

Data processing or orchestration tools like Kafka, Hadoop, or Airflow

Data visualization tools for data insights

Overall Responsibilities

Lead the development and implementation of data pipelines and solutions using Spark, Scala, and Python

Collaborate with business and technology teams to understand data requirements and translate them into scalable solutions

Mentor and guide junior team members on best practices in big data development

Evaluate and recommend new technologies and tools to improve data processing and quality

Stay informed about industry trends and emerging technologies relevant to big data and analytics

Ensure timely delivery of data projects with high standards of quality, performance, and security

Lead technical reviews, code reviews, and provide inputs to improve overall development standards and practices

Contribute to architecture design discussions and assist in establishing data governance standards

Technical Skills (By Category)

Programming Languages:

Essential: Spark (Scala), Python

Preferred: Knowledge of Java or other JVM languages

Data Management & Databases:

Experience with distributed data storage solutions (HDFS, S3, etc.)

Familiarity with NoSQL databases (e.g., Cassandra, HBase) and relational databases for data integration

Cloud Technologies:

Preferred: Cloud platforms (AWS, Azure, GCP) for data processing, storage, and deployment

Frameworks & Libraries:

Spark MLlib, Spark SQL, Spark Streaming

Data processing libraries in Python (pandas, PySpark)

Development Tools & Methodologies:

Version control (Git, Bitbucket)

Agile methodologies (Scrum, Kanban)

Data pipeline orchestration tools (Apache Airflow, NiFi)

Security & Compliance:

Understanding of data security best practices and data privacy regulations

Experience Requirements

5 to 10 years of hands-on experience in big data development and architecture

Proven experience in designing and developing large-scale data pipelines using Spark, Scala, and Python

Demonstrated ability to lead technical projects and mentor team members

Experience working with cross-functional teams including data analysts, data scientists, and business stakeholders

Track record of delivering scalable, efficient, and secure data solutions in complex environments

Day-to-Day Activities

Develop, test, and optimize scalable data pipelines using Spark, Scala, and Python

Collaborate with data engineers, analysts, and stakeholders to gather requirements and translate into technical solutions

Lead code reviews, mentor junior team members, and enforce coding standards

Participate in architecture design and recommend best practices in big data development

Monitor data workflows performance and troubleshoot issues to ensure data quality and reliability

Stay updated with industry trends and evaluate new tools and frameworks for potential implementation

Document technical designs, data flows, and implementation procedures

Contribute to continuous improvement initiatives to optimize data processing workflows

Qualifications

Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field

Relevant certifications in cloud platforms, big data, or programming languages are advantageous

Continuous learning on innovative data technologies and frameworks

Professional Competencies

Strong analytical and problem-solving skills with a focus on scalable data solutions

Leadership qualities with the ability to guide and mentor team members

Excellent communication skills to articulate technical concepts to diverse audiences

Ability to work collaboratively in cross-functional teams and fast-paced environments

Adaptability to evolving technologies and industry trends

Strong organizational skills for managing multiple projects and priorities

S YNECHRON’S DIVERSITY & INCLUSION STATEMENT

Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.

All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.

Candidate Application Notice

JR1027306