Data engineer - Bigdata/spark/Scala

TCS is an equal opportunity employer, and embraces diversity in race, nationality, ethnicity, gender, age, physical ability, neurodiversity, and sexual orientation, to create a workforce that reflects the societies we operate in. Our continued commitment to Culture and Diversity and is reflected in our people stories across our workforce implemented through equitable workplace policies and processes.

About TCS

TCS operates on a global scale, with a diverse talent base of more than 600,000 associates representing 153 nationalities across 55 countries. TCS has been recognized as a Global Top Employer by the Top Employers Institute - one of only eight companies worldwide to have achieved this status. Our organizational structure is domain-led and designed to offer businesses a single window into industry-specific solutions. Our agile industry units have embedded capabilities to enable rapid responses that provide a competitive edge to our customers. This, coupled with a unique Global Network Delivery Model™ (GNDM™), is recognized as the current benchmark of excellence in technology deployment. We have made significant investments in digital technology, horizontal, and vertical platforms, allowing us to successfully serve our clients for over 50 years.

Skills and Responsibilities:

• Design, develop, and maintain data processing pipelines using Apache Spark.

• Collaborate with data engineers, data scientists, and business analysts to understand data requirements and deliver solutions that meet business needs.

• Write efficient Spark code to process, transform, and analyze large datasets.

• Optimize Spark jobs for performance, scalability, and resource utilization.

• Integrate Hadoop, Hive, Spring, Hibernate, Kafka, and ETL processes into Spark applications.

• Troubleshoot and resolve issues related to data pipelines and Spark applications.

• Monitor and manage Spark clusters to ensure high availability and reliability.

• Implement data quality and validation processes to ensure accuracy and consistency of data.

• Stay up-to-date with industry trends and best practices related to Spark, big data technologies, Python, and AWS services.

• Document technical designs, processes, and procedures related to Spark development.

• Provide technical guidance and mentorship to junior developers on Spark-related projects.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

• Proven experience as a Spark Developer or in a similar role working with big data technologies.

• Strong proficiency in Apache Spark, including Spark SQL, Spark Streaming, and Spark MLlib.

• Proficiency in programming languages such as Scala or Python for Spark development.

• Experience with data processing and ETL concepts, data warehousing, and data modeling.

• Solid understanding of distributed computing principles and cluster management.

Tata Consultancy Services Canada Inc. is committed to meeting the accessibility needs of all individuals in accordance with the Accessibility for Ontarians with Disabilities Act (AODA) and the Ontario Human Rights Code (OHRC). Should you require accommodations during the recruitment and selection process, please inform Human Resource.

Thank you for your interest in TCS. Candidates that meet the qualification for this position will be contacted within a 2 week period. We invite you to continue to apply for other opportunities that match your profile.