QA TA Lead

Total Experience-3-6Yrs

Location-Gurgaon

Role and Responsibilities Overview:

Develop and implement a comprehensive quality assurance strategy for data engineering projects, ensuring that data quality standards are met throughout the data pipeline.

Collaborate with cross-functional teams to define test plans and strategies for validating data pipelines, ETL processes, and transformations using PySpark, Databricks, and SQL.

Design and execute tests using SQL queries and data profiling techniques to validate the accuracy, completeness, and integrity of data stored in various data repositories, including data lakes and databases.

Adhere to formal QA processes, ensuring that the Systems Implementation (SI) team is using industry-accepted best Practices.

Ensure data accuracy in both SQL and the Data Lakehouse platform. Use detailed and effective written communication skills to document the features tested and bugs found.

Conduct performance testing of data processing jobs and workflows to identify bottlenecks, optimize code, and improve overall system performance using PySpark performance tuning techniques.

Act as a key point of contact for all QA aspects of releases, providing test services and coordinating QA resources internally and externally in SQL, PySpark/Python, Databricks Delta, and Delta Live Tables.

Identify the critical defects and work with BI and Data Engineering teams to mitigate them; define and enforce the acceptance criteria for testing, which inherently determines the approval for code promotions to higher environments.

Candidate Profile

Bachelor’s/Master's degree in computer science/engineering, operations research, or related data and analytics areas; candidates with BA/BS degrees in the same fields from the top tier academic institutions are also welcome to apply

Proficiency in Data Engineering, Data Management (ETL) solutions with capabilities with HIVE (preferred)

In-depth understanding of PySpark, Python, SQL, and Databricks Delta and Delta Live Tables with Unity Catalog (preferred)

Insurance Industry knowledge preferred.

QA best practices and methodologies to design, implement, and automate processes.

Knowledge on SQL & PySpark (Mandatory)

Experience extracting and manipulating data from relational databases with advanced SQL, HIVE, Python/PySpark.

Proficiency in design and test bed creation to support various test cases and accounts for variety and volume of test data.

Test strategy development experience and demonstrate knowledge in the key areas of managing timelines and dependencies across multiple tests in an agile environment.

The entire software development life cycle and test cycles (unit, Regression, Functional, Systems, stress& scale, smoke and sanity)

Strong knowledge of design principles, color proposition, and typography.

3-6yrs years experience in data engineering, preferably in data lake/data Lakehouse design-based platforms

Outstanding written and verbal communication skills

Able to work in a fast-paced, continuously evolving environment and ready to take uphill challenges.

Able to understand cross-cultural differences and can work with clients across the globe.

Self-motivated, works well independently and with others.

Using metrics-driven approach and closed-loop feedback to improve software deliverables and improve the predictability and reliability of releases.