Role: Data Quality Engineer (Databricks & SQL Server)
Location: 100% Remote - EST Time
Type: Long-Term Contract
Overview: We are seeking a Data Quality Engineer with strong experience in Databricks, SQL Server, and test automation to ensure the accuracy, consistency, and reliability of enterprise data. The ideal candidate will have a solid understanding of data validation frameworks, Python-based automation, and data quality processes, preferably in the healthcare domain.
Key Responsibilities:
Design and implement data validation and quality frameworks using tools like Collibra or Great Expectations.
Develop and maintain automated data testing solutions across data pipelines and warehouse environments.
Perform data profiling, anomaly detection, and root-cause analysis to identify data issues proactively.
Create and execute SQL and Python scripts for data validation and reconciliation in Databricks and SQL Server.
Collaborate with Data Engineering, Analytics, and QA teams to define and enforce data quality standards.
Document test plans, validation rules, and quality metrics to support audit and governance requirements.
Support continuous improvement by developing reusable testing frameworks and data quality dashboards.
Required Skills & Experience:
Overall IT Experience: 8-12 Years.
5+ years of experience in Data Quality Engineering, Data Validation, or Data Testing.
Strong hands-on experience with Databricks and SQL Server for data validation.
Expertise in Python and SQL for automation and data analysis.
Experience with data validation and governance tools (e.g., Collibra, Great Expectations).
Proficiency in test automation frameworks (e.g., PyTest, Robot Framework).
Experience in data profiling, anomaly detection, and data reconciliation techniques.
Strong documentation and reporting skills with focus on quality metrics and process improvement.
Preferred Qualifications:
Experience in Healthcare data validation or HIPAA-compliant environments.
Understanding of ETL processes, data lineage, and data pipeline testing.
Familiarity with cloud data platforms (Azure, AWS, or GCP).
Soft Skills:
Strong analytical and problem-solving mindset.
Excellent communication and collaboration skills.
Detail-oriented, proactive, and process-driven approach to quality assurance.