Data Engineer (Databricks)

ComResource is searching for a highly skilled Data Engineer with a background in SQL and Databricks that can handle the design and construction of scalable management systems, ensure that all data systems meet company requirements, and also research new uses for data acquisition.

Requirements:Design, construct, install, test and maintain data management systems.Build high-performance algorithms, predictive models, and prototypes.Ensure that all systems meet the business/company requirements as well as industry practices.Integrate up-and-coming data management and software engineering technologies into existing data structures.Develop set processes for data mining, data modeling, and data production.Create custom software components and analytics applications.Research new uses for existing data.Employ an array of technological languages and tools to connect systems together.Recommend different ways to constantly improve data reliability and quality.

Qualifications:5+ years data quality engineeringExperience with Cloud-based systems, preferably AzureDatabricks and SQL Server testingExperience with ML tools and LLMsTest automation frameworksPython and SQL for data quality checksData profiling and anomaly detectionDocumentation and quality metricsHealthcare data validation experience preferredTest automation and quality process development Plus:Azure DatabricksAzure Cognitive Services integration Databricks Foundational model IntegrationClaude API implementation a plusPython and NLP frameworks (spaCy, Hugging Face, NLTK)