Software Engineer (Code QA)

Job Description

Mercor is hiring experienced Software Engineers to support high-impact research collaborations with leading AI labs. Freelancers will contribute to building evaluation datasets that assess AI reasoning, explanation quality, and technical judgment in coding-related interactions.

This is a unique opportunity to apply your engineering expertise toward shaping the next generation of intelligent systems.

About the Project

This evaluation dataset is code question and answer data. This data is designed to assess natural-language reasoning, explanation quality, and technical judgment in coding-related interactions, rather than executable correctness. Tasks are structured as chat-pasteable prompts that reflect realistic developer questions and include all necessary context inline (e.g., code snippets, error messages, logs, or requirements)

Key Responsibilities

Craft realistic developer prompts across multiple categories (code review, debugging, error diagnosis, configuration, and more)

Source and adapt content from real PRs to create authentic scenarios

Write clear, technically accurate model responses that demonstrate strong reasoning and explanation quality

Ideal Qualifications

2+ years of experience in software engineering, technical research, or educational content development

Degree in Software Engineering, Computer Science, or a related field (Bachelor’s minimum; advanced degree preferred)

Strong proficiency in languages like Python, JavaScript, Java, or C++

Experience with debugging, testing, and validating code

Comfortable with technical writing and attention to detail

Project Timeline

Start Date: Immediate

Duration: 1-2 months

Commitment: Part-time (15–25 hours/week, with flexibility up to 40 hours/week)

Application & Onboarding Process

Upload your resume

AI interview: A short, 15-minute conversational session to understand your background, experience, and interest in the role

Follow-up communication within a few days with next steps and onboarding details

Part-time