Rubric Graders -Generalist Evaluators for AI Projects

Rubric Graders – Generalist Evaluators for AI Projects

Role Overview Mercor is seeking highly analytical generalists to contribute as rubric graders for a leading AI research initiative. This project supports the evaluation of AI-generated content across multiple domains, ensuring accuracy, coherence, and alignment with task-specific criteria. Contractors will apply structured rubrics to assess outputs in fields such as humanities, STEM, business, and general knowledge. This is a flexible, short-term engagement ideal for intellectually curious professionals with a strong academic background.

Key Responsibilities

Apply detailed evaluation rubrics to assess AI-generated responses

Analyze outputs for factual correctness, clarity, and domain relevance

Work across diverse subject areas and adapt to evolving task requirements

Maintain high accuracy while completing a high volume of evaluations

Follow written instructions precisely and provide concise feedback when needed

Ideal Qualifications

Strong analytical and critical thinking skills

Exceptional attention to detail and reading comprehension

Proven ability to follow structured guidelines without deviation

Comfortable working independently and across unfamiliar subject areas

Master’s degree or higher, or undergraduate degree from a top global university

More About the Opportunity

Remote and asynchronous — set your own schedule

Expected commitment: ~40 hours/week

Short-term project with potential for extension or future engagements

Contractors must have access to a reliable internet connection

Compensation & Contract Terms

$40/hour, depending on experience and location

Independent contractor status

Weekly payments via Stripe Connect

Application Process

Submit your resume to get started

Complete an AI interview and complete a Google Form

Mercor will follow up with next steps within the next 24 hours

About Mercor

Mercor is a talent marketplace that connects top experts with leading AI labs and research organizations

Our investors include Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey

Thousands of professionals across law, research, engineering, and creative domains contract with Mercor to support cutting-edge AI projects

Let me know if you'd like a version that targets a different level of education or emphasizes a particular domain.

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Note:Application Link to be given to applicants.