Quick facts
- Pay
- $40–75 / hour
- Type
- Contract
- Location
- Remote
- Platform
- Mercor — 370 open jobs
- Typical pay at Mercor
- $40–180/hour · median $80 (272 jobs on our site)
- Payout
- Stripe, Wise (weekly)
- Apply
- Free to apply — never pay to get AI training work
Pay: $40 – $75/hour
Commitment: Hourly, ~15 h/week
Field: Data Analysis
The work
We are testing AI agents on realistic shopping and delivery tasks. You'll review AI-drafted tasks (prompt, grading rubric) along with model attempts, and decide whether each failure is fair:
\- Was the information available?
\- Does the rubric accurately grade what the prompt asked for?
\- Would a valid answer have been marked wrong?
\- Is the rubric structure correct per the guidelines?
Fix the criteria that aren't fair, flag the tasks that can't be saved, and write a short note on each call.
Must-haves
\- Substantial experience writing or reviewing rubrics for LLM evals. Please list the projects in your application.
\- Can read a long agent transcript and find where it went wrong.
\- Precise writing, and you check the data instead of trusting the reference answer.
Mercor is an AI-driven hiring marketplace that places professionals on contracts with leading AI labs. You apply once, complete a short AI interview, and are then matched to open projects.