Task & Evaluation Specialist
Quick facts
- Pay
- $60/hr
- Type
- Contract
- Location
- Remote
- Platform
- Verita AI — 18 open jobs
- Typical pay at Verita AI
- $35–115/hour · median $50 (13 jobs on our site)
- Payout
- Stripe (weekly (Friday) · reported by workers)
- Apply
- Free to apply — never pay to get AI training work
What this role does Design realistic, difficult shopping-related tasks (e.g. meal planning, restocking, gifting, travel prep) that stress-test AI models, then build the rubrics used to grade how well a model performs them.
Who we're looking for
- Subject-matter expertise in retail, e-commerce, consumer shopping behavior, or a related domain (nutrition, logistics, home/office management, etc.)
- Strong analytical and writing skills — able to define precise, low-variance grading criteria
- Comfortable working with AI outputs and evaluating them critically and objectively
- Prior experience in eval design, QA, tutoring, or curriculum/rubric design is a plus
Core responsibilities
- Author realistic shopping tasks with a clear, concrete end goal
- Build a grading rubric/verifier for each task that two independent reviewers would score consistently
- Review an AI agent's attempt at the task and grade it against the rubric
- Document where the AI succeeded or failed, with clear reasoning
Work style
- Independent, deadline-driven work
- High attention to detail and consistency
- Comfortable with ambiguity — tasks are self-directed with light guidelines
Verita engagements are remote and asynchronous, and Verita says they are paid weekly.