★ Saved 🔔 Job alerts
Workada hires in United States, Canada. Find all AI training jobs open in your country →
Workada logo

Software Engineer – AI Agent Evaluation

Workada Workada ContractRemote (US, Canada) $100/hr
Posted Nov 20, 2026 · ✓ Still open — checked 11 hours ago
Apply now →

Quick facts

Pay
$100/hr
Type
Contract
Location
Remote (US, Canada)
Background
PhD
Experience
5+ years
Skills
Python, SQL, Docker, Linux
Platform
Workada — 36 open jobs
Typical pay at Workada
$23–130/hour · median $100 (14 jobs on our site)
Payout
Stripe (3 times a week)
Apply
Free to apply — never pay to get AI training work

What You'll Do

As a contributor, you'll take problems from your own work and turn each one into a self-contained Task that an AI agent has to solve in a Linux terminal. For every Task, you'll write a clear brief, set up the environment with the files and tools the agent needs, build a reference solution, and write automated checks on the final result. The strongest Tasks are the ones today's best agents still get wrong. Your firsthand knowledge of where real work in your field goes wrong is what makes them valuable. That might be a race condition that only shows up under load, a migration that silently drops rows, a compiler pass that miscompiles an edge case, a consensus protocol that splits on a network partition, or a CI build that flakes because of test ordering.

Specialties We're Looking For

We're building depth across: Algorithms and data structures Distributed and operating systems , including concurrency and consensus Databases and SQL , including migrations and query planning Data engineering : pipelines, ETL, and streaming Compilers and programming languages Frontend and web DevOps, CI/CD, and build systems

You're a Great Fit If

You hold, or are completing, a PhD in computer science or software engineering or a closely related field, or you have 5+ years of professional experience in it. You've done technical or computational work in your field and are comfortable in a Linux terminal: shell commands, scripting in Python or Bash, and installing packages. You've used Docker, or can pick it up quickly. We share templates and worked examples. You can draw on your own research, projects, code, or files as the source material for your Tasks. You have a precise, adversarial eye; you enjoy finding the exact point where a workflow breaks and explaining it cleanly. You're honest about uncertainty and write instructions that leave no room for misreading. You're based in the US or Canada. Academics and practitioners are equally welcome. No AI or machine-learning background is needed. Your expertise and your comfort working in a terminal are what we're after.

Materials You Can Use

Almost any piece of your work can seed a Task, for example: Repositories, schemas, and failing test suites CI configs and build scripts Data pipelines and the datasets behind them Profiler traces, logs, and incident write-ups Pull requests and bug fixes from your own work Bugs, edge cases, and failure modes you've had to debug yourself