Scale AI
A large data-labelling company that supplies the human-annotated training and evaluation data AI labs run on.
Scale AI sells the unglamorous ingredient behind most modern models: labelled data. Its workforce annotates images, rates model answers against each other, writes reference solutions and checks whether a model’s reasoning holds up. Labs pay a great deal for this because the quality of a model’s training and evaluation data limits everything downstream.
It is part of a shift that killed off older crowdwork platforms like Amazon’s Mechanical Turk. The work moved from anonymous piecework at a few cents a task to vetted, better-paid specialists who can judge whether an expert-level answer is right. Mercor and Prolific occupy nearby corners of the same market.