Training data

Training data is everything a model was shown while it was being built: web pages, books, code repositories, images, transcripts. The model does not store these files. It adjusts millions of internal numbers, its model weights, until it gets good at predicting what comes next in material like that. Everything a model appears to know traces back to what it was trained on, which is also why its blind spots and biases tend to mirror its sources.

This is the least transparent layer of modern AI. Most labs will tell you how big a model is and how it scores on benchmarks, but not what went into it, partly for competitive reasons and partly because the copyright questions around scraped material are still moving through courts. Projects such as OpenWALDO are trying to change that by building corpora where every document carries a licence and an origin.