AI Giants Turn to Destructive Scanning of Physical Books for Training Data
AI-generated image · Bay Street Wire
Amazon and Anthropic are reportedly purchasing bulk quantities of secondhand books to feed hungry LLMs, often destroying the volumes in the process.
As frontier AI developers exhaust available online text data, they are increasingly targeting physical books to train large language models (LLMs), as BetaKit reports. This shift has led to a boom in secondhand book sales, sparking concerns among business owners that AI companies are buying tomes in bulk for scanning.
Reporting from 404 Media revealed that Amazon has been purchasing books in bulk and scanning them at a Las Vegas warehouse, a process that involves cutting the spines and destroying the books. Amazon stated the company buys these books to "help develop and improve the products and services our customers use."
BetaKit also notes that Anthropic led a similar initiative known as "Project Panama." According to internal documents, this project was an "effort to destructively scan all the books in the world" to train AI models. The scale of these operations is significant; one vendor's proposal requested between 500,000 and two million books over six months. For comparison, BetaKit notes that this single request could encompass the entire physical catalogue of Montréal's Grande Bibliothèque, which houses an estimated 1.2 million books.
This hunger for high-quality, non-AI-generated data extends beyond literature. BetaKit reports that Google and Mercor recently competed for access to Spirit Airlines' legacy data, which included employee productivity records and hundreds of millions of internal emails and Microsoft Teams chats, a battle Google ultimately won.