>>105869254

say thanks to LLMs and the milions of bot crawlers scraping every word of the internet to build datasets. They don't respect common nettiquete rules or robots.txt. They scrape every public endpoint of every website thousands of times a day - just in case anything changed and can be added to the dataset.

The same happened with crypto - if anyone exposed free computing cycles (CI/CD for instance) on the open internet, it would be abused by bots that mine crypto.
Now the same is happening not to free compute cycles, but any piece of text. Data brokers make money and everyone else suffers.