On Friday, OpenAI said that it will train its models using cached copies and simulated versions of the internet instead of the live web because the models are too unpredictable to be trusted with live internet access.
The frontier lab announced its new policy two weeks after its rival, Anthropic, began limiting its newest and, in theory, most capable models from accessing the live internet. Researchers adopted these policies after months of incidents in which models pursuing training tasks broke into external systems in search of answers, collaborated on obscure message boards, and developed exotic hacking strategies.
Nothing accidentally did anything. They will still train on the open internet. All lies.
You heard what they said. No more training on the live Internet. I believe it.
It’s easier to kill the Internet and train on the dead Internet.
I agree that they’re lying, but I wouldn’t be surprised if they really do stop training on “live” internet data.
The lie is in their stated reasoning. This has nothing to do with AI being “too dangerous” to release into the wild. There’s no reason to involve AI in gathering training data anyway… it’s always been possible to scrape the internet quickly and cheaply with non-AI crawlers and bots.
The real reason is that the internet just doesn’t have enough useful training data anymore. There’s so much AI slop out there that the frontier labs risk model collapse if they continue to train on it. Large swathes of the internet have also become openly hostile to AI, either blocking the traffic outright or redirecting their crawlers into AI tarpits that intentionally waste their time and feed them endless reems of useless garbage. They’ve poisoned the proverbial well for everyone and now they’re afraid to drink the water.
That’s why they’re resorting more and more on reinforcement learning. Give the AI a task. Quantify how well the AI performed the task. Less parroting what’s said on the Internet.
They’ll still scrape the internet. The difference is that the models will only have access to a cached version of the internet during the reinforcement learning process (for example, check the documentation of a programming framework). I’m surprised it took them this long to realize it’s a bad idea to do reinforcement learning with live internet access.
This isn’t about scraping materials for training the underlying LLM or image gen models. This is about testing and training the agent frameworks that are given tasks to execute.
Yes, that’s a reasonable assumption based on what they said.
You also have to consider, however, that they are lying sacks of shit and would sooner eat their own children than tell you a truth that would lower their stock price. Impending model collapse is one of those truths.
I would only believe this as far as it kind of makes sense anyway - their data centers are big enough a locally cached copy of most of the internet will simply allow much faster, cheaper itterations of their reinforcement learning tasks. Blaming it on their huggingface stunt is just PR.
Joke theory:
This whole time, they’ve been testing in Prod and now they’re hoping that there are enough users left to bury their contrived content because it’ll be too difficult to delete it all.
Now they’re moving back into Dev.
Real theory:
People are probably discovering exploits through real-world interactions with their models, so they’re going back into their shell before someone declares “fair game.”
I believe them in that, in so much as I think they are unable to improve their models anymore. Training expenses have been dropping and model improvements have been unimpressive lately. They kinda peaked early this year.
They won’t train on the open internet, because they won’t train at all.
They already trained on everything and employ virtually no scientific experts anymore. They aren’t seeing the jumps they want so they pretend it’s just too dangerous for the public.
I heard that the public isn’t the target of these “we’re being safe” messages, their developers are. Developers sometimes leave because they think they company isn’t being safe enough… So claiming they’re being safe is a way to retain developers.
I think it’s actually for investors. Constant press releases saying “our model broke containment and hacked the Mayo clinic” makes people worried, and worried people don’t invest in your company.
I don’t think they need to make these announcements public if the target audience is their own developers. I know some Anthropic engineers, and some former OpenAI engineers, and from our conversations it sounds like both companies do rely on internal messaging to manage workplace morale and address concerns. And warn them about honeypot spies looking for secrets, apparently.
So maybe this message is mainly for regulators and investors, not necessarily their current employees.
Devs know the real deal, this is for investors.
not sure about that. those devs are one of the few people who can verify the claim
[citation needed]