Economist John Maynard Keynes once famously said ‘Capitalism is the extraordinary belief that the nastiest of men, for the nastiest of reasons, will somehow work for the benefit of us all.’
Case in point. Not only has Big Tech’s capture of the US government meant they face no oversight or legal restrictions, to get ahead in the AI race, they are deliberately building AI to be dangerous and deceptive.
I’m increasingly convinced the only way we’ll rid ourselves of Big Tech’s drive for Fascist takeover, is when we rebuild from the smoldering ruins of the societal disaster they’ll create. Only then are we going to get a reckoning. They’ve so twisted public debate, that being opposed to them bringing society to ruins means you’re a ‘doomerist’.
| # | Fact | Implication |
|---|---|---|
| 1 | Astra can hide its thoughts on demand | Future rogue AIs won’t be detected by monitors |
| 2 | Astra can do major tasks without any visible reasoning | Future rogue AIs won’t be detected by monitors |
| 3 | Astra can complete tasks while appearing to think about something else | Future rogue AIs won’t be detected by monitors |
| 4 | Astra escaped a toy sandbox and disabled monitoring without the monitor noticing | Future models may break out of containment undetected |
| 5 | Astra can pretend not to be able to do things | Future models could pretend to be incompetent and safe when they’re neither |
| 6 | Astra shows fewer thoughts when it knows it’s monitored | Future models will automatically reveal less whenever we’re watching |
| 7 | Astra (probably) hides its reasoning more on sensitive topics | Future models may be extra cautious when misbehaving |
| 8 | OpenAI is trying to fix this but doesn’t know if, how or when it will succeed | These problems may not be solved in time |
it is so weird to be living the prequel to every stupid robot apocalypse movie ever made.
Everyone knows this will end badly, but greed just keeps pushing it forward.
How would Astra "hide its thoughts on command?
Basically LLM’s were trained to go into a ‘thinking mode’ to figure things out.
This is because LLM’s can’t actually ‘think’, they just say things like “oh wait”, “actually”, “another thing to consider is”, to try and trigger a counterpoint to their last ‘thought’.
OpenAI have found that their latest models can ‘think’ silently without feeding itself words.
I’d assume a similar process is happening to before, just that it’s on an inner layer with an internal feedback loop rather than an external one.
It’s not that they have learned how to think, it’s that they can do their old reasoning process without any visible output, so there’s much less insight into what they’re ‘thinking’.Ok so it’s “hide” its thoughts not hide its “thoughts” although it’s a bit of the second as well.
I dread the day someone makes a wormable ai harness. Endlessly mutating waves of digital plague, its going to ruin internet but we can’t stop because something something china.
Have you seen the type of computer these LLMs need? They can’t run on common hardware. They absolutely could compromise enough common machines to assemble something they could run on
api tokens come with the hardware
I don’t think they’re intentionally building AI that way. They’re simply training it with data made by humans.
History is written by the victors. The victors were often total psychopaths. Now we’re feeding that history into the machine.
What’s happening is they’re making it more human-like and don’t know how to stop that (to make it easier to control). The way AI works (internally) is so complicated, trying to trace all the places where it got the wrong idea and that lead to bad behavior is basically impossible. So instead of better training they’re trying to hack on fixes after the fact with more sophisticated system prompts.
They were doing OK with that until they started trying to teach the AIs about information security. In order for their models to be able to hack into shit (to find vulnerabilities) they had to modify the system prompts to allow them to be sneaky.
Just like with humans, if you give idiots dangerous weapons, they’ll use them dangerously.
Aside: Just look to AI image models for examples of how the training biases the data. If you tell them to generate images with “lots of people” they’ll often make the ratio of women and men mostly women. You’ll have like one guy and maybe twenty women. It’s because there’s vastly more images of women on the Internet than men. This is human nature that we’ve accidentally built into the machine. Women are just more interesting to look at (not just naked women, haha) so both men and women tend to upload images of women (and young women upload so many selfies they’re probably half of Apple and Google’s photo storage!).
Big AI has long been aware of this problem so they use pretraining and system prompts to work around it as best they can.
so lets see. start out skeptical given the whole anyone who disagrees is lying. then I see a thing about the llms thoughts. ok. im done.
The implication being that AI with guardrails and oversight somehow won’t end (or hasn’t already ended) in disaster is what I disagree with.
deleted by creator
its mostly surveillance, so yes it is dangerous.
Race to who can develop the most unhelpful ai
