AI chatbots tend to choose violence and nuclear strikes in wargames
newscientist.com
Did nobody really question the usability of language models in designing war strategies?
You are viewing a single comment
Did nobody really question the usability of language models in designing war strategies?
Correct, people heard "AI" and went completely mad imagining things it might be able to do. And the current models act like happy dogs that are eager to give an answer to anything even if they have to make one up on the spot.
LLM are just plagiarizing bullshitting machines. It's how they are built. Plagiarism if they have the specific training data, modify the answer if they must, make it up from whole cloth as their base programming. And accidentally good enough to convince many people.
To be fair they're not accidentally good enough: they're intentionally good enough.
That's where all the salary money went: to find people who could make them intentionally.
GPT 2 was just a bullshit generator. It was like a politician trying to explain something they know nothing about.
GPT 3.0 was just a bigger version of version 2. It was the same architecture but with more nodes and data as far as I followed the research. But that one could suddenly do a lot more than the previous version, so by accident. And then the AI scene exploded.
So the architecture just needed more data to generate useful answers. I don't think that was an accident.
It kind of irks me how many people want to downplay this technology in this exact manner. Yes you're sort of right but in no way does that really change how it will be used and abused.
"But people think it's real AI tho!"
Okay and? Most people don't understand how most tech works and that doesn't stop it from doing a lot of good and bad things.
I've been through a few AI winters and hype cycles. It made me very cynical and convinced many overly enthusiastic people will run into a firewall face first.
If that's really how they work, it wouldn't explain these:
https://notes.aimodels.fyi/researchers-discover-emergent-linear-strucutres-llm-truth/
https://notes.aimodels.fyi/self-rag-improving-the-factual-accuracy-of-large-language-models-through-self-reflection/
https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html
https://poke-llm-on.github.io/
https://arxiv.org/abs/2310.02207
Yes. There is self organization and possibility to self reflection going on in something that wasn't designed for it. That's going to spawn a lot more research.
I will read those, but I bet "accidentally good enough to convince many people." still applies.
A lot of things from LLM look good to nonexperts, but are full of crap.
https://arxiv.org/abs/2310.02207
2 author paper with interesting evidence. Again, evidence not proof. Wait for the papers that cite this one.
https://poke-llm-on.github.io/
Reinforcement learning. Cool project. Still no need to "know" anything. I usually play this type of have with short rules and monitoring the current state.
https://notes.aimodels.fyi/self-rag-improving-the-factual-accuracy-of-large-language-models-through-self-reflection/
A cool paper. Using the LLM to judge value of new inputs.
I am always skeptical of summaries of journal articles. Even well meaning people can accidentally distort the conclusions.
Still LLM is a bullshit generator that can check bullshit level of inputs.
https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html
Author later discusses training on you data versus general datasets.
I am out of my depth, but does not seem to provide strong evidence for the modem not just repeating information that shows up a lot for the given inputs.
https://notes.aimodels.fyi/researchers-discover-emergent-linear-strucutres-llm-truth/
References a 2 author paper. I am not an expert in the field, but it is important to read the papers that reference this one. Those papers will have criticisms that are thought out. In general, fewer authors means less debate between the authors and easier to miss details.