How Quickly Do Large Language Models Learn Unexpected Skills?
nautil.us
So-called "emergent" behavior in LLMs may not be the breakthrough that researchers think.
You are viewing a single comment
So-called "emergent" behavior in LLMs may not be the breakthrough that researchers think.
TLDR: Let's say you want to teach an LLM a new skill. You give them training data pertaining to that skill. Currently, researchers believe that this skill development shows up suddenly in a breakthrough fashion. They think so because they measure this skill using some methods. The skill levels remain very low until they unpredictably jump up like crazy. This is the "breakthrough".
BUT, the paper that this article references points at flaws in the methods of measuring skills. This paper suggests that breakthrough behavior doesn't really exist and skill development is actually quite predictable.
Also, uhhh I'm not AI (I see that TLDR bot lurking everywhere, which is what made me specify this).
An AI would say that... 😂
Clearly, the AI is learning deception
re: your last point, AFAIK, the TLDR bot is also not AI or LLM; it uses more classical NLP methods for summarization.
https://github.com/RikudouSage/LemmyAutoTldrBot readme say summarization is in summarizer.py which use sumy, specifically LSA which documented here
Natural language processing falls under AI though, and so do large language models (see chapters 23 and 24 of Russell and Norvig, 2021 http://aima.cs.berkeley.edu/).
That's exactly what an AI would say that got an emergent skill to lie
🤥
Or a model that picked up on a pattern of sources saying that.