As expected, they can't be trusted. And the more AI evolves, the less likely AI content will be detectable IMO.
It will almost always be detectable if you just read what is written. Especially for academic work. It doesn't know what a citation is, only what one looks like and where they appear. It can't summarise a paper accurately. It's easy to force laughably bad output by just asking the right sort of question.
The simplest approach for setting homework is to give them the LLM output and get them to check it for errors and omissions. LLMs can't critique their own work and students probably learn more from chasing down errors than filling a blank sheet of paper for the sake of it.
given how much AI has advanced in the past year alone, saying it will "always" be easy to spot is extremely short sighted.
People seem to grasp onto weaknesses AI has now and say that they will have them forever, like how text AI lies, and image generation AI can't draw hands.
But these AIs are advancing unimaginably quick, 2 years ago generated text was pretty bad, becoming pretty incoherent, and 1 year ago generated images were mostly strange mush.
Spot on! Actually people still talk about hands but it's already been solved with many newer image gen models... The hands they produce look perfectly fine usually these days.
Some things are inherent in the way the current LLM's work. It doesn't reason, it doesn't understand, it just predicts the next word out of likely candidates based on the previous words. It can't look ahead to know if it's got an answer, and it can't backtrack to change previous words if it later finds out it's written itself into a corner. It won't even know it's written itself into a corner, it will just continue predicting in the pattern it's seen, even if it makes little or no sense for a human.
It just mimics the source data it's been trained on, following the patterns it's learned there. At no point does it have any sort of understanding of what it's saying. In some ways it's similar to this, where a man learned how enough french words were written to win the national scrabble competition, without any clue what the words actually mean.
And until we get a new approach to LLM's, we can only improve it by adding more training data and more layers allowing it to pick out more subtle patterns in larger amounts of data. But with the current approach, you can't guarantee that what it writes will be correct, or even make sense.
it just predicts the next word out of likely candidates based on the previous words
An entity that can consistently predict the next word of any conversation, book, news article with extremely high accuracy is quite literally a god because it can effectively predict the future. So it is not surprising to me that GPT's performance is not consistent.
It won't even know it's written itself into a corner
It many cases it does. For example, if GPT gives you a wrong answer, you can often just send an empty message (single space) and GPT will say something like: "Looks like my previous answer was incorrect, let me try again: blah blah blah".
And until we get a new approach to LLM's, we can only improve it by adding more training data and more layers allowing it to pick out more subtle patterns in larger amounts of data.
This says nothing. You are effectively saying: "Until we can find a new approach, we can only expand on the existing approach" which is obvious.
But new approaches come all the time! Advances in tokenization come all the time. Every week there is a new paper with a new model architecture. We are not stuck in some sort of hole.
An entity that can consistently predict the next word of any conversation, book, news article with extremely high accuracy is quite literally a god because it can effectively predict the future
I think you're reading something there other than what I said. Look, today's LLM's ingest a ton of text - more accurately tokens - and builds up statistics of which tokens it sees in that context. So statistically if you see the sentence "A nice cup of " statistically the next word is maybe 48% coffee, 28% tea, 17% water and so on. If earlier in the text it says something about heating a cup of oil, that will have a muuch higher chance. It then picks one of the top tokens at (weighted) random, and then the text (array of tokens) is fed in again into the LLM and a new prediction is made. And so on it continues until you stop the loop (usually from a end token or a keyword you're looking for). Larger LLM's are better at spotting more subtle patterns - or more accurate it got more layers of statistics that's applied - but it still has the fundamental issue of going one token at a time and just going by what's most likely to be the next token.
It many cases it does. For example, if GPT gives you a wrong answer, you can often just send an empty message (single space) and GPT will say something like: “Looks like my previous answer was incorrect, let me try again: blah blah blah”.
Have you tried that when it's correct too? And in that case you mention it has a clean break and then start anew with token generation, allowing it to go a different path. You can see it more clearly experimenting with local LLM's that have fewer layers to maintain the illusion.
This says nothing. You are effectively saying: “Until we can find a new approach, we can only expand on the existing approach” which is obvious.
But new approaches come all the time! Advances in tokenization come all the time. Every week there is a new paper with a new model architecture. We are not stuck in some sort of hole.
We're trying to make a flying machine by improving pogo sticks. No matter how well you design the pogo stick and the spring, it will not be a flying machine.
The issue here is that you are describing the goal of LLMs, not how they actually work. The goal of an LLM is to pick the next most likely token. However, it cannot achieve this via rudimentary statistics alone because the model simply does not have enough parameters to memorize which token is more likely to go next in all cases. So yes, the model "builds up statistics of which tokens it sees in which contexts" but it does so by building it's own internal data structures and organization systems which are complete black boxes.
Also, going "one token at a time" is only a "limitation" because LLMs are not accurate enough. If LLMs were more accurate, then generating "one token at a time" would not be an issue because the LLM would never need to backtrack.
And this limitation only exists because there isn't much research into LLMs backtracking yet! For example, you could give LLMs a "backspace" token: https://news.ycombinator.com/item?id=36425375
Have you tried that when it’s correct too? And in that case you mention it has a clean break and then start anew with token generation, allowing it to go a different path. You can see it more clearly experimenting with local LLM’s that have fewer layers to maintain the illusion.
If it's correct, then it gives a variety of responses. The space token effectively just makes it reflect on the conversation.
We’re trying to make a flying machine by improving pogo sticks. No matter how well you design the pogo stick and the spring, it will not be a flying machine.
To be clear, I do not believe LLMs are the future. But I do believe that they show us that AI research is on the right track.
Building a pogo stick is essential to building a flying machine. By building a pogo stick, you learn so much about physics. Over time, you replace the spring with some gunpowder to get a mortar. You shape the gunpowder into a tube to get a model rocket and discover the pendulum rocket fallacy. And finally, instead of gunpowder, you use liquid fuel and you get a rocket that can go into space.
The issue here is that you are describing the goal of LLMs, not how they actually work.
it cannot achieve this via rudimentary statistics alone because the model simply does not have enough parameters to memorize which token is more likely to go next in all cases.
True, hence the limitations. That would require infinite storage and infinite compute capability.
Also, going “one token at a time” is only a “limitation” because LLMs are not accurate enough.
No, it's done because one letter at a time is too slow. Tokens are a "happy" medium tradeoff.
The space token effectively just makes it reflect on the conversation.
It makes a "break" of the block, which lets it start a new answer instead of continuing on the previous. How it reacts to that depends on the fine tune and filters before the data hits the LLM.
To be clear, I do not believe LLMs are the future.
I have just said that LLM's we have today can't fix the problems with false data and hallucinations, because it's a core principle of how it operates. It will require a new approach.
You could add a rocket engine and wings to a pogo stick, but then it's no longer a pogo stick but an airplane with a weird landing gear. Today's LLM's could give us hints to how to make a better AI, but that would be a different thing than today's LLM's. From what has been leaked from OpenAI GPT4 has scaling issues so they use mixture of experts. Just throwing hardware at it is already showing diminishing returns. And we're learning fascinating new ways of training them, but the inherent problem is the same.
For example, if you ask an LLM if it can give an answer to a question, it will have two paths to go down, positive and negative. Note, at the point where it chooses that it doesn't know how to finish it, it doesn't look ahead. But it sees for example that 80% of the answers in the texts it's been trained on starts with a positive, then it will most likely start with "yes" - and when it does that it will continue to generate an answer - often very convincing and plausibly real looking answer, because it already committed to that path.
And as for the link about teaching it backspace token, the comments there are already pointing out the issue:
It's interesting that in the examples (Table 3 on page 21), the model uses the backspace token to erase the randomly-added token from the prompt, but it does not seem to ever use the token to correct its own output. I'm curious how frequently the model actually uses this backspace token in practice - and if the answer is "vanishingly rarely", what is the source of the improved Mauve score and sample diversity they show? Is it just that the different training procedure gives an improvement?
For it to use the backspace, wouldn't it have to predict the wrong token with greater confidence than the corrected token? I would think this would require more examples of a wrong token + correction than the correct token, which seems a bit odd.
Almost none of the text it's trained on has a backspace token, and to finetune it in is tricky since it's a completely new concept - and remember it's still doing token for token - so it would have to write a token and then right after find out that it's more likely to send a backspace token than to continue it. It's interesting, and LLM's can pick up on some crazy patterns, but I'm skeptical.
No, I am describing how they actually work.
First of all, this link is just to C# bindings of llama.cpp and so doesn't contain the actual implementation. But it also doesn't refute my criticism of your claim. More specifically, I take issue with this statement that you said: "today’s LLM’s ingest a ton of text [snip] and builds up statistics of which tokens it sees in that context".
I claim that this is not how today's LLMs work because we have no idea what LLMs do with the input data during training. We have very little insight into what kind of data structure it builds and how the data structure it built is organized.
No, it’s done because one letter at a time is too slow. Tokens are a “happy” medium tradeoff.
I think I worded my sentence ambiguously, let me re-word it for you: "Going one token at a time is only considered a limitation because LLMs are not accurate enough"
It makes a “break” of the block, which lets it start a new answer instead of continuing on the previous. How it reacts to that depends on the fine tune and filters before the data hits the LLM.
Once again, my sentence was not written well, my bad. I was commenting on the observed behavior, not on how it works from a technical perspective.
I have just said that LLM’s we have today can’t fix the problems with false data and hallucinations, because it’s a core principle of how it operates. It will require a new approach.
You could add a rocket engine and wings to a pogo stick, but then it’s no longer a pogo stick but an airplane with a weird landing gear. Today’s LLM’s could give us hints to how to make a better AI, but that would be a different thing than today’s LLM’s. From what has been leaked from OpenAI GPT4 has scaling issues so they use mixture of experts. Just throwing hardware at it is already showing diminishing returns. And we’re learning fascinating new ways of training them, but the inherent problem is the same.
Alright, we agree here for the most part so I'm just going to skip this.
For example, if you ask an LLM if it can give an answer to a question, it will have two paths to go down, positive and negative. Note, at the point where it chooses that it doesn’t know how to finish it, it doesn’t look ahead.
This is weird though. How do you know LLMs can't look ahead? When we prompt LLMs, we are basically asking them this question: "What is the next word of your response?" How do you know it hasn't written out the entire response in memory already after which it only shows you the first word? LLMs are neural networks. Neural networks have working memory. That's how neural networks work after all, it's just a vector of data that is repeatedly transformed as it passes through each layer. Of course, if it does write the entire response in memory, it is all thrown away after every word.
As far as the backspace tokens go, you are right to be skeptical but also do not be surprised if it works out. We've had LLMs trained to complete and edit text for some time already. They've fallen out of use today but they did perform acceptably well.
First of all, this link is just to C# bindings of llama.cpp and so doesn’t contain the actual implementation.
I know, it's my code. I refactored it from some much less readable and usable c# code. I picked it because it more clearly shows the steps involved in generating text.
How do you know LLMs can’t look ahead? [...] How do you know it hasn’t written out the entire response in memory already after which it only shows you the first word?
Firstly, it goes against everything we know so far of how they operate, and secondly.. because they can't.
If you look at the C# code, the first step is in _process_tokens function, where it feeds the context into llama_eval. That goes through each token and updates the internal memory / model state. Since it saves state, if you already have processed some of the tokens you can tell it to skip them and start on the new ones.
After this function you have a state in memory, the current state of the LLM, as a result of the tokens it's seen so far.
When we are done with that, we go to the more interesting part, the _predict_next_token function. Note that that takes a samplingparams parameter. It then set some options, like if top k is not set it's set to length of the model's vocabulary (number of tokens it knows about), and repeat_last_n, if not set, is set to the length of the existing context.
The code then gets the model's vocabulary, aka all the tokens it knows about, and then it generates the logits. The logits is an array the length of the vocabulary, with a number for each token showing how likely that one is the next token. The code then adds any specified token bias to that token's number. Already here, even if it already had a specific answer in mind, you can see problems starting.
Then the code adds token repetition penalty, based on the samplingparams. This means that if a token repeats inside the given history, it's value will be lowered according to the repeat_penalty. Again, even if it had a specific answer, this has a high chance of messing that up. The same is done for frequency and presence. For more details of what those native functions do, you can see the llama.cpp source - they have the same name there.
After all the penalties are applied, it's time to pick the token. If the temp is 0 or lower, it just picks the highest rated token (aka greedy sampling). This tends to give very boring and flat responses, but it's predictable and reproduceable, so it's often used in benchmarks of various kinds.
But if that's not used (which it almost never is in "real" use), there are several methods. You have MiroStat, which tries to create more consistent quality between different answer lengths, and the "traditional" using top-k, top-p and temperature.
Common for them is however that internally it produces a top list of candidates, and then pick one at random. And that's why a LLM can't plan ahead.
When a token ID is eventually produced it returns the new ID, that gets added to context, the text equivalent of the token is looked up and sent back to the UI, and the new context is fed into llama_eval and the process starts again.
For the LLM to even be able to plan an answer ahead it must know of all penalties and parameters (or have none applied), and greedy token prediction must be used.
And that is why, even if it had some sort of near magical ability to plan ahead that we just don't know is there, at the end of the day it could still not plan a specific response.
I know, it’s my code.
Wow, very nice! First of all, I will preface by admitting that I have not worked with LLMs to the degree of making a toy implementation. Your explanation of the sampling techniques is insightful but doesn't clear up my confusion. Why does sampling imply the absence of higher level structure in the model?
For example, even though poker is highly influenced by chance, I can still have a plan that will increase my likelihood of winning. I don't know what card will be drawn next but I can prepare strategies for each possible card. I can have preferences for which cards I want to be drawn next.
You know what, I don't have a good answer to you here. I did a few small experiments on ChatGPT and it seems like it has some knowledge of if it will be able to complete it or not. This was with a pretty well known question though.
I tried to recreate an earlier experiment where I asked it to write about a friend of mine, which was in the news some time ago and have apparently a few entries in it's training data, but very little. ChatGPT would then consistently hallucinate facts about the person, including date of birth and sometimes date of death. In that case it knew the pattern of writing about a person including date of birth, and sometimes date of death, but it didn't know it didn't have that info and just filled in plausible looking data there. Now it insists on not knowing who that person is at all and refuses to write anything about him.
Anyway, you've given me some things to think about, thanks.
This is not entirely correct, in my experience. With the current version pf gtp-4 you might be right, but the initial versions were extremely good. Clearly you have to work with it, you cannot ask for the whole work
I meant initial versions of chatGTP 4. ChatGTP isn't lying, simply because lying implies a malevolent intent. Gtp-4 has no intent, it just provides an output given an input, that can be either wrong or correct. A model able to provide more correct answers is a more accurate model. Computing accuracy for a LLM is not trivial, but gpt-4 is still a good model. User has to know how to use it, what to expect and how to evaluate the result. If they are unable to do so it's completely their fault.
Why are you so pissed of a good nlp model?
I think there’s a big difference between being able to identify an AI by talking to it and being able to identify something written by an AI, especially if a human has looked over it for obvious errors.
Whoops, meant to say: "In many cases, they can accurately (critique their own work)". Thanks for correcting me!
What you are describing is true of older LLMs. GPT4, it's less true of. GPT5 or whatever it is they are training now will likely begin to shed these issues.
The shocking thing that we discovered that lead to all of this is that this sort of LLM continues to scale in capabilities with the quality and size of the training set. AI researchers were convinced that this was not possible until GPT proved that it was.
So the idea that you can look at the limitations of the current generation of LLM and make blanket statements about the limitations of all future generations is demonstrably flawed.
They cannot be anything other than stochastic parrots because that is all the technology allows them to be. They are not intelligent, they don't understand the question you ask or the answer they give you, they don't know what truth is let alone how to determine it. They're just good at producing answers that sound like a human might have written them. They're a parlour trick. Hi-tech magic 8balls.
They cannot be anything other than stochastic parrots because that is all the technology allows them to be.
Are you referring to humans or AI? I'm not sure you're wrong about humans...
IMO it does not do a good job of disproving that "humans are stochastic parrots".
The example with the octopus isn't really about stochastic parrots. It's more about how LLMs are not multi-modal.
That article is super helpful.
Thanks!
I'm no GPT booster, but I think that the real problem with detectability here
It will almost always be detectable if you just read what is written. Especially for academic work.
is that it requires you to know the subject and content already, and to be giving the paper a relatively detailed reading. For a rube reading the paper, trying to learn from it - a lot of GPT content is easily mistaken as legitimate. And it's getting better. We're not safe simply assuming that AI today is as good as it will ever get and the clear errors we can detect cannot ever be addressed.
Penetrating academic writing, for academics, is probably one of the highest barriers of any writing task, AI or not.
But being dismissive of the threat of AI content because it's not able to convincingly fake some of the hardest writing that real people do is maybe sidestepping a lot of much more casual writing - that still carries significance and consequence.
Chad comment right here...
Clearly the Founding Fathers were not advanced enough to have crafted the US Constitution unaided. It's only reasonable to imagine that ancient aliens could have landed, given them an AI to assist them, and then departed with nobody the wiser.
I am certain we can find evidence of this if we dig hard enough.
James Madison was a fraud! He used ChatGPT instead of writing out his homework like he was supposed to. Smh my head.
Clearly the Founding Fathers were not advanced enough to have crafted the US Constitution unaided.
In a sense you are correct. They cribbed from lots of the most well known political philosophers at the time. For example, there are direct quotes from Locke in the Declaration and his influence over the Constitution can be felt clearly.
I am certain we can find Georgios Tsoukalos saying there is evidence of this sooner or later. Probably extraterrestrial!
As an AI language model, I am unable to found a country but here are some ideas which might help people feel less likely to revolt.
This is a simple example of a constitution. For a real constitution you might want to consider a more detailed list of amendaments and civil rights. Please refer to a professional constitutional writer if you want to create a constitution for productive purposes
GPT-4 is quite willing to help you create the constitution for your own country, well I guess I have to find a name now:
Certainly! Crafting a constitution for a new country is an exciting and monumental task. The constitution is the core document that dictates the legal obligations and rights of your country's government and its citizens. Here's a very basic outline to get us started. Please remember, this is a rudimentary draft and it will need to be adjusted based on the specific needs, culture, history, and goals of your country:
PREAMBLE
We, the people of [Your Country's Name], in order to form a more perfect union, establish justice, ensure domestic tranquility, provide for the common defense, promote the general welfare, and secure the blessings of liberty to ourselves and our posterity, do ordain and establish this Constitution for [Your Country's Name].
ARTICLE I: THE LEGISLATURE
Section 1: All legislative powers herein granted shall be vested in a [name of your legislative body].
Section 2: [Details on how members are elected, their terms, qualifications, powers and responsibilities]
Section 3: [Details on how legislation is passed, veto powers, etc.]
ARTICLE II: THE EXECUTIVE
Section 1: The executive power shall be vested in a [title of your head of state].
Section 2: [Details on how the head of state is elected, their term, qualifications, powers and responsibilities]
Section 3: [Details on how the executive branch operates, and other positions within the executive branch]
ARTICLE III: THE JUDICIARY
Section 1: The judicial power of [Your Country's Name] shall be vested in one Supreme Court, and in such inferior courts as the [Legislative body] may from time to time ordain and establish.
Section 2: [Details on how judges are appointed, their terms, qualifications, powers and responsibilities]
ARTICLE IV: THE RIGHTS OF THE PEOPLE
Section 1: All persons born or naturalized in [Your Country's Name] are citizens of [Your Country's Name] and of the state wherein they reside. No state shall make or enforce any law which shall abridge the privileges or immunities of citizens.
Section 2: [Further details on the rights, freedoms, and responsibilities of the citizens, these could include things like freedom of speech, freedom of religion, right to a fair trial, etc.]
ARTICLE V: AMENDMENTS
Section 1: The [Legislative body] whenever two thirds of both Houses deem it necessary, shall propose Amendments to this Constitution, or, on the Application of the Legislatures of two thirds of the several states, shall call a Convention for proposing Amendments...
ARTICLE VI: GENERAL PROVISIONS
Section 1: [Any other items that need to be addressed, such as the official language, the national religion (or lack thereof), the national emblem, the process for international treaties, etc.]
ARTICLE VII: RATIFICATION
This Constitution shall be considered ratified when agreed upon by a majority of citizens through a free and fair vote.
Please remember that a constitution is a living document that will grow and change over time as your country evolves. This outline is just a starting point and may not include all the specific sections you need. For example, you might want to add sections addressing human rights, environmental protection, financial regulations, and so on.
This process should involve broad input from many different stakeholders, including legal experts, and should be reflective of the country's culture, values, history, and aspirations for the future. Also, please note that the effective implementation of a constitution requires a strong rule of law and mechanisms for holding those in power accountable.
💰
Because AI detectors suck and are the modern day equivalent of dowsing rods?
They're circular. If the text is too predictable it was written by an LLM* but LLMs are designed to regurgitate the next word most commonly used by humans in any given context.
*AI is a complete misnomer for the hi-tech magic 8ball
The next most commonly used word would result in a loop of common word. LLMs do not work like that
In context. And that is exactly how they work. It's just a statistical prediction model with billions of parameters.
That’s not really how LLMs work. You’re basically describing Markov chains. The statement “It’s just a statistical prediction model with billions of parameters” also applies to the human brain. An LLM is much more of a black box than you’re implying.
regurgitate the next word most commonly used by humans in any given context.
is not what it does. That would create non sensical text (you can try yourself).
This is a summary of the method, as summarized by gtp-4:
Sure, here is a detailed description of how text is generated with ChatGPT, which is based on the GPT architecture:
Initial Prompt: The process begins with an input prompt. This could be something like "Tell me about the weather today" or any other string of text.
Tokenization: The input text is broken down into smaller parts, called tokens, which can represent words, parts of words, or punctuation. GPT uses a byte pair encoding (BPE) tokenization, which essentially breaks down text into commonly occurring chunks.
Embedding: Each token is then turned into a vector via an embedding. This vector captures semantic information about the token and serves as the input for the model.
Processing the Input: The GPT model processes the input vectors sequentially with a stack of transformer layers. Each layer applies self-attention and feeds its output into the next layer.
Self-Attention Mechanism: The self-attention mechanism in the Transformer model allows it to weigh the importance of different words when predicting the next word. For example, when trying to predict the last word in the sentence "The cat sat on the ____," the words "cat" and "on" are likely to have more influence on the prediction than "The". This weighing is learned during training and allows the model to generate more coherent and contextually appropriate responses.
Output Layer: The output from the final transformer layer for the last input token goes through a linear layer followed by a softmax function, which turns it into a probability distribution over the possible next tokens in the vocabulary. Each possible next token is assigned a probability.
Sampling with Temperature: The next token is chosen based on these probabilities. One common method is to sample from this distribution, which introduces some randomness into the process. The temperature parameter controls the amount of randomness: a higher temperature makes the distribution more uniform and the output more random, while a lower temperature makes the model more likely to choose the highest-probability token.
Decoding: The chosen token is then decoded back into text and appended to the output.
Next Iteration: The process then repeats for the next token: the model takes the output so far (including the newly-generated token), processes it, and generates probabilities for the next token. This continues until a maximum length is reached, or an end-of-sequence token is produced.
Post-Processing: Any necessary post-processing is applied, such as cleaning up tokenization artifacts.
In this way, the model generates a sequence of tokens, one at a time, based on the input prompt and the tokens it has generated so far. Please note that while this process typically uses sampling with a temperature parameter, other methods like beam search or top-k sampling can also be used to choose the next token. These methods have different trade-offs in terms of computational efficiency, diversity, and quality of output.
You are missing the key part where the text is tranformed in a vector space of "concepts" where semanticic relationships are represented, that is where the inference happens. The inference is not on words to get the next commonly used word, otherwise it wouldn't work. And you also missed the final sampling to introduce a randomness in the word selection.
I don't understand why are you so upset for a chain of complex mathematical functions that complete and input sentence. Why are you angry?
You're agreeing with me but using more words.
I'm more annoyed than upset. This technology is eating resources which are badly needed elsewhere and all we get in return is absolute junk which will infest the literature for decades to come.
I am not agreeing with you because "regurgitate the next most commonly world" is not what it does.
That said, the technology is not doing anything wrong. The people using it are doing it. The technology is a great achievement of human kind, possibly one of the greatest. If people decide to use it to print sh*t is people fault. Quantum mechanics is one of the greatest achievement of human kind, if people decided to use it to kill people, it is a fault of people. Many humans are simply shitty, don't blame a clever mathematical function and its clever implementation
This article was written to keep people as long on the page as possible. It didn't get to the point before i left. Someone has a tl;dr?
Constitution is a text that appears many times on the internet. ChatGPT's training set probably has multiple copies of it. So it's likely ChatGPT will generate it. Therefore, the detectors are likely to flag it as AI-generated. That's what I got from it, but I also found it difficult to parse. Maybe someone can correct me on this.
I thimk we need and AI to summarize the article.
Edit: oh god, soon shit articles are gonna be optimized to not be AI summarizable
Bet it was written by an AI.
Thanks!
I knew it. The founding fathers were robots from the future!
I've recently checked my years-old essay using one of these AI plagiarism detectors and it said that the essay was 90% AI written. So either it's all bs or I'm a time travelling AI.
I’m convinced that it’s been trained on top of the essays of middle and high school students that have gone their whole lives without proper education on vocabulary, grammar, and the like. So when asked to evaluate something written properly, it’s flagged as AI.
Garbage in, garbage out. Same as is ever was.
CAUGHT!
Easy fix: run a constitution detector before the AI text detector
Interesting fact: The same parent company owns both Reddit and Turnitin.
I’m waiting for new conspiracy theories after that article hahah.
Obviously the US Constitution was written by AI, we're living in a simulation. Wake up sheeple, the Matrix is real!
Don’t mistake artificial intelligence for actual intelligence.
Aliens.jpg
Awwww. Beat me to it.
This. Changes. Everything.
They only know what they have been fed.
What more likely first/base feeding than the US Constitution's declarations and it's amendments?
As expected, they can't be trusted. And the more AI evolves, the less likely AI content will be detectable IMO.
It will almost always be detectable if you just read what is written. Especially for academic work. It doesn't know what a citation is, only what one looks like and where they appear. It can't summarise a paper accurately. It's easy to force laughably bad output by just asking the right sort of question.
The simplest approach for setting homework is to give them the LLM output and get them to check it for errors and omissions. LLMs can't critique their own work and students probably learn more from chasing down errors than filling a blank sheet of paper for the sake of it.
given how much AI has advanced in the past year alone, saying it will "always" be easy to spot is extremely short sighted.
People seem to grasp onto weaknesses AI has now and say that they will have them forever, like how text AI lies, and image generation AI can't draw hands.
But these AIs are advancing unimaginably quick, 2 years ago generated text was pretty bad, becoming pretty incoherent, and 1 year ago generated images were mostly strange mush.
Spot on! Actually people still talk about hands but it's already been solved with many newer image gen models... The hands they produce look perfectly fine usually these days.
Some things are inherent in the way the current LLM's work. It doesn't reason, it doesn't understand, it just predicts the next word out of likely candidates based on the previous words. It can't look ahead to know if it's got an answer, and it can't backtrack to change previous words if it later finds out it's written itself into a corner. It won't even know it's written itself into a corner, it will just continue predicting in the pattern it's seen, even if it makes little or no sense for a human.
It just mimics the source data it's been trained on, following the patterns it's learned there. At no point does it have any sort of understanding of what it's saying. In some ways it's similar to this, where a man learned how enough french words were written to win the national scrabble competition, without any clue what the words actually mean.
And until we get a new approach to LLM's, we can only improve it by adding more training data and more layers allowing it to pick out more subtle patterns in larger amounts of data. But with the current approach, you can't guarantee that what it writes will be correct, or even make sense.
An entity that can consistently predict the next word of any conversation, book, news article with extremely high accuracy is quite literally a god because it can effectively predict the future. So it is not surprising to me that GPT's performance is not consistent.
It many cases it does. For example, if GPT gives you a wrong answer, you can often just send an empty message (single space) and GPT will say something like: "Looks like my previous answer was incorrect, let me try again: blah blah blah".
This says nothing. You are effectively saying: "Until we can find a new approach, we can only expand on the existing approach" which is obvious.
But new approaches come all the time! Advances in tokenization come all the time. Every week there is a new paper with a new model architecture. We are not stuck in some sort of hole.
I think you're reading something there other than what I said. Look, today's LLM's ingest a ton of text - more accurately tokens - and builds up statistics of which tokens it sees in that context. So statistically if you see the sentence "A nice cup of " statistically the next word is maybe 48% coffee, 28% tea, 17% water and so on. If earlier in the text it says something about heating a cup of oil, that will have a muuch higher chance. It then picks one of the top tokens at (weighted) random, and then the text (array of tokens) is fed in again into the LLM and a new prediction is made. And so on it continues until you stop the loop (usually from a end token or a keyword you're looking for). Larger LLM's are better at spotting more subtle patterns - or more accurate it got more layers of statistics that's applied - but it still has the fundamental issue of going one token at a time and just going by what's most likely to be the next token.
Have you tried that when it's correct too? And in that case you mention it has a clean break and then start anew with token generation, allowing it to go a different path. You can see it more clearly experimenting with local LLM's that have fewer layers to maintain the illusion.
We're trying to make a flying machine by improving pogo sticks. No matter how well you design the pogo stick and the spring, it will not be a flying machine.
The issue here is that you are describing the goal of LLMs, not how they actually work. The goal of an LLM is to pick the next most likely token. However, it cannot achieve this via rudimentary statistics alone because the model simply does not have enough parameters to memorize which token is more likely to go next in all cases. So yes, the model "builds up statistics of which tokens it sees in which contexts" but it does so by building it's own internal data structures and organization systems which are complete black boxes.
Also, going "one token at a time" is only a "limitation" because LLMs are not accurate enough. If LLMs were more accurate, then generating "one token at a time" would not be an issue because the LLM would never need to backtrack.
And this limitation only exists because there isn't much research into LLMs backtracking yet! For example, you could give LLMs a "backspace" token: https://news.ycombinator.com/item?id=36425375
If it's correct, then it gives a variety of responses. The space token effectively just makes it reflect on the conversation.
To be clear, I do not believe LLMs are the future. But I do believe that they show us that AI research is on the right track.
Building a pogo stick is essential to building a flying machine. By building a pogo stick, you learn so much about physics. Over time, you replace the spring with some gunpowder to get a mortar. You shape the gunpowder into a tube to get a model rocket and discover the pendulum rocket fallacy. And finally, instead of gunpowder, you use liquid fuel and you get a rocket that can go into space.
No, I am describing how they actually work.
True, hence the limitations. That would require infinite storage and infinite compute capability.
No, it's done because one letter at a time is too slow. Tokens are a "happy" medium tradeoff.
It makes a "break" of the block, which lets it start a new answer instead of continuing on the previous. How it reacts to that depends on the fine tune and filters before the data hits the LLM.
I have just said that LLM's we have today can't fix the problems with false data and hallucinations, because it's a core principle of how it operates. It will require a new approach.
You could add a rocket engine and wings to a pogo stick, but then it's no longer a pogo stick but an airplane with a weird landing gear. Today's LLM's could give us hints to how to make a better AI, but that would be a different thing than today's LLM's. From what has been leaked from OpenAI GPT4 has scaling issues so they use mixture of experts. Just throwing hardware at it is already showing diminishing returns. And we're learning fascinating new ways of training them, but the inherent problem is the same.
For example, if you ask an LLM if it can give an answer to a question, it will have two paths to go down, positive and negative. Note, at the point where it chooses that it doesn't know how to finish it, it doesn't look ahead. But it sees for example that 80% of the answers in the texts it's been trained on starts with a positive, then it will most likely start with "yes" - and when it does that it will continue to generate an answer - often very convincing and plausibly real looking answer, because it already committed to that path.
And as for the link about teaching it backspace token, the comments there are already pointing out the issue:
Almost none of the text it's trained on has a backspace token, and to finetune it in is tricky since it's a completely new concept - and remember it's still doing token for token - so it would have to write a token and then right after find out that it's more likely to send a backspace token than to continue it. It's interesting, and LLM's can pick up on some crazy patterns, but I'm skeptical.
First of all, this link is just to C# bindings of llama.cpp and so doesn't contain the actual implementation. But it also doesn't refute my criticism of your claim. More specifically, I take issue with this statement that you said: "today’s LLM’s ingest a ton of text [snip] and builds up statistics of which tokens it sees in that context".
I claim that this is not how today's LLMs work because we have no idea what LLMs do with the input data during training. We have very little insight into what kind of data structure it builds and how the data structure it built is organized.
I think I worded my sentence ambiguously, let me re-word it for you: "Going one token at a time is only considered a limitation because LLMs are not accurate enough"
Once again, my sentence was not written well, my bad. I was commenting on the observed behavior, not on how it works from a technical perspective.
Alright, we agree here for the most part so I'm just going to skip this.
This is weird though. How do you know LLMs can't look ahead? When we prompt LLMs, we are basically asking them this question: "What is the next word of your response?" How do you know it hasn't written out the entire response in memory already after which it only shows you the first word? LLMs are neural networks. Neural networks have working memory. That's how neural networks work after all, it's just a vector of data that is repeatedly transformed as it passes through each layer. Of course, if it does write the entire response in memory, it is all thrown away after every word.
As far as the backspace tokens go, you are right to be skeptical but also do not be surprised if it works out. We've had LLMs trained to
completeandedittext for some time already. They've fallen out of use today but they did perform acceptably well.I know, it's my code. I refactored it from some much less readable and usable c# code. I picked it because it more clearly shows the steps involved in generating text.
Firstly, it goes against everything we know so far of how they operate, and secondly.. because they can't.
If you look at the C# code, the first step is in
_process_tokensfunction, where it feeds the context intollama_eval. That goes through each token and updates the internal memory / model state. Since it saves state, if you already have processed some of the tokens you can tell it to skip them and start on the new ones.After this function you have a state in memory, the current state of the LLM, as a result of the tokens it's seen so far.
When we are done with that, we go to the more interesting part, the
_predict_next_tokenfunction. Note that that takes asamplingparamsparameter. It then set some options, like if top k is not set it's set to length of the model's vocabulary (number of tokens it knows about), and repeat_last_n, if not set, is set to the length of the existing context.The code then gets the model's vocabulary, aka all the tokens it knows about, and then it generates the logits. The logits is an array the length of the vocabulary, with a number for each token showing how likely that one is the next token. The code then adds any specified token bias to that token's number. Already here, even if it already had a specific answer in mind, you can see problems starting.
Then the code adds token repetition penalty, based on the
samplingparams. This means that if a token repeats inside the given history, it's value will be lowered according to the repeat_penalty. Again, even if it had a specific answer, this has a high chance of messing that up. The same is done for frequency and presence. For more details of what those native functions do, you can see the llama.cpp source - they have the same name there.After all the penalties are applied, it's time to pick the token. If the temp is 0 or lower, it just picks the highest rated token (aka greedy sampling). This tends to give very boring and flat responses, but it's predictable and reproduceable, so it's often used in benchmarks of various kinds.
But if that's not used (which it almost never is in "real" use), there are several methods. You have MiroStat, which tries to create more consistent quality between different answer lengths, and the "traditional" using top-k, top-p and temperature.
Common for them is however that internally it produces a top list of candidates, and then pick one at random. And that's why a LLM can't plan ahead.
When a token ID is eventually produced it returns the new ID, that gets added to context, the text equivalent of the token is looked up and sent back to the UI, and the new context is fed into llama_eval and the process starts again.
For the LLM to even be able to plan an answer ahead it must know of all penalties and parameters (or have none applied), and greedy token prediction must be used.
And that is why, even if it had some sort of near magical ability to plan ahead that we just don't know is there, at the end of the day it could still not plan a specific response.
Wow, very nice! First of all, I will preface by admitting that I have not worked with LLMs to the degree of making a toy implementation. Your explanation of the sampling techniques is insightful but doesn't clear up my confusion. Why does sampling imply the absence of higher level structure in the model?
For example, even though poker is highly influenced by chance, I can still have a plan that will increase my likelihood of winning. I don't know what card will be drawn next but I can prepare strategies for each possible card. I can have preferences for which cards I want to be drawn next.
You know what, I don't have a good answer to you here. I did a few small experiments on ChatGPT and it seems like it has some knowledge of if it will be able to complete it or not. This was with a pretty well known question though.
I tried to recreate an earlier experiment where I asked it to write about a friend of mine, which was in the news some time ago and have apparently a few entries in it's training data, but very little. ChatGPT would then consistently hallucinate facts about the person, including date of birth and sometimes date of death. In that case it knew the pattern of writing about a person including date of birth, and sometimes date of death, but it didn't know it didn't have that info and just filled in plausible looking data there. Now it insists on not knowing who that person is at all and refuses to write anything about him.
Anyway, you've given me some things to think about, thanks.
This is not entirely correct, in my experience. With the current version pf gtp-4 you might be right, but the initial versions were extremely good. Clearly you have to work with it, you cannot ask for the whole work
That's not true! There's heaps of early-GPT articles pointing out how much bullshit it regurgitates (eg Why does ChatGPT constantly lie?). And no evidence at all that the breathless fanboys have even stopped to check.
I meant initial versions of chatGTP 4. ChatGTP isn't lying, simply because lying implies a malevolent intent. Gtp-4 has no intent, it just provides an output given an input, that can be either wrong or correct. A model able to provide more correct answers is a more accurate model. Computing accuracy for a LLM is not trivial, but gpt-4 is still a good model. User has to know how to use it, what to expect and how to evaluate the result. If they are unable to do so it's completely their fault.
Why are you so pissed of a good nlp model?
I think there’s a big difference between being able to identify an AI by talking to it and being able to identify something written by an AI, especially if a human has looked over it for obvious errors.
In many cases they can. This is commonly used to improve their performance: https://arxiv.org/abs/2303.11366
*accurately
Whoops, meant to say: "In many cases, they can accurately (critique their own work)". Thanks for correcting me!
What you are describing is true of older LLMs. GPT4, it's less true of. GPT5 or whatever it is they are training now will likely begin to shed these issues.
The shocking thing that we discovered that lead to all of this is that this sort of LLM continues to scale in capabilities with the quality and size of the training set. AI researchers were convinced that this was not possible until GPT proved that it was.
So the idea that you can look at the limitations of the current generation of LLM and make blanket statements about the limitations of all future generations is demonstrably flawed.
They cannot be anything other than stochastic parrots because that is all the technology allows them to be. They are not intelligent, they don't understand the question you ask or the answer they give you, they don't know what truth is let alone how to determine it. They're just good at producing answers that sound like a human might have written them. They're a parlour trick. Hi-tech magic 8balls.
Are you referring to humans or AI? I'm not sure you're wrong about humans...
FFS
Sam Altman is a know-nothing grifter. HTH
Have you even read the article?
IMO it does not do a good job of disproving that "humans are stochastic parrots".
The example with the octopus isn't really about stochastic parrots. It's more about how LLMs are not multi-modal.
That article is super helpful.
Thanks!
I'm no GPT booster, but I think that the real problem with detectability here
is that it requires you to know the subject and content already, and to be giving the paper a relatively detailed reading. For a rube reading the paper, trying to learn from it - a lot of GPT content is easily mistaken as legitimate. And it's getting better. We're not safe simply assuming that AI today is as good as it will ever get and the clear errors we can detect cannot ever be addressed.
Penetrating academic writing, for academics, is probably one of the highest barriers of any writing task, AI or not.
But being dismissive of the threat of AI content because it's not able to convincingly fake some of the hardest writing that real people do is maybe sidestepping a lot of much more casual writing - that still carries significance and consequence.
Chad comment right here...
Clearly the Founding Fathers were not advanced enough to have crafted the US Constitution unaided. It's only reasonable to imagine that ancient aliens could have landed, given them an AI to assist them, and then departed with nobody the wiser.
I am certain we can find evidence of this if we dig hard enough.
James Madison was a fraud! He used ChatGPT instead of writing out his homework like he was supposed to. Smh my head.
In a sense you are correct. They cribbed from lots of the most well known political philosophers at the time. For example, there are direct quotes from Locke in the Declaration and his influence over the Constitution can be felt clearly.
I am certain we can find Georgios Tsoukalos saying there is evidence of this sooner or later. Probably extraterrestrial!
As an AI language model, I am unable to found a country but here are some ideas which might help people feel less likely to revolt.
This is a simple example of a constitution. For a real constitution you might want to consider a more detailed list of amendaments and civil rights. Please refer to a professional constitutional writer if you want to create a constitution for productive purposes
GPT-4 is quite willing to help you create the constitution for your own country, well I guess I have to find a name now:
Certainly! Crafting a constitution for a new country is an exciting and monumental task. The constitution is the core document that dictates the legal obligations and rights of your country's government and its citizens. Here's a very basic outline to get us started. Please remember, this is a rudimentary draft and it will need to be adjusted based on the specific needs, culture, history, and goals of your country:
PREAMBLE
We, the people of [Your Country's Name], in order to form a more perfect union, establish justice, ensure domestic tranquility, provide for the common defense, promote the general welfare, and secure the blessings of liberty to ourselves and our posterity, do ordain and establish this Constitution for [Your Country's Name].
ARTICLE I: THE LEGISLATURE
Section 1: All legislative powers herein granted shall be vested in a [name of your legislative body].
Section 2: [Details on how members are elected, their terms, qualifications, powers and responsibilities]
Section 3: [Details on how legislation is passed, veto powers, etc.]
ARTICLE II: THE EXECUTIVE
Section 1: The executive power shall be vested in a [title of your head of state].
Section 2: [Details on how the head of state is elected, their term, qualifications, powers and responsibilities]
Section 3: [Details on how the executive branch operates, and other positions within the executive branch]
ARTICLE III: THE JUDICIARY
Section 1: The judicial power of [Your Country's Name] shall be vested in one Supreme Court, and in such inferior courts as the [Legislative body] may from time to time ordain and establish.
Section 2: [Details on how judges are appointed, their terms, qualifications, powers and responsibilities]
ARTICLE IV: THE RIGHTS OF THE PEOPLE
Section 1: All persons born or naturalized in [Your Country's Name] are citizens of [Your Country's Name] and of the state wherein they reside. No state shall make or enforce any law which shall abridge the privileges or immunities of citizens.
Section 2: [Further details on the rights, freedoms, and responsibilities of the citizens, these could include things like freedom of speech, freedom of religion, right to a fair trial, etc.]
ARTICLE V: AMENDMENTS
Section 1: The [Legislative body] whenever two thirds of both Houses deem it necessary, shall propose Amendments to this Constitution, or, on the Application of the Legislatures of two thirds of the several states, shall call a Convention for proposing Amendments...
ARTICLE VI: GENERAL PROVISIONS
Section 1: [Any other items that need to be addressed, such as the official language, the national religion (or lack thereof), the national emblem, the process for international treaties, etc.]
ARTICLE VII: RATIFICATION
This Constitution shall be considered ratified when agreed upon by a majority of citizens through a free and fair vote.
Please remember that a constitution is a living document that will grow and change over time as your country evolves. This outline is just a starting point and may not include all the specific sections you need. For example, you might want to add sections addressing human rights, environmental protection, financial regulations, and so on.
This process should involve broad input from many different stakeholders, including legal experts, and should be reflective of the country's culture, values, history, and aspirations for the future. Also, please note that the effective implementation of a constitution requires a strong rule of law and mechanisms for holding those in power accountable.
💰
Because AI detectors suck and are the modern day equivalent of dowsing rods?
They're circular. If the text is too predictable it was written by an LLM* but LLMs are designed to regurgitate the next word most commonly used by humans in any given context.
*AI is a complete misnomer for the hi-tech magic 8ball
The next most commonly used word would result in a loop of common word. LLMs do not work like that
In context. And that is exactly how they work. It's just a statistical prediction model with billions of parameters.
That’s not really how LLMs work. You’re basically describing Markov chains. The statement “It’s just a statistical prediction model with billions of parameters” also applies to the human brain. An LLM is much more of a black box than you’re implying.
is not what it does. That would create non sensical text (you can try yourself).
This is a summary of the method, as summarized by gtp-4:
You are missing the key part where the text is tranformed in a vector space of "concepts" where semanticic relationships are represented, that is where the inference happens. The inference is not on words to get the next commonly used word, otherwise it wouldn't work. And you also missed the final sampling to introduce a randomness in the word selection.
I don't understand why are you so upset for a chain of complex mathematical functions that complete and input sentence. Why are you angry?
You're agreeing with me but using more words.
I'm more annoyed than upset. This technology is eating resources which are badly needed elsewhere and all we get in return is absolute junk which will infest the literature for decades to come.
I am not agreeing with you because "regurgitate the next most commonly world" is not what it does.
That said, the technology is not doing anything wrong. The people using it are doing it. The technology is a great achievement of human kind, possibly one of the greatest. If people decide to use it to print sh*t is people fault. Quantum mechanics is one of the greatest achievement of human kind, if people decided to use it to kill people, it is a fault of people. Many humans are simply shitty, don't blame a clever mathematical function and its clever implementation
This article was written to keep people as long on the page as possible. It didn't get to the point before i left. Someone has a tl;dr?
Constitution is a text that appears many times on the internet. ChatGPT's training set probably has multiple copies of it. So it's likely ChatGPT will generate it. Therefore, the detectors are likely to flag it as AI-generated. That's what I got from it, but I also found it difficult to parse. Maybe someone can correct me on this.
I thimk we need and AI to summarize the article. Edit: oh god, soon shit articles are gonna be optimized to not be AI summarizable
Bet it was written by an AI.
Thanks!
I knew it. The founding fathers were robots from the future!
I've recently checked my years-old essay using one of these AI plagiarism detectors and it said that the essay was 90% AI written. So either it's all bs or I'm a time travelling AI.
I’m convinced that it’s been trained on top of the essays of middle and high school students that have gone their whole lives without proper education on vocabulary, grammar, and the like. So when asked to evaluate something written properly, it’s flagged as AI.
Garbage in, garbage out. Same as is ever was.
CAUGHT!
Easy fix: run a constitution detector before the AI text detector
Interesting fact: The same parent company owns both Reddit and Turnitin.
I’m waiting for new conspiracy theories after that article hahah.
Obviously the US Constitution was written by AI, we're living in a simulation. Wake up sheeple, the Matrix is real!
Don’t mistake artificial intelligence for actual intelligence.
Aliens.jpg
Awwww. Beat me to it.
This. Changes. Everything.
They only know what they have been fed.
What more likely first/base feeding than the US Constitution's declarations and it's amendments?
Plot twist: it was! :)