Machine learning has some pretty cool potential in certain areas, especially in the medical field. Unfortunately the predominant use of it now is slop produced by copyright laundering shoved down our throats by every techbro hoping they'll be the next big thing.
It's marketing hype, even in the name. It isn't "AI" as decades of the actual AI field would define it, but credulous nerds really want their cyberpunkerino fantasies to come true so they buy into the hype label.
The term AI was coined in 1956 at a computer science conference and was used to refer to a broad range of topics that certainly would include machine learning and neural networks as used in large language models.
I don't get the "it's not really AI" point that keeps being brought up in discussions like this. Are you thinking of AGI, perhaps? That's the sci-fi "artificial person" variety, which LLMs aren't able to manage. But that's just a subset of AI.
Yeah, these are pattern reproduction engines. They can predict the most likely next thing in a sequence, whether that's words or pixels or numbers or whatever. There's nothing intelligent about it and this bubble is destined to pop.
That "Frightful Hobgoblin" computer toucher would insist otherwise, claiming that a sufficient number of Game Boys bolted together equals or even exceeds human sapience, but I think that user is currently too busy being a bigoted sex pest.
Both are happening. Samples of casual writing are more valuable to use to generate an article than research papers though.
Yeah. Scientific papers may teach an AI about science, but Reddit posts teach AI how to interact with people and "talk" to them. Both are valuable.
Hopefully not too pedantic, but no one is “teaching” AI anything. They’re just feeding it data in the hopes that it can learn probabilities for certain types of output. It “understands” neither the Reddit post nor the scientific paper.
This might be a wild take but people always make AI out to be way more primitive than it is.
Yes, in it's most basic for an LLM can be described as an auto-complete for conversations. But let's be real: the amount of different optimizations and adjustments made before and after the fact is pretty complex, and the way the AI works is pretty close already to a brain. Hell that's where we started out; emulating a brain. And you can look into this, the base for AI is usually neural networks, which learn to give specific parts of an input a specific amount of weight when generating the output. And when the output is not what we want, the AI slowly adjusts those weights to get closer.
Our brain works the same in it's most basic form. We use electric signals and we think associative patterns. When an electric signal enters one node, this node is connected via stronger or lighter bridges to different nodes, forming our associations. Those bridges is exactly what we emulate when we use nodes with weighted connectors in artificial neural networks.
Our AI output is quality wise right now pretty good, but integrity and security wise pretty bad (hallucinations, not following prompts, etc.), but saying it is performing at the level of a three year old is simultaneously under-selling and overselling how AI performs. We should be aware that just because it's AI doesn't mean it's good, but it also doesn't mean it's bad either. It just means there's a feature (which is hopefully optional) and then we can decide if it's helpful or not.
I do music production and I need cover art. As a student, I can't afford commissioning good artworks every now and then, so AI is the way to go and it's been nailing it.
As a software developer, I've come to appreciate that after about 2y of bad code completion AIs, there's finally one that is a net positive for me.
AI is just like anything else, it's a tool that brings change. How that change manifests depends on us as a collective. Let's punish bad AI, dangerous AI or similar (copilot, Tesla self driving, etc.) and let's promote good AI (Gmail text completion, chatgpt, code completion, image generators) and let's also realize that the best things we can get out of AI will not hit the ceiling of human products for a while. But if it costs too much, or you need quick pointers, at least you know where to start.
This shows so many gross misconceptions and with such utter conviction, I’m not even sure where to start. And as you seem to have decided you like to get free stuff that is the result of AI trained off the work of others without them receiving any compensation, nothing I say will likely change your opinion because you have an emotional stake in not acknowledging the problems of AI.
Describe how you ‘learned’ to speak. How do you know what word comes after the next. Until you can describe this process in a way that doesn’t make it ‘human’ or ‘biological’ only it’s no different. The only thing they can’t do is adjust their weights dynamically. But that’s a limitation we gave it not intrinsic to the system.
I inherited brain structures that are natural language processors. As well as the ability to understand and repeat any language sounds. Over time, my brain focused in on only the language sounds I heard the most and through trial and repetition learned how to understand and make those sounds.
AI - as it currently exists - is essentially a babbling infant with none of the structures necessary to do anything more than repeat sounds back without understanding any of them. Anyone who tells you different is selling you something.
Because AI needs a lot of training data to reliably generate something appropriate. It's easier to get millions of reddit posts than millions of research papers.
Even then, LLMs simply generate text but have no idea what the text means. It just knows those words have a high probability of matching the expected response. It doesn't check that what was generated is factual.
How do you know that's not what YOU'RE doing when you converse?
Because we have brains that are capable of critical thinking. It makes no sense to compare the human brain to the infancy and current inanity of LLMs.
I find it amusing that everyone is answering the question with the assumption that the premise of OP's question is correct. You're all hallucinating the same way that an LLM would.
LLMs are rarely trained on a single source of data exclusively. All the big ones you find will have been trained on a huge dataset including Reddit, research papers, books, letters, government documents, Wikipedia, GitHub, and much more.
Ignore facts, don’t do research to see if the comment/post is correct, don’t look at other comments to see if anyone else has corrected the post/comment already, there is only one right side (and that is the side of the loudest group)
When humans do it, it’s called “confabulation”
"AI" is a parlor trick. Very impressive at first, then you realize there isn't much to it that is actually meaningful. It regurgitates language patterns, patterns in images, etc. It can make a great Markov chain. But if you want to create an "AI" that just mines research papers, it will be unable to do useful things like synthesize information or describe the state of a research field. It is incapable of critical or analytical approaches. It will only be able to answer simple questions with dubious accuracy and to summarize texts (also with dubious accuracy).
Let's say you want to understand research on sugar and obesity using only a corpus from peer reviewed articles. You want to ask something like, "what is the relationship between sugar and obesity?". What will LLMs do when you ask this question? Well, they will just attempt to do associations and to construct reasonable-sounding sentences based on their set of research articles. They might even just take an actual semtence from an article and reframe it a little, just like a high schooler trying to get away with plagiarism. But they won't be able to actually mechanistically explain the overall mechanisms and will fall flat on their face when trying to discern nonsense funded by food lobbies from critical research. LLMs do not think or criticize. Of they do produce an answer that suggests controversy it will be because they either recognized diversity in the papers or, more likely, their corpus contains reviee articles that criticize articles funded by the food industry. But it will be unable to actually criticize the poor work or provide a summary of the relationship between sugar and obesity based on any actual understanding that questions, for example, whether this is even a valid question to ask in the first place (bodies are not simple!). It can only copy and mimic.
They might even just take an actual semtence from an article and reframe it a little
Case for many things that can be answered via stackoverflow searches. Even the order in which GPT-4o brings up points is the exact same as SO answers or comments.
Yeah it's actually one of the ways I caught a previous manager using AI for their own writing (things that should not have been done with AI). They were supposed to write about something in a hyper-specific field and an entire paragraph ended up just being a rewording of one of two (third party) website pages that discuss this topic directly.
Why does everyone keep calling them Markov chains? They're missing all the required properties, including the eponymous Markovian property. Wouldn't it be more correct to call them stochastic processes?
Edit: Correction, turns out the only difference between a stochastic process and a Markov process is the Markovian property. It's literally defined as "stochastic process but Markovian".
Because it's close enough. Turn off beam and redefine your state space and the property holds.
Why settle for good enough when you have a term that is both actually correct and more widely understood?
What term is that?
Stochastic process
But that's so vague. Molecules semi-randomly smashin into each other is a stochastic process
That's basically like saying that typical smartphones are square because it's close enough to rectangle and rectangle is too vague of a term. The point of more specific terms is to narrow down the set of possibilities. If you use "square" to mean the set of rectangles, then you lose the ability to do that and now both words are equally vague.
Is this referring to what I said about Markov chains or stochastic processes? If it's the former the only discriminating factor is beam and not all LLMs use that. If it's the latter then I don't know what you mean. Molecular dffusion is a classic stochastic process, I am 100% correct in my example.
It's in reference to your complaint about the imprecision of "stochastic process". I'm not disagreeing that molecular diffusion is a stochastic process. I'm saying that if you want to use "Markov process" to describe a non-Markovian stochastic process, then you no longer have the precision you're looking for and now molecular diffusion also falls under your new definition of Markov process.
Okay so both of those ideas are incorrect.
As I said, many are literally Markovian and the main discriminator is beam, which does not really matter for helping people understand my meaning nor should it confuse anyone that understands this topic. I will repeat: there are examples that are literally Markovian. In your example, it would be me saying there are rectangular phones but you step in to say, "but look those ones are curved! You should call it a shape, not a rectangle." I'm not really wrong and your point is a nitpick that makes communication worse.
In terms of stochastic processes, no, that is incredibly vague just like calling a phone a "shape" would not be more descriptive or communicate better. So many things follow stochastic processes that are nothing like a Markov chain, whereas LLMs are like Markov Chains, either literally being them or being a modified version that uses derived tree representations.
I'm not familiar with the term "beam" in the context of LLMs, so that's not factored into my argument in any way. LLMs generate text based on the history of tokens generated thus far, not just the last token. That is by definition non-Markovian. You can argue that an augmented state space would make it Markovian, but you can say that about any stochastic process. Once you start doing that, both become mathematically equivalent. Thinking about this a bit more, I don't think it really makes sense to talk about a process being Markovian or not without a wider context, so I'll let this one go.
nitpick that makes communication worse
How many readers do you think know what "Markov" means? How many would know what "stochastic" or "random" means? I'm willing to bet that the former is a strict subset of the latter.
Surely that is because we make it do that. We cripple it. Could we not unbound AI so that it genuinely weighed alternatives and made value choices? Write self-improvement algorithms?
It COULD help us. It WILL be smarter and faster than we are. We need to find ways to help it help us.
If AI is only a "parrot" as you say, then why should there be worries about extinction from AI?
You should look closer who is making those claims that "AI" is an extinction threat to humanity. It isn't researchers that look into ethics and safety (not to be confused with "AI safety" as part of "Alignment"). It is the people building the models and investors. Why are they building and investing in things that would kill us?
AI doomers try to 1. Make "AI"/LLMs appear way more powerful than they actually are. 2. Distract from actual threats and issues with LLMs/"AI". Because they are societal, ethical, about copyright and how it is not a trustworthy system at all. Cause admitting to those makes it a really hard sell.
If you look at the signatories (in the link) there are plenty of people who are not builders and investors, people who are in fact scientists in the field.
We cripple things by not programming the the abilities we obviously could give them.
We could have AI do an integrity check before printing an answer. No problem at all. We don't.
We could do many things to unbound the limitations AI has.
That's not how it works at all. If it were as easy as adding a line of code that says "check for integrity" they would've done that already. Fundamentally, the way these models all work is you give them some text and they try to guess the next word. It's ultra autocomplete. If you feed it "I'm going to the grocery store to get some" then it'll respond "food: 32%, bread: 15%, milk: 13%" and so on.
They get these results by crunching a ton of numbers, and those numbers, called a model, were tuned by training. During training, they collect every scrap of human text they can get their hands on, feed bits of it to the model, then see what the model guesses. They compare the model's guess to the actual text, tweak the numbers slightly to make the model more likely to give the right answer and less likely to give the wrong answers, then do it again with more text. The tweaking is an automated process, just feeding the model as much text as possible, until eventually it gets shockingly good at predicting. When training is done, the numbers stop getting tweaked, and it will give the same answer to the same prompt every time.
Once you have the model, you can use it to generate responses. Feed it something like "Question: why is the sky blue? Answer:" and if the model has gotten even remotely good at its job of predicting words, the next word should be the start of an answer to the question. Maybe the top prediction is "The". Well, that's not much, but you can tack one of the model's predicted words to the end and do it again. "Question: why is the sky blue? Answer: The" and see what it predicts. Keep repeating until you decide you have enough words, or maybe you've trained the model to also be able to predict "end of response" and use that to decide when to stop. You can play with this process, for example, making it more or less random. If you always take the top prediction you'll get perfectly consistent answers to the same prompt every time, but they'll be predictable and boring. You can instead pick based on the probabilities you get back from the model and get more variety. You can "increase the temperature" of that and intentionally choose unlikely answers more often than the model expects, which will make the response more varied but will eventually devolve into nonsense if you crank it up too high. Etc, etc. That's why even though the model is unchanging and gives the same word probabilities to the same input, you can get different answers in the text it gives back.
Note that there's nothing in here about accuracy, or sources, or thinking, or hallucinations, anything. The model doesn't know whether it's saying things that are real or fiction. It's literally a gigantic unchanging matrix of numbers. It's not even really "saying" things at all. It's just tossing out possible words, something else is picking from that list, and then the result is being fed back in for more words. To be clear, it's really good at this job, and can do some eerily human things, like mixing two concepts together, in a way that computers have never been able to do before. But it was never trained to reason, it wasn't trained to recognize that it's saying something untrue, or that it has little knowledge of a subject, or that it is saying something dangerous. It was trained to predict words.
At best, what they do with these things is prepend your questions with instructions, trying to guide the model to respond a certain way. So you'll type in "how do I make my own fireworks?" but the model will be given "You are a chatbot AI. You are polite and helpful, but you do not give dangerous advice. The user's question is: how do I make my own fireworks? Your answer:" and hopefully the instructions make the most likely answer something like "that's dangerous, I'm not discussing it." It's still not really thinking, though.
Surely that is because we make it do that. We cripple it. Could we not unbound AI so that it genuinely weighed alternatives and made value choices?
It's not that we cripple it, it's that the term "AI" has been used as a marketing term for generative models using LLMs and similar technology. The mimicry is inherent to how these models function, they are all about patterns.
A good example is "hallucinations" with LLMs. When the models give wrong answers because they appear to be making things up. Really, they are incapable of differentiating, they're just producing sophisticated patterns from a very large models. There is no real underlying conceptualization or notion of true answers, only answers that are often true when the training material was true and the model captured the patterns and they were highly weighted. The hot topic for thevlast year has just been to augment these models with a more specific corpus, pike a company database, for a given application so that it is more biased towards relevant things.
This is also why these models are bad at basic math.
So the fundamental problem here is companies calling this AI as if reasoning is occurring. It is useful for marketing because they want to sell the idea that this can replace workers but it usually can't. So you get funny situations like chatbots at airlines that offer money to people without there being any company policy to do so.
There are a lot of very intelligent academics and technical experts that have completely unrealistic ideas of what is an actual real-world threat. For example, I know one that worked on military drones, the kind that drop bombs on kids, that was worried about right wing grifters getting protested at a college campus like it was the end of the world. Not his material contribution to military domination and instability but whether a racist he clearly sympathized with would have to see some protest signs.
That petition seems to be based on the ones against nuclear proliferation from the 80s. They could be simple because nuclear war was obviously a substantial threat. It still is but there is no propaganda fear campaign to keep the concern alive. For AI, it is in no way obvious what threat they are talking about.
I have persobal concepts of AI threats. Having ridiculously high energy requirements compared to their utility when energy is still a major contributor to climate change. The potential for it to kill knowledge bases, like how it is making search engines garbage with a flood of nonsense websites. Enclosure of creative works and production by some monopoly "AU" companies. They are already suing others based on IP infringement when their models are all based on it! But I can't tell if this petition is about that at all, it doesn't explain. Maybe they're thinking of a Terminator scenario, which is absurd.
It COULD help us. It WILL be smarter and faster than we are. We need to find ways to help it help us.
Technology is both a reflection and determinent of social relations. As we can see with this round if "AI", it is largely vaporware that has not helped much with productivity but is nevertheless very appealing to businesses that feel they need to get on the hype train or be left behind. What they really want to do is have a smaller workforce so they can make more money that they can then use to make more money etc etc. For example, plenty of people use "AI" to generate questionably appealing graphics for their websites rather than paying an artist. So we can see that " A" tech is a solution searching for a problem, that its actual use cases are about profit over real utility, and that this is not the fault of the technology, but how we currently organize society: not for people, but for profit.
So yes, of course, real AI could be very helpful! How nice would it be to let computers do the boring work and then enjoy the fruits of huge productivity increases? The real risk is not the technology, it is our social relations, who has power, and how technology is used. Is making the production of art a less viable career path an advancement? Is it helping people overall? What are the graphic designers displaced by what is basically an infinite pile of same-y stock images going to do now? They still have to have jobs to live. The fruits of "AI" removing much of their job market hasn't really been shared equally, nor has it meant an early retirement. This is because the fundamental economic system remains in place and it cannot survive without forcing people to do jobs.
You could feed all the research papers in the world to an LLM and it will still have zero understanding of what you trained it on. It will still make shit up, it can't save the world.
Training it on research papers wouldn’t make it smarter, it would just make it better at mimicking their writing style.
Don’t fall for the hype.
Short answer: they already are
Slightly longer answer: GPT models like ChatGPT are part of an experiment in "if we train the AI model on shedloads of data does it make a more powerful AI model?" and after OpenAI made such big waves every company is copying them including trying to train models similar to ChatGPT rather than trying to innovate and do more
Even longer answer: There's tons of different AI models out there for doing tons of different things. Just look at the over 1 million models on Hugging Face (a company which operates as a repository for AI models among other services) and look at all of the different types of models you can filter for on the left.
Training an image generation model on research papers probably would make it a lot worse at generating pictures of cats, but training a model that you want to either generate or process research papers on existing research papers would probably make a very high quality model for either goal.
More to your point, there's some neat very targeted models with smaller training sets out there like Microsoft's PHI-3 model which is primarily trained on textbooks
As for saving the world, I'm curious what you mean by that exactly? These generative text models are great at generating text similar to their training data, and summarization models are great at summarizing text. But ultimately AI isn't going to save the world. Once the current hype cycle dies down AI will be a better known and more widely used technology, but ultimately its just a tool in the toolbox.
also the answer to that question, shitloads of data for a better ai, is yes… with logarithmic returns. massively underpriced (by cost to generate) returns that have questionable value statement at best.
Who is "we"? My understanding is LLMs are mostly being trained on a large amount of publicly available texts, including both reddit posts and research papers.
I bet the first and only research paper fed is that infamous one about vaccines and autism
editor's note: it will not save the world
They already do that. You're being a troglodyte.
Hmmm. Not sure if I'm being insulted. Is that one of those fish fossils that looks kind of like a horseshoe crab?
You're thinking of a trilobite
Dictionary
Definitions from Oxford Languages · Learn more
noun
(especially in prehistoric times) a person who lived in a cave.
a hermit.
a person who is regarded as being deliberately ignorant or old-fashioned.
Still not getting you. Like, you're angry?
Papers are most importantly a documentation of exactly what and how a procedure was performed, adding a vagueness filter over that is only going to decrease its value infinitely.
Real question is why are we using generative ai at all (gets money out of idiot rich people)
They're trained on both, and the kitchen sink.
The Ghost of Aaron Schwartz
What he was fighting for was an awful lot more important than a tool to write your emails while causing a ginormous tech bubble.
They are. T&F recently cut a deal with Microsoft. Without author's consent, of course.
I'm fairly sure a few others have too, but that's the only article I could find quickly.
Anyone running a webserver and looking at their logs will know AI is being trained on EVERYTHING. There are so many crawlers for AI that are literally ripping the internet wholesale. Reddit just got in on charging the AI companies for access to freely contributed content. For everyone else, they're just outright stealing it.
We are. I just read an article yesterday about how Microsoft paid research publishers so they could use the papers to train AI, with or without the consent of the papers' authors. The publishers also reduced the peer review window so they could publish papers faster and get more money from Microsoft. So... expect AI to be trained on a lot of sloppy, poorly-reviewed research papers because of corporate greed.
Nobody wants an AI that talks like that.
I kind of think my question is WHY ARE WE FOCUSING ON TALKING TO IT?
Because "ai" ad we colloquially know today are language models: they train on and can produce language, that's what they are designed on. Yes, they can produce images and also videos, but they don't have any form of real knowledge or understanding, they only predict the next word or the next pixel based on their prompt and their vast examples of words and images. You can only talk to them because that's what they are for.
Feeding research papers will make it spit research-sounding words, which probably will contain some correct information, but at best an llm trained on that would be useful to search through existing research, it would not be able to make new one
Because that's what it's designed for? I'm curious what else it could be good for. A machine capable of independent, intelligent research sounds like a totally different invention entirely.
It's sort of like the communication aspect of it isn't the sole purpose of it. It's as if we invented computers but the only thing we cared about was the monitor and the keyboard.
We want it to DO things. Stick to the truth, not just placate.
Didn't realize that. The only applications I've seen for it are conversation or generating media based on text input. I thought all it did was analyze text and create a response based on patterns it had observed.
I haven't done much with it myself though so that's probably a very limited POV.
I saw an article about one trained on research papers. (Built by Meta, maybe?) It also spewed out garbage: it would make up answers that mimicked the style of the papers but had its own fabricated content! Something about the largest nuclear reactor made of cheese in the world...
Because they are looking for conversations.
Brain damage is cheaper than professionals
Part of it is the same "human speech" aspects that have plagued NLP work over the past few years. Nobody (except the poor postdoctoral bastard who is running the paper farm for their boss) actually speaks in the same way that scholarly articles are written because... that should be obvious.
This combines with the decades of work by right wing fascists to vilify intellectuals and academia. If you have ever seen (or written) a comment that boils down to "This youtuber sounds smug" or "They are presenting their opinion as fact" then you see why people prefer "natural human speech" over actual authoritatively researched and tested statements.
And... while not all pay to publish journals are trash, I feel confident saying that most are. And filtering those can be shockingly hard by design.
But the big one? Most of the owners of the various journals are REALLY fucking litigious and will go scorched earth on anyone who is using their work (because Elsevier et al own your work) to train a model.
Saving the world isn't profitable in the short term.
Vulture capitalists don't care about the future. They care about the immediate. Short term profitability. And nothing else.
How does that help disempower the fossil fuel mafia?
Who's going to peer review that?
Tons of people already are. The following site is useful for searching papers using ai https://consensus.app/
Thank you! That was thoughtful
Because broken English from research paper and relatively structured style will be even worse than reddit posts
Came to wonder about this.
The few I've seen weren't shining examples of the language, and could have used some editing.
As well, the rumours abound that a lot of papers are available before review, and that's likely to cause some harm if we trust a model predicting on bad data.
(Yes, I know: reddit isn't going to be better; but it has its own warning because, well, Reddit)
I think I read this post wrong.
I was thinking the sentence "We could be saving the world!" meant 'we' as in humans only.
No need to be training AI. No need to do anything with AI at all. Humans simply start saving the world. Our Research Papers can train on Reddit. We cannot be training, we are saving the world. Let the Research Papers run a train on Reddit AI. Humanity Saves World.
No cynical replies please.
money. theres no money in saving the world. lots of money in not saving the world.
greed will be humanities downfall
AuroraGPT. They are trying to do it.
Its cause number of people who can read, understand, and then create the necessary dataset to train and test the LLM are very very very few for research papers vs the data for pop culture is easilier to source.
Why are we training kids on civics with Fox News or MSNBC? People are dumb and will continue to be so.
Who says they’re not?
Most research papers are likely ad valid as an average reddit point.
Getting published is a circlejerk, and rarely are they properly tested, or does anyone actually read them.
Those research papers are expensive to procure ethically, I'd imagine
This is a damn good question!
It won't worm. Autocomplete can't make new information.
Because scientific journals are paywalled - gibberish on Reddit is free*.
AI isn't saving the world lol
Machine learning has some pretty cool potential in certain areas, especially in the medical field. Unfortunately the predominant use of it now is slop produced by copyright laundering shoved down our throats by every techbro hoping they'll be the next big thing.
It's marketing hype, even in the name. It isn't "AI" as decades of the actual AI field would define it, but credulous nerds really want their cyberpunkerino fantasies to come true so they buy into the hype label.
The term AI was coined in 1956 at a computer science conference and was used to refer to a broad range of topics that certainly would include machine learning and neural networks as used in large language models.
I don't get the "it's not really AI" point that keeps being brought up in discussions like this. Are you thinking of AGI, perhaps? That's the sci-fi "artificial person" variety, which LLMs aren't able to manage. But that's just a subset of AI.
Yeah, these are pattern reproduction engines. They can predict the most likely next thing in a sequence, whether that's words or pixels or numbers or whatever. There's nothing intelligent about it and this bubble is destined to pop.
That "Frightful Hobgoblin" computer toucher would insist otherwise, claiming that a sufficient number of Game Boys bolted together equals or even exceeds human sapience, but I think that user is currently too busy being a bigoted sex pest.
Both are happening. Samples of casual writing are more valuable to use to generate an article than research papers though.
Yeah. Scientific papers may teach an AI about science, but Reddit posts teach AI how to interact with people and "talk" to them. Both are valuable.
Hopefully not too pedantic, but no one is “teaching” AI anything. They’re just feeding it data in the hopes that it can learn probabilities for certain types of output. It “understands” neither the Reddit post nor the scientific paper.
This might be a wild take but people always make AI out to be way more primitive than it is.
Yes, in it's most basic for an LLM can be described as an auto-complete for conversations. But let's be real: the amount of different optimizations and adjustments made before and after the fact is pretty complex, and the way the AI works is pretty close already to a brain. Hell that's where we started out; emulating a brain. And you can look into this, the base for AI is usually neural networks, which learn to give specific parts of an input a specific amount of weight when generating the output. And when the output is not what we want, the AI slowly adjusts those weights to get closer.
Our brain works the same in it's most basic form. We use electric signals and we think associative patterns. When an electric signal enters one node, this node is connected via stronger or lighter bridges to different nodes, forming our associations. Those bridges is exactly what we emulate when we use nodes with weighted connectors in artificial neural networks.
Our AI output is quality wise right now pretty good, but integrity and security wise pretty bad (hallucinations, not following prompts, etc.), but saying it is performing at the level of a three year old is simultaneously under-selling and overselling how AI performs. We should be aware that just because it's AI doesn't mean it's good, but it also doesn't mean it's bad either. It just means there's a feature (which is hopefully optional) and then we can decide if it's helpful or not.
I do music production and I need cover art. As a student, I can't afford commissioning good artworks every now and then, so AI is the way to go and it's been nailing it.
As a software developer, I've come to appreciate that after about 2y of bad code completion AIs, there's finally one that is a net positive for me.
AI is just like anything else, it's a tool that brings change. How that change manifests depends on us as a collective. Let's punish bad AI, dangerous AI or similar (copilot, Tesla self driving, etc.) and let's promote good AI (Gmail text completion, chatgpt, code completion, image generators) and let's also realize that the best things we can get out of AI will not hit the ceiling of human products for a while. But if it costs too much, or you need quick pointers, at least you know where to start.
This shows so many gross misconceptions and with such utter conviction, I’m not even sure where to start. And as you seem to have decided you like to get free stuff that is the result of AI trained off the work of others without them receiving any compensation, nothing I say will likely change your opinion because you have an emotional stake in not acknowledging the problems of AI.
Describe how you ‘learned’ to speak. How do you know what word comes after the next. Until you can describe this process in a way that doesn’t make it ‘human’ or ‘biological’ only it’s no different. The only thing they can’t do is adjust their weights dynamically. But that’s a limitation we gave it not intrinsic to the system.
I inherited brain structures that are natural language processors. As well as the ability to understand and repeat any language sounds. Over time, my brain focused in on only the language sounds I heard the most and through trial and repetition learned how to understand and make those sounds.
AI - as it currently exists - is essentially a babbling infant with none of the structures necessary to do anything more than repeat sounds back without understanding any of them. Anyone who tells you different is selling you something.
Because AI needs a lot of training data to reliably generate something appropriate. It's easier to get millions of reddit posts than millions of research papers.
Even then, LLMs simply generate text but have no idea what the text means. It just knows those words have a high probability of matching the expected response. It doesn't check that what was generated is factual.
How do you know that's not what YOU'RE doing when you converse?
Because we have brains that are capable of critical thinking. It makes no sense to compare the human brain to the infancy and current inanity of LLMs.
I find it amusing that everyone is answering the question with the assumption that the premise of OP's question is correct. You're all hallucinating the same way that an LLM would.
LLMs are rarely trained on a single source of data exclusively. All the big ones you find will have been trained on a huge dataset including Reddit, research papers, books, letters, government documents, Wikipedia, GitHub, and much more.
Example datasets:
Rules of lemmy
Ignore facts, don’t do research to see if the comment/post is correct, don’t look at other comments to see if anyone else has corrected the post/comment already, there is only one right side (and that is the side of the loudest group)
When humans do it, it’s called “confabulation”
"AI" is a parlor trick. Very impressive at first, then you realize there isn't much to it that is actually meaningful. It regurgitates language patterns, patterns in images, etc. It can make a great Markov chain. But if you want to create an "AI" that just mines research papers, it will be unable to do useful things like synthesize information or describe the state of a research field. It is incapable of critical or analytical approaches. It will only be able to answer simple questions with dubious accuracy and to summarize texts (also with dubious accuracy).
Let's say you want to understand research on sugar and obesity using only a corpus from peer reviewed articles. You want to ask something like, "what is the relationship between sugar and obesity?". What will LLMs do when you ask this question? Well, they will just attempt to do associations and to construct reasonable-sounding sentences based on their set of research articles. They might even just take an actual semtence from an article and reframe it a little, just like a high schooler trying to get away with plagiarism. But they won't be able to actually mechanistically explain the overall mechanisms and will fall flat on their face when trying to discern nonsense funded by food lobbies from critical research. LLMs do not think or criticize. Of they do produce an answer that suggests controversy it will be because they either recognized diversity in the papers or, more likely, their corpus contains reviee articles that criticize articles funded by the food industry. But it will be unable to actually criticize the poor work or provide a summary of the relationship between sugar and obesity based on any actual understanding that questions, for example, whether this is even a valid question to ask in the first place (bodies are not simple!). It can only copy and mimic.
Case for many things that can be answered via stackoverflow searches. Even the order in which GPT-4o brings up points is the exact same as SO answers or comments.
Yeah it's actually one of the ways I caught a previous manager using AI for their own writing (things that should not have been done with AI). They were supposed to write about something in a hyper-specific field and an entire paragraph ended up just being a rewording of one of two (third party) website pages that discuss this topic directly.
Why does everyone keep calling them Markov chains? They're missing
all the required properties, includingthe eponymous Markovian property. Wouldn't it be more correct to call them stochastic processes?Edit: Correction, turns out the only difference between a stochastic process and a Markov process is the Markovian property. It's literally defined as "stochastic process but Markovian".
Because it's close enough. Turn off beam and redefine your state space and the property holds.
Why settle for good enough when you have a term that is both actually correct and more widely understood?
What term is that?
Stochastic process
But that's so vague. Molecules semi-randomly smashin into each other is a stochastic process
That's basically like saying that typical smartphones are square because it's close enough to rectangle and rectangle is too vague of a term. The point of more specific terms is to narrow down the set of possibilities. If you use "square" to mean the set of rectangles, then you lose the ability to do that and now both words are equally vague.
Is this referring to what I said about Markov chains or stochastic processes? If it's the former the only discriminating factor is beam and not all LLMs use that. If it's the latter then I don't know what you mean. Molecular dffusion is a classic stochastic process, I am 100% correct in my example.
It's in reference to your complaint about the imprecision of "stochastic process". I'm not disagreeing that molecular diffusion is a stochastic process. I'm saying that if you want to use "Markov process" to describe a non-Markovian stochastic process, then you no longer have the precision you're looking for and now molecular diffusion also falls under your new definition of Markov process.
Okay so both of those ideas are incorrect.
As I said, many are literally Markovian and the main discriminator is beam, which does not really matter for helping people understand my meaning nor should it confuse anyone that understands this topic. I will repeat: there are examples that are literally Markovian. In your example, it would be me saying there are rectangular phones but you step in to say, "but look those ones are curved! You should call it a shape, not a rectangle." I'm not really wrong and your point is a nitpick that makes communication worse.
In terms of stochastic processes, no, that is incredibly vague just like calling a phone a "shape" would not be more descriptive or communicate better. So many things follow stochastic processes that are nothing like a Markov chain, whereas LLMs are like Markov Chains, either literally being them or being a modified version that uses derived tree representations.
I'm not familiar with the term "beam" in the context of LLMs, so that's not factored into my argument in any way. LLMs generate text based on the history of tokens generated thus far, not just the last token. That is by definition non-Markovian. You can argue that an augmented state space would make it Markovian, but you can say that about any stochastic process. Once you start doing that, both become mathematically equivalent. Thinking about this a bit more, I don't think it really makes sense to talk about a process being Markovian or not without a wider context, so I'll let this one go.
How many readers do you think know what "Markov" means? How many would know what "stochastic" or "random" means? I'm willing to bet that the former is a strict subset of the latter.
Surely that is because we make it do that. We cripple it. Could we not unbound AI so that it genuinely weighed alternatives and made value choices? Write self-improvement algorithms?
If AI is only a "parrot" as you say, then why should there be worries about extinction from AI? https://www.safe.ai/work/statement-on-ai-risk#open-letter
It COULD help us. It WILL be smarter and faster than we are. We need to find ways to help it help us.
You should look closer who is making those claims that "AI" is an extinction threat to humanity. It isn't researchers that look into ethics and safety (not to be confused with "AI safety" as part of "Alignment"). It is the people building the models and investors. Why are they building and investing in things that would kill us?
AI doomers try to 1. Make "AI"/LLMs appear way more powerful than they actually are. 2. Distract from actual threats and issues with LLMs/"AI". Because they are societal, ethical, about copyright and how it is not a trustworthy system at all. Cause admitting to those makes it a really hard sell.
If you look at the signatories (in the link) there are plenty of people who are not builders and investors, people who are in fact scientists in the field.
We cripple things by not programming the the abilities we obviously could give them.
We could have AI do an integrity check before printing an answer. No problem at all. We don't.
We could do many things to unbound the limitations AI has.
That's not how it works at all. If it were as easy as adding a line of code that says "check for integrity" they would've done that already. Fundamentally, the way these models all work is you give them some text and they try to guess the next word. It's ultra autocomplete. If you feed it "I'm going to the grocery store to get some" then it'll respond "food: 32%, bread: 15%, milk: 13%" and so on.
They get these results by crunching a ton of numbers, and those numbers, called a model, were tuned by training. During training, they collect every scrap of human text they can get their hands on, feed bits of it to the model, then see what the model guesses. They compare the model's guess to the actual text, tweak the numbers slightly to make the model more likely to give the right answer and less likely to give the wrong answers, then do it again with more text. The tweaking is an automated process, just feeding the model as much text as possible, until eventually it gets shockingly good at predicting. When training is done, the numbers stop getting tweaked, and it will give the same answer to the same prompt every time.
Once you have the model, you can use it to generate responses. Feed it something like "Question: why is the sky blue? Answer:" and if the model has gotten even remotely good at its job of predicting words, the next word should be the start of an answer to the question. Maybe the top prediction is "The". Well, that's not much, but you can tack one of the model's predicted words to the end and do it again. "Question: why is the sky blue? Answer: The" and see what it predicts. Keep repeating until you decide you have enough words, or maybe you've trained the model to also be able to predict "end of response" and use that to decide when to stop. You can play with this process, for example, making it more or less random. If you always take the top prediction you'll get perfectly consistent answers to the same prompt every time, but they'll be predictable and boring. You can instead pick based on the probabilities you get back from the model and get more variety. You can "increase the temperature" of that and intentionally choose unlikely answers more often than the model expects, which will make the response more varied but will eventually devolve into nonsense if you crank it up too high. Etc, etc. That's why even though the model is unchanging and gives the same word probabilities to the same input, you can get different answers in the text it gives back.
Note that there's nothing in here about accuracy, or sources, or thinking, or hallucinations, anything. The model doesn't know whether it's saying things that are real or fiction. It's literally a gigantic unchanging matrix of numbers. It's not even really "saying" things at all. It's just tossing out possible words, something else is picking from that list, and then the result is being fed back in for more words. To be clear, it's really good at this job, and can do some eerily human things, like mixing two concepts together, in a way that computers have never been able to do before. But it was never trained to reason, it wasn't trained to recognize that it's saying something untrue, or that it has little knowledge of a subject, or that it is saying something dangerous. It was trained to predict words.
At best, what they do with these things is prepend your questions with instructions, trying to guide the model to respond a certain way. So you'll type in "how do I make my own fireworks?" but the model will be given "You are a chatbot AI. You are polite and helpful, but you do not give dangerous advice. The user's question is: how do I make my own fireworks? Your answer:" and hopefully the instructions make the most likely answer something like "that's dangerous, I'm not discussing it." It's still not really thinking, though.
It's not that we cripple it, it's that the term "AI" has been used as a marketing term for generative models using LLMs and similar technology. The mimicry is inherent to how these models function, they are all about patterns.
A good example is "hallucinations" with LLMs. When the models give wrong answers because they appear to be making things up. Really, they are incapable of differentiating, they're just producing sophisticated patterns from a very large models. There is no real underlying conceptualization or notion of true answers, only answers that are often true when the training material was true and the model captured the patterns and they were highly weighted. The hot topic for thevlast year has just been to augment these models with a more specific corpus, pike a company database, for a given application so that it is more biased towards relevant things.
This is also why these models are bad at basic math.
So the fundamental problem here is companies calling this AI as if reasoning is occurring. It is useful for marketing because they want to sell the idea that this can replace workers but it usually can't. So you get funny situations like chatbots at airlines that offer money to people without there being any company policy to do so.
There are a lot of very intelligent academics and technical experts that have completely unrealistic ideas of what is an actual real-world threat. For example, I know one that worked on military drones, the kind that drop bombs on kids, that was worried about right wing grifters getting protested at a college campus like it was the end of the world. Not his material contribution to military domination and instability but whether a racist he clearly sympathized with would have to see some protest signs.
That petition seems to be based on the ones against nuclear proliferation from the 80s. They could be simple because nuclear war was obviously a substantial threat. It still is but there is no propaganda fear campaign to keep the concern alive. For AI, it is in no way obvious what threat they are talking about.
I have persobal concepts of AI threats. Having ridiculously high energy requirements compared to their utility when energy is still a major contributor to climate change. The potential for it to kill knowledge bases, like how it is making search engines garbage with a flood of nonsense websites. Enclosure of creative works and production by some monopoly "AU" companies. They are already suing others based on IP infringement when their models are all based on it! But I can't tell if this petition is about that at all, it doesn't explain. Maybe they're thinking of a Terminator scenario, which is absurd.
Technology is both a reflection and determinent of social relations. As we can see with this round if "AI", it is largely vaporware that has not helped much with productivity but is nevertheless very appealing to businesses that feel they need to get on the hype train or be left behind. What they really want to do is have a smaller workforce so they can make more money that they can then use to make more money etc etc. For example, plenty of people use "AI" to generate questionably appealing graphics for their websites rather than paying an artist. So we can see that " A" tech is a solution searching for a problem, that its actual use cases are about profit over real utility, and that this is not the fault of the technology, but how we currently organize society: not for people, but for profit.
So yes, of course, real AI could be very helpful! How nice would it be to let computers do the boring work and then enjoy the fruits of huge productivity increases? The real risk is not the technology, it is our social relations, who has power, and how technology is used. Is making the production of art a less viable career path an advancement? Is it helping people overall? What are the graphic designers displaced by what is basically an infinite pile of same-y stock images going to do now? They still have to have jobs to live. The fruits of "AI" removing much of their job market hasn't really been shared equally, nor has it meant an early retirement. This is because the fundamental economic system remains in place and it cannot survive without forcing people to do jobs.
You could feed all the research papers in the world to an LLM and it will still have zero understanding of what you trained it on. It will still make shit up, it can't save the world.
Training it on research papers wouldn’t make it smarter, it would just make it better at mimicking their writing style.
Don’t fall for the hype.
Short answer: they already are
Slightly longer answer: GPT models like ChatGPT are part of an experiment in "if we train the AI model on shedloads of data does it make a more powerful AI model?" and after OpenAI made such big waves every company is copying them including trying to train models similar to ChatGPT rather than trying to innovate and do more
Even longer answer: There's tons of different AI models out there for doing tons of different things. Just look at the over 1 million models on Hugging Face (a company which operates as a repository for AI models among other services) and look at all of the different types of models you can filter for on the left.
Training an image generation model on research papers probably would make it a lot worse at generating pictures of cats, but training a model that you want to either generate or process research papers on existing research papers would probably make a very high quality model for either goal.
More to your point, there's some neat very targeted models with smaller training sets out there like Microsoft's PHI-3 model which is primarily trained on textbooks
As for saving the world, I'm curious what you mean by that exactly? These generative text models are great at generating text similar to their training data, and summarization models are great at summarizing text. But ultimately AI isn't going to save the world. Once the current hype cycle dies down AI will be a better known and more widely used technology, but ultimately its just a tool in the toolbox.
also the answer to that question, shitloads of data for a better ai, is yes… with logarithmic returns. massively underpriced (by cost to generate) returns that have questionable value statement at best.
How are the “returns” measured numerically here?
Hillusionations per GWH iirc.
Redditors are always right, peer reviewed papers always wrong. Pretty obvious really. :D
Dank memes > science
Who is "we"? My understanding is LLMs are mostly being trained on a large amount of publicly available texts, including both reddit posts and research papers.
I bet the first and only research paper fed is that infamous one about vaccines and autism
editor's note: it will not save the world
They already do that. You're being a troglodyte.
Hmmm. Not sure if I'm being insulted. Is that one of those fish fossils that looks kind of like a horseshoe crab?
You're thinking of a trilobite
Dictionary Definitions from Oxford Languages · Learn more noun (especially in prehistoric times) a person who lived in a cave. a hermit. a person who is regarded as being deliberately ignorant or old-fashioned.
Still not getting you. Like, you're angry?
Papers are most importantly a documentation of exactly what and how a procedure was performed, adding a vagueness filter over that is only going to decrease its value infinitely.
Real question is why are we using generative ai at all (gets money out of idiot rich people)
They're trained on both, and the kitchen sink.
The Ghost of Aaron Schwartz
What he was fighting for was an awful lot more important than a tool to write your emails while causing a ginormous tech bubble.
They're trained on technical material too.
They are. T&F recently cut a deal with Microsoft. Without author's consent, of course.
I'm fairly sure a few others have too, but that's the only article I could find quickly.
Anyone running a webserver and looking at their logs will know AI is being trained on EVERYTHING. There are so many crawlers for AI that are literally ripping the internet wholesale. Reddit just got in on charging the AI companies for access to freely contributed content. For everyone else, they're just outright stealing it.
We are. I just read an article yesterday about how Microsoft paid research publishers so they could use the papers to train AI, with or without the consent of the papers' authors. The publishers also reduced the peer review window so they could publish papers faster and get more money from Microsoft. So... expect AI to be trained on a lot of sloppy, poorly-reviewed research papers because of corporate greed.
Nobody wants an AI that talks like that.
I kind of think my question is WHY ARE WE FOCUSING ON TALKING TO IT?
Because "ai" ad we colloquially know today are language models: they train on and can produce language, that's what they are designed on. Yes, they can produce images and also videos, but they don't have any form of real knowledge or understanding, they only predict the next word or the next pixel based on their prompt and their vast examples of words and images. You can only talk to them because that's what they are for.
Feeding research papers will make it spit research-sounding words, which probably will contain some correct information, but at best an llm trained on that would be useful to search through existing research, it would not be able to make new one
Because that's what it's designed for? I'm curious what else it could be good for. A machine capable of independent, intelligent research sounds like a totally different invention entirely.
It's sort of like the communication aspect of it isn't the sole purpose of it. It's as if we invented computers but the only thing we cared about was the monitor and the keyboard.
We want it to DO things. Stick to the truth, not just placate.
Didn't realize that. The only applications I've seen for it are conversation or generating media based on text input. I thought all it did was analyze text and create a response based on patterns it had observed.
I haven't done much with it myself though so that's probably a very limited POV.
I saw an article about one trained on research papers. (Built by Meta, maybe?) It also spewed out garbage: it would make up answers that mimicked the style of the papers but had its own fabricated content! Something about the largest nuclear reactor made of cheese in the world...
Because they are looking for conversations.
Brain damage is cheaper than professionals
Part of it is the same "human speech" aspects that have plagued NLP work over the past few years. Nobody (except the poor postdoctoral bastard who is running the paper farm for their boss) actually speaks in the same way that scholarly articles are written because... that should be obvious.
This combines with the decades of work by right wing fascists to vilify intellectuals and academia. If you have ever seen (or written) a comment that boils down to "This youtuber sounds smug" or "They are presenting their opinion as fact" then you see why people prefer "natural human speech" over actual authoritatively researched and tested statements.
And... while not all pay to publish journals are trash, I feel confident saying that most are. And filtering those can be shockingly hard by design.
But the big one? Most of the owners of the various journals are REALLY fucking litigious and will go scorched earth on anyone who is using their work (because Elsevier et al own your work) to train a model.
Saving the world isn't profitable in the short term.
Vulture capitalists don't care about the future. They care about the immediate. Short term profitability. And nothing else.
How does that help disempower the fossil fuel mafia?
Who's going to peer review that?
Tons of people already are. The following site is useful for searching papers using ai https://consensus.app/
Thank you! That was thoughtful
Because broken English from research paper and relatively structured style will be even worse than reddit posts
Came to wonder about this.
The few I've seen weren't shining examples of the language, and could have used some editing.
As well, the rumours abound that a lot of papers are available before review, and that's likely to cause some harm if we trust a model predicting on bad data.
(Yes, I know: reddit isn't going to be better; but it has its own warning because, well, Reddit)
I think I read this post wrong.
I was thinking the sentence "We could be saving the world!" meant 'we' as in humans only.
No need to be training AI. No need to do anything with AI at all. Humans simply start saving the world. Our Research Papers can train on Reddit. We cannot be training, we are saving the world. Let the Research Papers run a train on Reddit AI. Humanity Saves World.
No cynical replies please.
money. theres no money in saving the world. lots of money in not saving the world.
greed will be humanities downfall
AuroraGPT. They are trying to do it.
Its cause number of people who can read, understand, and then create the necessary dataset to train and test the LLM are very very very few for research papers vs the data for pop culture is easilier to source.
Why are we training kids on civics with Fox News or MSNBC? People are dumb and will continue to be so.
Who says they’re not?
Most research papers are likely ad valid as an average reddit point.
Getting published is a circlejerk, and rarely are they properly tested, or does anyone actually read them.
Those research papers are expensive to procure ethically, I'd imagine
This is a damn good question!
It won't worm. Autocomplete can't make new information.
Because scientific journals are paywalled - gibberish on Reddit is free*.
*Content is free unless you get caught and sued.