New models already train on synthetic data. It's already a solved solution.
Is it really a solution, though, or is it just GIGO?
For example, GPT-4 is about as biased as the medical literature it was trained on, not less biased than its training input, and thereby more inaccurate than humans:
All the latest models are trained on synthetic data generated on got4. Even the newer versions of gpt4. Openai realized it too late and had to edit their license after Claude was launched. Human generated data could only get us so far, recent phi 3 models which managed to perform very very well for their respective size (3b parameters) can only achieve this feat because of synthetic data generated by AI.
I didn't read the paper you mentioned, but recent LLM have progressed a lot in not just benchmarks but also when evaluated by real humans.
You’re two years late.
Maybe not for the reputable ones, that’s 2026, but these sheisters have been digging out the bottom of the swimming pool for years.
https://theconversation.com/researchers-warn-we-could-run-out-of-data-to-train-ai-by-2026-what-then-216741
New models already train on synthetic data. It's already a solved solution.
Is it really a solution, though, or is it just GIGO?
For example, GPT-4 is about as biased as the medical literature it was trained on, not less biased than its training input, and thereby more inaccurate than humans:
https://www.thelancet.com/journals/landig/article/PIIS2589-7500(23)00225-X/fulltext
All the latest models are trained on synthetic data generated on got4. Even the newer versions of gpt4. Openai realized it too late and had to edit their license after Claude was launched. Human generated data could only get us so far, recent phi 3 models which managed to perform very very well for their respective size (3b parameters) can only achieve this feat because of synthetic data generated by AI.
I didn't read the paper you mentioned, but recent LLM have progressed a lot in not just benchmarks but also when evaluated by real humans.