It won't generate random numbers. It'll generate random numbers from its training data.
If it's asked to generate passwords I wouldn't be surprised if it generated lists of leaked passwords available online.
These models are created from masses of data scraped from the internet. Most of which is unreviewed and unverified. They really don't want to review and verify it because it's expensive and much of their data is illegal.
Also, researchers asking ChatGPT for long lists of random numbers were able to extract its training data from the output (which OpenAI promptly blocked).
It won't generate random numbers. It'll generate random numbers from its training data.
If it's asked to generate passwords I wouldn't be surprised if it generated lists of leaked passwords available online.
These models are created from masses of data scraped from the internet. Most of which is unreviewed and unverified. They really don't want to review and verify it because it's expensive and much of their data is illegal.
Also, researchers asking ChatGPT for long lists of random numbers were able to extract its training data from the output (which OpenAI promptly blocked).
Or maybe that’s what you meant?