for ML engineers: why can't you simply exclude the word "fuck"?

finally debunked@slrpnk.net to Ask Lemmy@lemmy.world – 20 points –

So, I've heard that ML manipulates tokens and specifically for the English corpora they take place of words. If we want model to be polite and not to speak uncomfortable language we can remove certain words from the internal array where all tokens and their associative data are stored, for example "fuck".

11

You are viewing a single comment