Deepfake lip syncs video to edit out profanity, match translation

cyu@sh.itjust.worksbanned from community to Technology@lemmy.world – 133 points –
Flawless use of TrueSync on the movie "Fall"
youtube.com
44

You are viewing a single comment

You know what, I think AI/deepfakes could have a legitimate use in generating fully realistic dubs, provided the appropriate parties are compensated fairly. It must be annoying as hell seeing English-language movies clumsily dubbed over with mismatched voices and completely out of sync lip movements.

It must be annoying as hell seeing English-language movies clumsily dubbed over with mismatched voices and completely out of sync lip movements.

It is. On my country we only use subtitles (except for movies aimed towards kids) and I think it's much better

Same in mine, Portugal.

It has the added benefit that people naturaly improve their learning of foreign languages when foreign language programming on TV is subtitled: just compare the percentage of people who speak "good" english in Portugal with that of next door Spain were everything is dubbed.

That and I always found it weird when I went to Spain on vacations and saw things like Westerns with John Wayne were his voice was dubbed by some (comparativelly) whinny spanish guy.

Well you think not every language uses the exact same number of words or syllables to convert the same amount of information. So on translation not only do people have to translate the words they have to basically rewrite the entire movie for every language because the speech time has to match the screen time

Of course not, but a well-developed deepfake is so robust that you can shorten or lengthen the dialog and rate of speech and it has no problem compensating.

Throw in just a bit more AI and you can shorten or lengthen the duration of the scene a bit, too.

1 more...