Could we use AI to update 4:3 media to 16:9?
I've been re-watching star trek voyager recently, and I've heard when filming, they didn't clear the wide angle of filming equipment, so it's not as simple as just going back to the original film. With the advancement of AI, is it only a matter of time until older programs like this are released with more updated formats?
And if yes, do you think AI could also upgrade to 4K. So theoretically you could change a SD 4:3 program and make it 4k 16:9.
I'd imagine it would be easier for the early episodes of Futurama for example due to it being a cartoon and therefore less detailed.
You are viewing a single comment
You should be able to but remember that aspect ratios and framing are done intentionally so what is generated won't be at all true to what should be in scene once the frame is there. You'd be watching derivative media. Upscaling should be perfectly doable but eventually details will be generated that will not have originally existed in scenes as well.
Probably would be fun eventually to try the conversion and see what differences you get.
4:3 - Jumpscare, gremlin jumps in from off-camera.
16:9 AI upsized - Gremlin hangs out awkwardly to the left of the characters for half a minute, then jumps in.
I would watch the hell out of that movie.
Well for sure there's some value in it, but let's not pretend it wouldn't completely change the intention and impact.
I don't know who would downvote that. You're absolutely right. And I would still watch the hell out of that movie.
I was just thinking that. Or something like a comedy bit where the camera pans to a character who had just been out of frame.
Overall it seems like impressive technology to be able to reform old media, but I’d rather put it to use in tastefully sharpening image quality rather than reframing images.
Haha, yes. I spent 15 minutes trying to remember the term for the pan/zoom-to-reveal comedy effect before giving up and settling on a botched jumpscare.
Exactly, and to add to it, you can't know the director's vision or opinion on how the framing should be adjusted. AI can make images easily but it won't understand subtext and context that was intended. No time soon at least.
Very true, I remember a few years ago someone converting old cartoons to a consistent 60 frames a second.
If they’d asked an animator they’d have found out that animation purposely uses different rates of change to give a different feel to scenes. So the improvement actually ruined what they were trying to improve.
There's a video from the YouTuber Noodle who does a good job explaining it
Yes, sometimes frame rates are intentional choices for artistic reasons and sometimes they are economic choices that animators work around.
Old Looney Tunes used a lot of smear frames in order to speed up production. They were 24 frames per second broadcast on doubles, which meant 12 drawn frames per second with each frame being shown twice. The smear frames gave the impression of faster movement. Enhancing the frame rate on those would almost certainly make them look weird.
If you want to see an artistic choice, the first Spiderverse movie is an easy example. It’s on doubles (or approximates being on doubles in CG) for most scenes to gives them a sort of almost stop motion look, and then goes into singles for action scenes to make them feel smoother.
This.
Surprise! If you want to go from 4:3 to wide screen, you still have exactly the same problem as when using pan&scan for going from wide to 4:3.
Definitely. I remember hearing about The Wire being released in a 16:9 format, even though it was shot and broadcast in 4:3, and how that potentially messes up some of the shot framing.
They did I by cropping from top and bottom rather than AI infilling, but the issue is the same.
IIRC, David Simon wrote a really interesting piece about how they did it but did everything they could to try and stay true to Robert Colesberry's carefully planned framing, as they were aware that had it been intended for 16:9 he'd have framed things differently.
Personally I wish they had kept it at 4:3 and only released it in a higher resolution. Glad I still have my old 4:3 DVDs.