4K Blu-Rays Have an AI Upscaling Problem: Buyer Beware

4K TVs are commonplace, 4K streaming is within reach for many broadband users, and 4K Blu-rays are still sought after by film geeks. The problem is most content is not in 4K, and many studios are turning to AI upscaling technology to convert content to 4K, with (to put it mildly) very mixed results.

Not All Movies Are Shot in 4K

This is likely obvious to most people, but the vast majority of TV shows, movies, and other professionally produced content was not shot and produced at 4K UHD quality. It makes perfect sense, since there’s more than a century of movies and almost a century of TV shows in existence. So, 4K TVs and content production is still very recent if you look at the whole library of existing video media.

These days, even indie movies are shot in 4K and big-budget Hollywood productions will even shoot at higher resolutions to have some measure of overhead in the editing room. So there’s no problem when it comes to producing a 4K version of a show or movie today, because that’s the format it was made in.

If a movie or show was shot on film, you can scan every frame of the film stock at incredibly high resolutions. Images on film are represented by photoreactive chemicals, and there’s a lot of detail in those frames.

Currently, the scanners that are used to digitize film stock can scan at up to 16K resolution. It’s a good idea to scan them at extremely high resolutions now, in case you need to use those intermediate files again later for newer formats.

This is the gold-standard method to get a 4K version of a movie that’s only been available in older formats such as VHS, DVD, or standard Blu-ray. However, this is a painstaking and expensive process.

Some content, like Star Trek: The Next Generation was shot on film, but certain special effects are not present in the film, and have to be redone. The TNG standard Blu-ray collection required painstaking recreations of all episodes based on scans of film negatives and the original editing production notes. The results are absolutely stunning, but don’t expect this treatment for anything that isn’t super-popular.

Your changes have been saved

Follow Followed

Follow with Notifications Follow Unfollow

Star Trek: The Next Generation – The Complete Series Blu Ray

One of the few Star Trek TV series to get a painstaking HD remaster using the original camera film negatives. To date, this is the most ‘HD’ that classic TV Star Trek gets!

Not All Upscaling Is the Same

If your content was shot digitally, then you are limited to whatever resolution that camera was capable of producing. There is no high-quality “original” footage to reprocess. A camera with an SD sensor will produce video with the same number of pixels as an SD display. There is no additional data captured.

This is where “upscaling” comes into the picture. Literally.

Upscaling describes a wide range of techniques used to convert a video from a lower resolution to a higher one, making it look good on a high-resolution TV or monitor. Integer scaling is probably the simplest form of upscaling and works well when the lower resolution is an exact multiple of the higher resolution.

For example, since 1080p video is exactly a quarter of the resolution of 4K UHD, you simply double the pixels on both axes. This means that you create one big pixel out of groups of four pixels.

This image doesn’t look as sharp as 4K UHD, but apart from being a bit softer in terms of detail, everything else is preserved nicely. From here there are various clever mathematical algorithms that can infer detail from the original image and make a sharper image as a result. However, they aren’t that sophisticated, and you usually have a tradeoff with some form of artifacting in the upscaled video, depending on how it was upscaled.

In recent years, machine learning methods have been introduced to upscale video. These AI (an umbrella term that includes “machine learning”) upscalers “understand” what they are looking at, having been trained on massive datasets. They can produce a 4K UHD version of a TV show or film that actually has more detail in it than the original.

However, this detail never existed in the original material. The AI upscaler “imagined” what the high-res version of the media would look like and outputs it.

The Results Can Be Terrible

I want to preface this by saying that AI upscaling, when done right, can produce amazing results. For example, DS9:Redefined is a fan project to upscale Star Trek: Deep Space 9. The creator of the project uses Topaz AI to upscale DVD and LaserDisc footage. However, that’s just a tiny part of a painstaking workflow. It’s not just a single button press and hey-presto, here’s your 4K Star Trek.

So, the technology is not fundamentally an issue, it’s how it’s been used to create some 4K content, especially 4K Blu-rays, that should not have passed quality control. If you want to see examples of this in motion, I recommend Nerrel’s excellent video cataloging and showcasing this issue across several major releases. Just be aware that he uses some PG-13 salty language at points. So don’t play this video anywhere some occasionally sweary language can get you in trouble.

Iconic films like True Lies now have actors that look absolutely strange, with hyper-craggy facial creases and hair that looks like CG. Likewise, background faces in Aliens deform as the AI upscaler can’t figure out where an actor’s mouth should go. Sunglasses in Terminator 2 become gaping black eye sockets, as if that movie didn’t have enough skulls in it already.

The most egregious example I’ve seen is from this post on X by WOWmd

Here are the two images side-by-side:

The AI upscaler has drawn two creepy faces on the out-of-focus background characters. It’s mistaken a blurred background for a lack of detail, and the results are creepy!

What’s most egregious about all the films I’ve mentioned and I Love Lucy is that they were made using film! So, in principle, you have the option of scanning the original film negatives and making pristine 4K versions of them.

However, most likely due to time and cost, the creators of these versions opted to use AI upscaling software, and then signed off on alterations that software made which are, in my opinion, inexcusable.

Even worse, there’s a good chance these 4K versions will be used as the basis for 4K streams of these movies, and that the lower resolution streams will just be downscaled versions of the AI-altered version. This is just another reason I’m happy to collect DVD and standard Blu-ray versions of movies.

How to Avoid Bad 4K Blu-Rays

4K UHD Blu-rays are quite niche, and not many movies get the 4K treatment. It’s a film format for enthusiasts, and so you can expect the most anticipated 4K movie releases to get detailed scrutiny in reviews. So the best way to avoid movies that have been damaged by poor AI-upscaling is to hop on YouTube, or your favorite film-nerd blog, and read a review of those discs.

For that matter, standard Blu-rays won’t be automatically OK, since there’s nothing stopping someone from, say, upscaling DVD footage to 1080p. Either way, if you are anticipating a physical release of a film that was not made in 4K (or 1080p FHD), don’t buy it until your favorite film geeks have watched it.