We’re starting to roll out some music videos, the first of which came out yesterday (Jan 17, 2021) https://youtu.be/UJk4JJz5OXI. The next two videos will roll out weekly on Sundays at 1200 CST (Jan 24th and Jan 31st 2021).
During this release process we ran into some issues with YouTube’s video compression that are relevant to other creators.
The problem and solution
I always knew this segment at the end would cause a problem for compression. Moving particles and confetti like things are notorious for wrecking temporal video compression like h.264, but I didn’t anticipate that YouTube would handle it so poorly.
After some research I discovered that YouTube uses AVC compression for 1080p videos. If YouTube thinks your channel is important you get upgraded to VP9 compression on all videos. Otherwise, the only current solution to get upgraded to VP9 is to upload at least 2k video. I tested it with 1440p and 4k videos and was upgraded (as of January 2021). There seem to have been alternate ways to do this in the past, but none of them are still working to my knowledge.
More on AVC vs VP9
AVC seems to be the same thing as H.264, but YouTube must use a low bit rate because the output is garbage. VP9 on the other hand is an open compression format developed by Google to compete with H.265(HEVC). It’s likely YouTube defaults to AVC because it has a similar file size to VP9 at 1080p and below, and VP9 takes longer to encode than AVC. However, VP9 offers superior visual quality as shown below.
That’s the same file scaled up to 1440p (60Mbps) which triggers the VP9 encoding. It’s not perfect, but it is FAR better.
Overshoot recommended bitrate to further improve quality
It also seems that you should overshoot YouTube’s recommended bitrate as seen here: https://support.google.com/youtube/answer/1722171?hl=en#zippy=%2Cbitrate
My only real world test of this was on a 1080p h.264 60Mbps upload which only gets you the AVC codec. But you can see below that it looks even better than the original 12Mbps version I uploaded at the top of this post.
Checking YouTube encoding settings
If you right click the YouTube video and click “stats for nerds” there is a section that tells you the codec used. If it says “VP09” next to “Codecs”, your video got upgraded.
Since I wanted to actually increase the rendered pixels (instead of just interpolating pixels or using nearest neighbor), I tried AI upscaling.
Found a cool thing on GitHub that lets you use Waifu2x (for real, that’s what its called) or some other algorithms for upscaling video. It’s called Video 2X: https://github.com/k4yt3x/video2x
Unfortunately for this particular video it did a terrible job, but it usually works well on normal animation (waifu2x algorithm) or live action (SRMD or RealSR algorithms). Keep in mind that its encoding process TAKES FOREVER. Also note that Video 2X appears to be doing absolutely nothing for quite a while as it transcodes your movie file to a PNG sequence for upscaling each frame.
The depth map image confused Waifu2X so I had to go back to the source files and re-render the depth maps at the highest resolution possible (which wasn’t very high because I quickly ran into VRAM limitations because I only have 6GB of VRAM right now)
2d to depth map
For those curious, the depth maps used in this video series are generated from normal camera footage using this machine learning model: http://stereo.jpn.org/jpn/stphmkr/google/indexe.html