Video Compression via Predictable Coding over Virtual Frames

Abstract

Video applications have become more and more common in the past few decades, in the meantime, optimizing video coding has received more attention. Existing video codecs usually focus on the encoder itself, and try to do everything possible to compress video with spatial (intraframe) compression and temporal (interframe) compression with the premise of reasonable distortion rate and video performance. In this work, we proposed a practical approach to improve video coding efficiency at a lower bitrate, which is to combine traditional Video Codec with interpolation neural network. A new concept called ``virtual frames'' was proposed and applied to the video coding process. We use raw frames as Ground Truth and virtual frames to train the interpolation neural network GDCN (Generalized Deformable Convolution Network), then encode the video synthesized with virtual frames via traditional AV1 video codec. With the pre-trained network, we could simply reconstruct the frames. This method can significantly improve the video compression effect compared with traditional video codec technology.