Apparatus and method for accelerating super-resolution in real-time video streaming

An apparatus for super-resolution (SR) acceleration installed in a mobile device, the apparatus comprising:

• an input/output unit configured
• to obtain a manifest file for a video from a server,
• to select, from the manifest file, an option on a super-resolution deep neural network (SR DNN) and information on a cache profile, and
• to transmit the option and the information on the cache profile to the server and obtain, from the server, the SR DNN corresponding to the option, the cache profile, and video chunks;
• and
• a decoding unit configured to initialize an SR-integrated decoder by using the SR DNN and then enhance, based on the cache profile, a resolution of a current frame that forms the video chunks by using the SR-integrated decoder.

A method performed by a server for super-resolution (SR) acceleration, the method comprising:

• obtaining a video;
• generating multiple video versions corresponding to multiple bitrates from the video by using an image encoder;
• training, by using the video and the multiple video versions, corresponding multiple super-resolution deep neural networks (SR DNNs);
• selecting anchor frames, one each for each of the multiple SR DNNs from the video versions, as bounded by a preset quality margin and generating a cache profile including information on the anchor frames;
• and
• generating a manifest file including information on the cache profile and information on the multiple SR DNNs.