Method and apparatus for video coding using an in-loop filter based on a transformer
AssigneesHyundai Motor Company, Kia Corporation, Ewha University-Industry Collaboration Foundation
InventorsJe Won Kang, Jin Heo, Seung Wook Park
Life of this patent
Estimating the term…
Abstract
A method and an apparatus are disclosed for video using an in-loop filter based on Transformer. The video coding method and the apparatus apply a current video block to an attention module of a Transformer, which is a deep learning model. The video coding method and the apparatus utilize the resultant Transformer-based in-loop filter.
Claims16
A method performed by a video decoding device for enhancing a picture quality of a reconstructed frame, the method comprising:
- obtaining an input region of a preset size from the reconstructed frame which is a reconstruction of an original frame and has been reconstructed in advance by the video decoding device;
- and
- generating an enhanced video region that approximates the original frame by feeding the input region into an in-loop filter that is deep learning-based,
- wherein the in-loop filter comprises K consecutive Transformer blocks, and K is a natural number,
- wherein generating the enhanced video region includes converting, by using the K consecutive Transformer blocks, an input image into a final output feature based on an attention operation, and
- wherein at least one of the K consecutive Transformer blocks performs the attention operation based on a query vector, a key vector, and a value vector.
A method performed by a video encoding device for enhancing a picture quality of a reconstructed frame, the method comprising:
- obtaining an input region of a preset size from the reconstructed frame, which is a reconstruction of an original frame and has been reconstructed in advance by the video encoding device;
- and
- generating an enhanced video region that approximates the original frame by feeding the input region into an in-loop filter that is deep learning-based,
- wherein the in-loop filter comprises K consecutive Transformer blocks, and K is a natural number,
- wherein generating the enhanced video region includes converting, by using the K consecutive Transformer blocks, an input image into a final output feature based on an attention operation, and
- wherein at least one of the K consecutive Transformer blocks performs the attention operation based on a query vector, a key vector, and a value vector.
A non-transitory computer-readable recording medium storing a bitstream generated by a video encoding method, the video encoding method comprising:
- obtaining an input region of a preset size from a reconstructed frame, which is a reconstruction of an original frame and has been reconstructed in advance by a video encoding device;
- and
- generating an enhanced video region that approximates the original frame by feeding the input region into an in-loop filter that is deep learning-based,
- wherein the in-loop filter comprises K consecutive Transformer blocks, and K is a natural number,
- wherein generating the enhanced video region includes converting, by using the K consecutive Transformer blocks, an input image into a final output feature based on an attention operation, and
- wherein at least one of the K consecutive Transformer blocks performs the attention operation based on a query vector, a key vector, and a value vector.
Dependencies are inferred from the claim text.