Method and apparatus with neural codec

A neural codec, comprising:

• a first simulated predictor configured to generate first candidate blocks by referring to a target block of a to-be-predicted current frame and pixels of neighbor blocks adjacent to the target block;
• a second simulated predictor configured to generate second candidate blocks having the same size as the target block by unfolding reference blocks of a reference frame adjacent to the current frame;
• a combination module configured to generate combined blocks acquired by concatenating the first candidate blocks and the second candidate blocks;
• and
• a selection network configured to select one of the combined blocks as a predicted block based on a difference between the combined blocks and the target block.

A learning method of a neural codec comprising a first simulated predictor, a second simulated predictor, and a selection network, the learning method comprising:

• training a first network of the first simulated predictor to predict a first block corresponding to a target block within a learning frame, in accordance with the target block of the to-be-predicted learning frame and pixels of neighbor blocks adjacent to the target block being input;
• training a second network of the second simulated predictor to predict a second block corresponding to a target block using a reference block selected based on a prediction mode, in accordance with the reference block of a reference frame adjacent to the learning frame and the target block being input;
• and
• training the selection network to select one of the first block and the second block as a predicted block based on the prediction mode.