Interpolation model learning method and device for learning interpolation frame generating module

A device configured to learn an interpolation frame generating model, the device comprising:

• a memory storing a program configured to train the interpolation frame generating model;
• and
• a processor configured to execute the program stored in the memory,
• wherein the processor is configured to, by executing the program,
• extract a first frame, a second frame, and a first ground truth (GT) frame, temporally arranged between the first frame and the second frame, from a plurality of continuous frames and
• generate a first interpolation frame between the first frame and the second frame using the interpolation frame generating model,
• extract a first complex feature between a plurality of frames included in a first frame group, based on a first feature extraction model, the first frame group comprising the first GT frame and at least two frames,
• extract a second complex feature between a plurality of frames included in a second frame group, based on the first feature extraction model, the second frame group comprising the first interpolation frame and the at least two frames, and train the interpolation frame generating model, based on the first complex feature and the second complex feature.

A non-transitory computer-readable storage medium storing instructions, which when executed by a processor, cause the processor to perform learning of an interpolation model using a plurality of continuous frames, the plurality of continuous frames comprise a first frame, a second frame, and a first ground truth (GT) frame temporally arranged between the first frame and the second frame, the instructions including:

• generate a first interpolation frame between the first frame and the second frame using the interpolation model;
• extract a first temporal-spatial feature of a first frame group, including at least three frames including the first GT frame, using a neural network model for extracting a temporal-spatial feature between frames;
• extract a second temporal-spatial feature of a second frame group, including at least three frames including the first interpolation frame, using the neural network model;
• determine temporal-spatial loss information based on a difference between the first temporal-spatial feature and the second temporal-spatial feature;
• and
• train the interpolation model by using the temporal-spatial loss information.