Image processing system, image processing method, and program

A method comprising:

• acquiring 1st to Nth input frames (N is a natural number equal to or greater than 2) having a prescribed input pixel number;
• acquiring, based on each of the input frames, 1st to Nth intermediate frames by generating an intermediate frame for each input frame which corresponds to the input frame and which includes an intermediate pixel number equal to or greater than the input pixel number;
• and
• inputting each of the intermediate frames to a machine learning model;
• and
• acquiring 1st to Nth estimation frames including an estimated pixel number equal to or greater than the intermediate pixel number which is greater than the input pixel number, wherein the machine learning model includes:
• a cumulative feature information output layer to which the nth intermediate frame (n=2, 3, . . . , N) and (n−1)th auxiliary information based on (n−1)th cumulative feature information indicating the features of the 1st to (n−1)th intermediate frames are inputted, wherein the cumulative feature information output layer outputs nth cumulative feature information indicating the features of the 1st to nth intermediate frames;
• and
• an estimation frame output layer to which the nth cumulative feature information is inputted, wherein the estimation frame output layer outputs the nth estimation frame, wherein the machine learning model was trained using a plurality of training data including:
• a learning intermediate frame including the intermediate pixel number generated based on a learning input frame having the input pixel number;
• and
• a learning estimation frame including the estimated pixel number.

One or more non-transitory computer-readable storage media storing instructions that, upon execution by one or more processors of a system, cause the system to perform operations comprising:

• acquiring 1st to Nth input frames (N is a natural number equal to or greater than 2) having a prescribed input pixel number;
• acquiring, based on each of the input frames, 1st to Nth intermediate frames by generating an intermediate frame for each input frame which corresponds to the input frame and which includes an intermediate pixel number equal to or greater than the input pixel number;
• and
• inputting each of the intermediate frames to a machine learning model;
• and
• acquiring 1st to Nth estimation frames including an estimated pixel number equal to or greater than the intermediate pixel number which is greater than the input pixel number, wherein the machine learning model includes:
• a cumulative feature information output layer to which the nth intermediate frame (n=2, 3, . . . , N) and (n−1)th auxiliary information based on (n−1)th cumulative feature information indicating the features of the 1st to (n−1)th intermediate frames are inputted, wherein the cumulative feature information output layer outputs nth cumulative feature information indicating the features of the 1st to nth intermediate frames;
• and
• an estimation frame output layer to which the nth cumulative feature information is inputted, wherein the estimation frame output layer outputs the nth estimation frame, wherein the machine learning model was trained using a plurality of training data including:
• a learning intermediate frame including the intermediate pixel number generated based on a learning input frame having the input pixel number;
• and
• a learning estimation frame including the estimated pixel number.