Image processing apparatus for converting a lower resolution image into a higher resolution image using cyclic coefficients

An image processing method comprising the steps of:

• converting a standard definition interlace image including a first number of pixels into a first standard definition progressive image;
• generating a first high definition progressive image including a second number of pixels, higher than the first number of pixels, which are equal to a number of pixels of a target image by interpolating the first standard definition progressive image;
• classifying, in accordance with a feature of the first high definition progressive image, into classes, subject pixels forming a second high definition progressive image, which serves as the target high definition image, including the second number of pixels and having a quality higher than the first high definition progressive image, the second high definition progressive image being determined from the first high definition progressive image;
• and
• determining the second high definition progressive image, which serves as the target high definition image, from the first high definition progressive image by performing computation using the first high definition progressive image and a prediction coefficient for each of the classes into which the subject pixels are classified among prediction coefficients obtained by conducting learning using a plurality of high definition progressive images, each including the second number of pixels, wherein said step of converting includes,
• converting the standard definition interlace image into a progressive intermediate image,
• detecting motion vectors and macro-block differences of the standard definition interlace image by using, as a minimum unit, a distance shorter than an interval between pixels forming the progressive intermediate image,
• setting, based on a vertical motion, which is a vertical component of each of the motion vectors, a first cyclic coefficient for a first type of pixel that is located at a position where a pixel forming the standard definition interlace image exists, the first cyclic coefficient calculated by using the corresponding motion vector and the macro-block difference, and a second cyclic coefficient for a second type of pixel, which is located at a position where a pixel forming the standard definition interlace image does not exist, the second cyclic coefficient calculated by using the corresponding motion vector and the macro-block difference,
• motion-compensating, on a basis of the motion vectors, a past first progressive image to generate a motion-compensated image, and
• generating the first standard definition progressive image by adding pixel values of the first type of pixels located at corresponding positions of the intermediate image and the motion-compensated image by using the first cyclic coefficients as weights and by adding pixel values of the second type of pixels located at corresponding positions of the intermediate image and the motion-compensated image by using the second cyclic coefficients as weights.

A non-transitory computer readable medium having computer instructions recorded thereon, the computer instructions configured to perform a method when executed on a computer, the method comprising the steps of:

• converting a standard definition interlace image including a first number of pixels into a first standard definition progressive image;
• generating a first high definition progressive image including a second number of pixels, higher than the first number of pixels, which are equal to a number of pixels of a target high definition image by interpolating the first standard definition progressive image;
• classifying, in accordance with a feature of the first high definition progressive image, into classes, subject pixels forming a second high definition progressive image, which serves as the target high definition image, including the second number of pixels and having a quality higher than the first high definition progressive image, the second high definition progressive image being determined from the first high definition progressive image;
• and
• determining the second high definition progressive image, which serves as the target high definition image, from the first high definition progressive image by performing computation using the first high definition progressive image and a prediction coefficient for each of the classes into which the subject pixels are classified among prediction coefficients obtained by conducting learning using a plurality of high definition progressive images, each including the second number of pixels, wherein said step of converting includes,
• converting the standard definition interlace image into a progressive intermediate image,
• detecting motion vectors and macro-block differences of the standard definition interlace image by using, as a minimum unit, a distance shorter than an interval between pixels forming the progressive intermediate image,
• setting, based on a vertical motion, which is a vertical component of each of the motion vectors, a first cyclic coefficient for a first type of pixel that is located at a position where a pixel forming the standard definition interlace image exists, the first cyclic coefficient calculated by using the corresponding motion vector and the macro-block difference, and a second cyclic coefficient for a second type of pixel, which is located at a position where a pixel forming the standard definition interlace image does not exist, the second cyclic coefficient calculated by using the corresponding motion vector and the macro-block difference,
• motion-compensating, on a basis of the motion vectors, a past first progressive image to generate a motion-compensated image, and
• generating the first standard definition progressive image by adding pixel values of the first type of pixels located at corresponding positions of the intermediate image and the motion-compensated image by using the first cyclic coefficients as weights and by adding pixel values of the second type of pixels located at corresponding positions of the intermediate image and the motion-compensated image by using the second cyclic coefficients as weights.

An image processing apparatus comprising:

• a converter configured to convert a standard definition interlace image including a first number of pixels into a first standard definition progressive image;
• an interpolator configured to interpolate the first standard definition progressive image to generate a first high definition progressive image including a second number of pixels, higher than the first number of pixels, which are equal to a number of pixels of a target high definition image;
• a classification unit configured to classify, in accordance with a feature of the first high definition progressive image, into classes, subject pixels forming a second high definition progressive image, which serves as the target high definition image, including the second number of pixels and having a quality higher than the first high definition progressive image, the second high definition progressive image being determined from the first high definition progressive image;
• a storage unit configured to store a prediction coefficient for each of the classes, the prediction coefficient being obtained by conducting learning using a plurality of high definition progressive images, each including the second number of pixels;
• and
• a computation unit configured to perform computation using the first high definition progressive image and the prediction coefficient for each of the classes into which the subject pixels are classified to determine the second high definition progressive image, which serves as the target high definition image, from the first high definition progressive image, wherein the converter includes,
• an interlace-progressive converter configured to convert the standard definition interlace image into a progressive intermediate image,
• a motion-vector detector configured to detect motion vectors and macro-block differences of the standard definition interlace image by using, as a minimum unit, a distance shorter than an interval between pixels forming the progressive intermediate image,
• a cyclic-coefficient setting unit configured to set, based on a vertical motion, which is a vertical component of each of the motion vectors, a first cyclic coefficient for a first type of pixel that is located at a position where a pixel forming standard definition the interlace image exists, the first cyclic coefficient calculated by using the corresponding motion vector and the macro-block difference, and a second cyclic coefficient for a second type of pixel, which is located at a position where a pixel forming the standard definition interlace image does not exist, the second cyclic coefficient calculated by using the corresponding motion vector and the macro-block difference,
• a motion compensator configured to motion-compensate, on a basis of the motion vectors, a past first progressive image to generate a motion-compensated image, and
• an output image generator configured to generate the first standard definition progressive image by adding pixel values of the first type of pixels located at corresponding positions of the intermediate image and the motion-compensated image by using the first cyclic coefficients as weights and by adding pixel values of the second type of pixels located at corresponding positions of the intermediate image and the motion-compensated image by using the second cyclic coefficients as weights.