Creating image enhancement training data pairs

A method comprising:

• receiving a collection of image pairs, each image pair comprising a high resolution source image and a captured image corresponding to a degraded version of the high resolution source image, and wherein each captured image is obtained by performing an image capture of a corresponding displayed high resolution source image;
• for each image pair in the collection of image pairs:
• performing pixel-level geometric alignment between the high resolution source image and the captured image to generate a geometrically aligned high resolution source image, and
• performing color and luminance matching between the geometrically aligned high resolution source image and the captured image, wherein performing the color and luminance matching comprises:
• separating the geometrically aligned high resolution source image into a low frequency part and a high frequency part;
• determining a ratio base between the low frequency part of the geometrically aligned high resolution source image and the captured image;
• calculating a color and luminance matching ratio between the low frequency part of the geometrically aligned high resolution source image and the low frequency part of the captured image based on the ratio base;
• applying the color and luminance matching ratio to the low frequency part of the geometrically aligned high resolution source image to form a color and luminance matched low frequency part;
• and
• combining the color and luminance matched low frequency part of the geometrically aligned high resolution source image with the high frequency part of the geometrically aligned high resolution source image to form a single color and luminance matched source image;
• and
• forming a collection of geometrically aligned and color and luminance matched image pairs.

A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to perform respective operations, the operations comprising:

• generating a collection of image pairs comprising:
• positioning an image capture device relative to a display device;
• capturing a plurality of images, wherein capturing each image comprises:
• displaying a high resolution image on the display device;
• and
• capturing a degraded version of the high resolution image using the image capture device;
• and
• generating image pairs, each image pair comprising a high resolution image and a corresponding degraded version of the high resolution image,
• wherein positioning the image capture device relative to the display device comprises positioning the image capture device at a distance from the display device such that a resolution of the displayed high resolution image is a specific multiple of a resolution of the image capture device.