Image processing method, image processing apparatus, imaging apparatus, and storage medium, that correct a captured image using a neutral network

An image processing apparatus for correcting at least part of a captured image using a neural network, the image processing apparatus comprising:

• at least one processor configured to execute a plurality of tasks, including:
• an inversion determining task that determines, according to a position of a partial image of the captured image, an inversion axis that is a line on the partial image or on a filter of the neural network, the inversion axis being defined by an azimuth;
• a color determining task that determines, based on the inversion axis, a positional relationship among pixels of color components in an input image that is generated from the captured image based on the positional relationship and is input to the neural network;
• an image generating task that:
• in a case where the inversion determining task determines the inversion axis on the partial image, generates the input image by spatially inverting the partial image with respect to the inversion axis determined by the inversion determining task and by adjusting arrangement of color components in the input image so as to correspond to the positional relationship determined by the color determining task;
• and
• in a case where the inversion determining task determines the inversion axis on the filter of the neural network, spatially inverts the filter of the neural network with respect to the inversion axis determined by the first step and generates the input image by adjusting the arrangement of color components in the input image so as to correspond to the positional relationship determined by the color determining task;
• and
• a corrected image generating task that generates a corrected image by correcting the input image by processing the generated input image using the neural network.

An image processing system comprising:

• a first apparatus;
• and
• a second apparatus,
• wherein the first apparatus includes a transmitter configured to transmit a request for causing the second apparatus to execute processing for correcting a captured image, and
• wherein the second apparatus includes:
• a receiver configured to receive the request;
• at least one processor configured to execute a plurality of tasks, including:
• an inversion determining task that determines, according to a position of a partial image of the captured image, an inversion axis that is a line on the partial image or on a filter of the neural network, the inversion axis being defined by an azimuth;
• a color determining task determines, based on the inversion axis, a positional relationship among pixels in color components of an input image that is generated from the captured image based on the positional relationship and is input to the neural network;
• an image generating task that:
• in a case where the inversion determining task determines the inversion axis on the partial image, generates the input image by spatially inverting the partial image with respect to the inversion axis determined by the inversion determining task and by adjusting arrangement of color components in the input image so as to correspond to the positional relationship determined by the color determining the task;
• and
• in a case where the inversion determining task determines the inversion axis on the filter of the neural network, spatially inverts the filter of the neural network with respect to the inversion axis determined by the first step and generates the input image by adjusting the arrangement of color components in the input image so as to correspond to the positional relationship determined by the color determining task;
• and
• a corrected image generating task that generates a corrected image by correcting the input image by processing the generated input image using the neural network.

A non-transitory computer-readable storage medium storing a program executable by a computer to execute an image processing method of corrrecting at least part of a captured image using a neural network, the image processing method comprising:

• a first step of determining, according to a position of a partial image of the captured image, an inversion axis that is a line on the partial image or on a filter of the neural network, the inversion axis being defined by an azimuth;
• a second step of determining, based on the inversion axis, a positional relationship among pixels of color components in an input image that is generated from the captured image based on the positional relationship and is input to the neural network;
• a third step of:
• in a case where the first step determines the inversion axis on the partial image, generating the input image by spatially inverting the partial image with respect to the inversion axis determined by the first step and by adjusting arrangement of color components in the input image so as to correspond to the positional relationship determined by the second step;
• and
• in a case where the first step determines the inversion axis on the filter of the neural network, spatially inverting the filter of the neural network with respect to the inversion axis determined by the first step and generating the input image by adjusting the arrangement of color components in the input image so as to correspond to the positional relationship determined by the second step;
• and
• a fourth step of generating a corrected image by correcting the generated input image by processing the input image using the neural network.