Image processing method, image processing apparatus, image processing system, and manufacturing method of learnt weight

An image processing apparatus comprising:

• at least one processor or circuit configured to execute a plurality of tasks including:
• an acquiring task configured to acquire weight information of a machine learning model, and input data including a captured image and a map generated based on first information relating to a state of an optical system at a time of capturing the captured image, the optical system having been used for capturing the captured image;
• and
• a generating task configured to generate an estimated image by (i) inputting the input data to the machine learning model and (ii) sharpening the captured image using the machine learning model or reshaping blurs included in the captured image using the machine learning model,
• wherein the weight information has been subjected to learning in advance by using training input data including a training image and information relating to a state of the optical system corresponding to the training image so that different processing is performed according to the first information.

An image processing system having a first apparatus and a second apparatus, wherein

• the first apparatus includes
• at least one processor or circuit configured to execute a transmitting task configured to transmit, to the second apparatus, a request relating to execution of processing on a captured image, and
• wherein the second apparatus includes:
• at least one processor or circuit configured to execute a plurality of tasks including:
• a receiving task configured to receive the request;
• an acquiring task configured to acquire weight information of a machine learning model, and input data including the captured image and a map generated based on first information relating to a state of an optical system at a time of capturing the captured image, the optical system having been used for capturing the captured image;
• and
• a generating task configured to generate an estimated image by (i) inputting the input data to the machine learning model and (ii) sharpening the captured image using the machine learning model or reshaping blurs included in the captured image using the machine learning model, and
• wherein the weight information has been subjected to learning in advance by using training input data including a training image and information relating to a state of the optical system corresponding to the training image so that different processing is performed according to the first information.

An image processing method comprising:

• a first step of acquiring input data including a training image and a map generated based on first information relating to a state of an optical system corresponding to the training image;
• a second step of generating an output image by (i) inputting the input data to a machine learning model and (ii) sharpening the training image using the machine learning model or reshaping blurs included in the training image using the machine learning model;
• and
• third step of updating a weight of the machine learning model based on the output image and a ground truth image so that the machine learning mode performs different processing according to information relating to a state of the optical system.

A manufacturing method of learnt weight comprising:

• a first step of acquiring input data including a training image and a map generated based on first information relating to a state of an optical system corresponding to the training image;
• a second step of generating an output image by (i) inputting the input data toa machine learning model and (ii) sharpening the training image using the machine learning model or reshaping blurs included in the training image using the machine learning model;
• and
• third step of updating a weight of the machine learning model based on the output image and a ground truth image so that the machine learning mode performs different processing according to information relating to a state of the optical system.

An image processing apparatus comprising:

• at least one processor or circuit configured to execute a plurality of tasks including:
• an acquiring task configured to acquire input data including a training image and a map generated based on first information relating to a state of an optical system corresponding to the training image;
• a generating task configured to generate an output image by (i) inputting the input data to a machine learning model and (ii) sharpening the training image using the machine learning model or reshaping blurs included in the training image using the machine learning model;
• and
• an updating task configured to update a weight of the machine learning model based on the output image and a ground truth image so that the machine learning mode performs different processing according to information relating to a state of the optical system.