# US 2021/0243395 A1 · Image pickup apparatus for inferring noise and learning device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0243395 A1 - Document ID: US20210243395A1 - Status: Published application - Kind: A1 - Application number: 17/154,365 - Filed: 2021-01-21 - Published: 2021-08-05 - Earliest claimed: 2020-01-31 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Makiko Saito - Web page: https://patentgrep.com/patent/US20210243395A1 ## CPC - H04N 23/617: Upgrading or updating of programs or applications for camera control - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image} - H04N 25/68: applied to defects - H04N 25/70: SSIS architectures; Circuits associated therewith - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract An image pickup apparatus includes a solid-state image pickup device, a learning unit configured to generate a learned model by performing machine learning using, as supervised data, first correction information for identifying a pixel signal for which noise is to be corrected among a plurality of pixels of the solid-state image pickup device and using, as input, an image acquired from the solid-state image pickup device and that has not been corrected based on the first correction information, and an inference unit configured to infer a pixel signal on which noise is superimposed by inputting an image corrected based on the first correction information to the learned model. ## Claims ### Claim 1 An image pickup apparatus comprising: a solid-state image pickup device; at least one processor or circuit configured to function as following units; a learning unit configured to generate a learned model by performing machine learning using, as supervised data, first correction information for identifying a pixel signal for which noise is to be corrected among a plurality pixels of the solid-state image pickup device and using, as input, an image acquired from the solid-state image pickup device and that has not been corrected based on the first correction information; and an inference unit configured to infer a pixel signal on which noise is superimposed by inputting an image corrected based on the first correction information to the learned model. ### Claim 2 (depends on 1) The image pickup apparatus according to claim 1, wherein the first correction information is information in which address information of a defective pixel to be corrected is registered in advance. ### Claim 3 (depends on 2) The image pickup apparatus according to claim 2, wherein predetermined correction is performed on a pixel signal of a pixel that is not the defective pixel among the pixels. ### Claim 4 (depends on 3) The image pickup apparatus according to claim 3, wherein the predetermined correction is correction of noise that occurs due to variations of column circuits of the solid-state image pickup device. ### Claim 5 (depends on 1) The image pickup apparatus according to claim 1, wherein the learning unit performs the machine learning of the learned model so that an error between the first correction information and second correction information output when an image corrected based on the first correction information is input to the learned model becomes small. ### Claim 6 (depends on 1) The image pickup apparatus according to claim 1, further comprising a first correction unit configured to correct an image output by the solid-state image pickup device by using the first correction information, wherein an image corrected by the first correction unit is input to the inference unit. ### Claim 7 (depends on 6) The image pickup apparatus according to claim 6, wherein the solid-state image pickup device, the first correction unit, and the inference unit are provided on respective different substrates, which are laminated on one another. ### Claim 8 (depends on 6) The image pickup apparatus according to claim 6, wherein the first correction unit determines a pixel signal to be corrected based on at least one of an accumulation time, a readout gain, and a temperature and corrects only a pixel signal determined as a pixel signal to be corrected by using the first correction information. ### Claim 9 (depends on 1) The image pickup apparatus according to claim 1, further comprising a second correction unit configured to correct an image that has been corrected based on the first correction information, based on second correction information for identifying a pixel signal inferred by the inference unit. ### Claim 10 (depends on 1) The image pickup apparatus according to claim 1, further comprising a third correction unit configured to correct an image that is acquired from the solid-state image pickup device and that has not been corrected based on the first correction information, based on second correction information for identifying a pixel signal inferred by the inference unit. ### Claim 11 (depends on 1) The image pickup apparatus according to claim 1, wherein the inference unit calculates a difference value in pixel signal of each pixel between an image input to the learned model and an image based on a result of inference using the learned model and infers a pixel signal that exceeds the difference value as a pixel signal on which the noise is superimposed. ### Claim 12 (depends on 1) The image pickup apparatus according to claim 1, wherein a part of an image acquired from the solid-state image pickup device is excluded from a learning target in accordance with processing capability of the inference unit and a frame rate at which an image is acquired. ### Claim 13 An image pickup apparatus comprising: a solid-state image pickup device; at least one processor or circuit configured to function as following units; a communication unit configured to acquire a learned model from a learning device that generates the learned model by performing machine learning using, as supervised data, first correction information for identifying a pixel signal for which noise is to be corrected among a plurality of pixels of the solid-state image pickup device and using, as input, an image acquired from the solid-state image pickup device and that has not been corrected based on the first correction information; and an inference unit configured to infer a pixel signal on which noise is superimposed by inputting an image corrected based on the first correction information to the learned model. ### Claim 14 An image pickup apparatus comprising: a solid-state image pickup device; and at least one processor or circuit configured to function as following units; a communication unit configured to acquire an inference result from an inference device that infers a pixel signal on which noise is superimposed by inputting an image corrected based on first correction information for identifying a pixel signal for which noise is to be corrected among a plurality of pixels of the solid-state image pickup device to a learned model generated by performing machine learning using, as supervised data, the first correction information and using as input, an image acquired from the solid-state image pickup device and that has not been corrected based on the first correction information. ### Claim 15 A learning device comprising at least one processor or circuit configured to function as following unit; a learning unit configured to generate a learned model by performing machine learning using, as supervised data, first correction information for identifying a pixel signal for which noise is to be corrected among a plurality of pixels of a solid-state image pickup device and using, as input, an image acquired from the solid-state image pickup device and that has not been corrected based on the first correction information.