# US 10,769,473 B2 · Image processing apparatus, image processing method, and non-transitory computer-readable storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,473 B2 - Document ID: US10769473B2 - Status: Granted - Kind: B2 - Application number: 16/100,915 - Filed: 2018-08-10 - Published: 2019-02-21 - Granted: 2020-09-08 - Earliest claimed: 2017-08-17 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Masaki Inaba - Web page: https://patentgrep.com/patent/US10769473B2 ## CPC - G06V 10/82: using neural networks - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06F 18/2414: {Smoothing the distance, e.g. radial basis function networks [RBFN]} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G06T 7/73: using feature-based methods - G06V 30/19147: {Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting} - G06V 30/1916: {Validation; Performance evaluation} - G06V 30/19173: {Classification techniques} - G06V 30/194: References adjustable by an adaptive method, e.g. learning - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A plurality of recognition positions each recognized by a recognizer as a position of a target object on an input image are acquired. At least one representative position is obtained by performing clustering for the plurality of recognition positions. The representative position is edited in accordance with an editing instruction from a user for the representative position. The input image and the representative position are saved as learning data to be used for learning of the recognizer. ## Claims ### Claim 1 An image processing apparatus comprising: an acquisition unit configured to acquire a plurality of recognition positions each recognized by a recognizer as a position of a target object on an input image; a calculation unit configured to obtain at least one representative position by performing clustering for the plurality of recognition positions; an editing unit configured to edit the representative position in accordance with an editing instruction from a user for the representative position; and a saving unit configured to save the input image and the representative position as learning data to be used for learning of the recognizer. ### Claim 2 (depends on 1) The apparatus according to claim 1, wherein the acquisition unit acquires, as the plurality of recognition positions, a plurality of pixel positions each recognized by the recognizer as the position of the target object in pixel positions on the input image. ### Claim 3 (depends on 1) The apparatus according to claim 1, wherein when an output from the recognizer to which an image in an image region including a pixel position on the input image is input represents that the target object is included in the image region, the acquisition unit acquires the pixel position as the recognition position. ### Claim 4 (depends on 1) The apparatus according to claim 1, wherein the calculation unit classifies the plurality of recognition positions into a predetermined number of classes by performing clustering for the plurality of recognition positions, and obtains a representative position corresponding to each class based on recognition positions belonging to the class. ### Claim 5 (depends on 4) The apparatus according to claim 4, wherein the calculation unit obtains, as the representative position corresponding to the class, an average position of the recognition positions belonging to the class, a median of the recognition positions belonging to the class, or a peak position formed by applying the recognition positions belonging to the class to a Gaussian distribution. ### Claim 6 (depends on 4) The apparatus according to claim 4, further comprising a computation unit configured to obtain a reliability for the class, wherein the editing unit operates when the number of classes whose reliabilities are less than a threshold becomes not less than a predetermined number or when a ratio of the number of classes whose reliabilities are less than the threshold to the total number of classes becomes not less than a predetermined number. ### Claim 7 (depends on 6) The apparatus according to claim 6, wherein the computation unit obtains the reliability for the class in accordance with the number of recognition positions belonging to the class. ### Claim 8 (depends on 6) The apparatus according to claim 6, wherein the computation unit obtains the reliability for the class in accordance with an output of the recognizer corresponding to a recognition position belonging to the class. ### Claim 9 (depends on 6) The apparatus according to claim 6, wherein the computation unit obtains the reliability for the class in accordance with a distance between a recognition position belonging to the class and a representative position corresponding to the class. ### Claim 10 (depends on 1) The apparatus according to claim 1, wherein the editing unit displays information representing the representative position, which is superimposed on the input image, and accepts an editing operation from the user for the representative position. ### Claim 11 (depends on 6) The apparatus according to claim 6, wherein the editing unit displays information representing a representative position of a class whose reliability is not less than a threshold, which is superimposed on the input image, and accepts an editing operation from the user for the representative position. ### Claim 12 (depends on 1) The apparatus according to claim 1, wherein the editing unit moves the representative position in accordance with a moving instruction from the user for the representative position, deletes the representative position in accordance with a deletion instruction from the user for the representative position, and adds a representative position in accordance with an instruction from the user to add a new representative position. ### Claim 13 (depends on 1) The apparatus according to claim 1, further comprising a learning unit configured to perform the learning of the recognizer using the learning data saved by the saving unit. ### Claim 14 (depends on 13) The apparatus according to claim 13, further comprising an updating unit configured to update the recognizer to the recognizer that has undergone the learning. ### Claim 15 (depends on 14) The apparatus according to claim 14, wherein when an evaluation value for the recognizer before the learning is higher than an evaluation value for the recognizer after the learning, the updating unit updates the recognizer before the learning to the recognizer after the learning. ### Claim 16 An image processing apparatus comprising: an acquisition unit configured to acquire a plurality of recognition positions/orientations each recognized by a recognizer as a position/orientation of a target object on an input image; a calculation unit configured to obtain at least one representative position/orientation by performing clustering for the plurality of recognition positions/orientations; an editing unit configured to edit the representative position/orientation in accordance with an editing instruction from a user for the representative position/orientation; and a saving unit configured to save the input image and the representative position/orientation as learning data to be used for learning of the recognizer. ### Claim 17 An image processing method comprising: acquiring a plurality of recognition positions each recognized by a recognizer as a position of a target object on an input image; obtaining at least one representative position by performing clustering for the plurality of recognition positions; editing the representative position in accordance with an editing instruction from a user for the representative position; and saving the input image and the representative position as learning data to be used for learning of the recognizer. ### Claim 18 An image processing method comprising: acquiring a plurality of recognition positions/orientations each recognized by a recognizer as a position/orientation of a target object on an input image; obtaining at least one representative position/orientation by performing clustering for the plurality of recognition positions/orientations; editing the representative position/orientation in accordance with an editing instruction from a user for the representative position/orientation; and saving the input image and the representative position/orientation as learning data to be used for learning of the recognizer. ### Claim 19 A non-transitory computer-readable storage medium storing a computer program configured to cause a computer to function as: an acquisition unit configured to acquire a plurality of recognition positions each recognized by a recognizer as a position of a target object on an input image; a calculation unit configured to obtain at least one representative position by performing clustering for the plurality of recognition positions; an editing unit configured to edit the representative position in accordance with an editing instruction from a user for the representative position; and a saving unit configured to save the input image and the representative position as learning data to be used for learning of the recognizer. ### Claim 20 A non-transitory computer-readable storage medium storing a computer program configured to cause a computer to function as: an acquisition unit configured to acquire a plurality of recognition positions/orientations each recognized by a recognizer as a position/orientation of a target object on an input image; a calculation unit configured to obtain at least one representative position/orientation by performing clustering for the plurality of recognition positions/orientations; an editing unit configured to edit the representative position/orientation in accordance with an editing instruction from a user for the representative position/orientation; and a saving unit configured to save the input image and the representative position/orientation as learning data to be used for learning of the recognizer.