# US 12,112,518 B2 · Object detection device, learning method, and recording medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,518 B2 - Document ID: US12112518B2 - Status: Granted - Kind: B2 - Application number: 17/768,923 - Filed: 2019-11-08 - Published: 2024-03-28 - Granted: 2024-10-08 - Earliest claimed: 2019-11-08 - Assignees: NEC Corporation - Applicants: NEC Corporation - Inventors: Katsuhiko Takahashi; Yuichi Nakatani; Tetsuo Inoshita; Asuka Ishii; Gaku Nakano - Web page: https://patentgrep.com/patent/US12112518B2 ## CPC - G06V 10/759: {Region-based matching} - G06V 10/82: using neural networks - G06T 7/00: Image analysis - G06V 10/809: {of classification results, e.g. where the classifiers operate on the same input data} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract In the object detection device, the plurality of object detection units output a score indicating a probability that a predetermined object exists, for each partial region set with respect to image data inputted. The weight computation unit computes a weight for each of the plurality of object detection units by using weight computation parameters and based on the image data. The weights are used when the scores outputted by the plurality of object detection units are merged. The weight redistribution unit changes the weight for a predetermined object detection unit, among the weights computed by the weight computation unit, to 0 and output the weights. The merging unit merges the scores outputted by the plurality of object detection units for each of the partial regions, by using the weights computed by the weight computation unit and including the weight changed by the weight redistribution unit. The loss computation unit computes a difference between a ground truth label of the image data and the merged score merged by the merging unit as a loss. Then, the parameter correction unit corrects the weight computation parameters so as to reduce the loss. ## Claims ### Claim 1 An object detection device comprising: a memory configured to store instructions; and one or more processors configured to execute the instructions to: output, by a plurality of object detection units, a score indicating a probability that a predetermined object exists, for each partial region set with respect to image data inputted; compute a weight for each of the plurality of object detection units by using weight computation parameters and based on the image data, the weight being used when the scores outputted by the plurality of object detection units are merged; to change the weight for a predetermined object detection unit, among the computed weights, to 0 and output the weights; merge the scores outputted by the plurality of object detection units for each of the partial regions, by using the computed weights and including the changed weight; compute a difference between a ground truth label of the image data and the merged score as a loss; and correct the weight computation parameters so as to reduce the loss. ### Claim 2 (depends on 1) The object detection device according to claim 1, wherein the one or more processors are configured to normalize the weights which has not been changed to 0, among the computed weights, so that the sum of the weights becomes 1, and output the weights. ### Claim 3 (depends on 1) The object detection device according to claim 1, wherein the one or more processors are configured to perform redistribution of the weights with a predetermined probability. ### Claim 4 (depends on 1) The object detection device according to claim 1, wherein the one or more processors are configured to change the weight, to 0, for the object detection unit that is learned and constructed using the image data or a part of the image data inputted when the object detection device is learned. ### Claim 5 (depends on 1) The object detection device according to claim 1, wherein the one or more processors are configured to compute a single weight with respect to the image data as a whole, and wherein the one or more processors are configured to merge the scores outputted by the plurality of object detection units by using the single weight. ### Claim 6 (depends on 1) The object detection device according to claim 1, wherein the one or more processors are configured to compute the weight for each partial region of the image data, and wherein the one or more processors are configured to merge the scores outputted by the plurality of object detection units by using the weight computed for each partial region. ### Claim 7 (depends on 1) The object detection device according to claim 1, wherein the one or more processors are configured to compute the weight for each class indicating the object, and wherein the one or more processors are configured to merge the scores outputted by the plurality of object detection units by using the weight computed for each class. ### Claim 8 (depends on 1) The object detection device according to claim 1, wherein the one or more processors are configured to multiply the scores outputted by the plurality of object detection units by the computed weight for each object detection unit, add the multiplied scores together, and calculate an average value of the multiplied scores. ### Claim 9 (depends on 1) The object detection device according to claim 1, wherein the plurality of object detection units are configured to output coordinate information about a rectangular region where the object exists for each partial region, wherein the one or more processors are configured to merge the coordinate information about the rectangular region where the object exists by using the computed weight, and wherein the one or more processors are configured to compute a loss including a difference between the ground truth label and the coordinate information merged. ### Claim 10 (depends on 9) The object detection device according to claim 9, wherein the one or more processors are configured to multiply the coordinate information outputted by the plurality of object detection units by the computed weight for each object detection unit, add the multiplied scores together, and calculate an average value of the multiplied scores. ### Claim 11 (depends on 1) The object detection device according to claim 1, wherein the one or more processors are configured to use shooting environment prediction parameters to predict a shooting environment of the image data, and output predicted environment information, wherein the one or more processors are configured to compute a shooting environment prediction loss on a basis of shooting environment information about the image data prepared in advance and the predicted environment information, and wherein the one or more processors are configured to correct the shooting environment prediction parameters so as to reduce the prediction loss. ### Claim 12 (depends on 11) The object detection device according to claim 11, wherein the one or more processors are provided with a first network including the weight computation parameters and a second network including the shooting environment prediction parameters, and wherein the first network and the second network have a portion shared in common. ### Claim 13 An object detection device comprising: a memory configured to store instructions; and one or more processors configured to execute the instructions to: output, by a plurality of object detection units, a score indicating a probability that a predetermined object exists, for each partial region set with respect to image data inputted; compute weights used for merging the scores outputted by the plurality of object detection units by using weight computation parameters and based on the image data; change the weight for a predetermined object detection unit, among the computed weights, to 0 and output the weights; merge the scores outputted by the plurality of object detection units for each of the partial regions, by using the computed weights and including the changed weight; output, by an object detection unit of a target model, a score indicating the probability that the predetermined object exists, for each partial region set with respect to the image data; compute a loss indicating a difference of the output of the object detection unit of the target model from a ground truth label of the image data and the merged score; and correct the parameters of the object detection unit of the target model so as to reduce the loss. ### Claim 14 A learning method of an object detection device, comprising: outputting, by a plurality of object detection units, a score indicating a probability that a predetermined object exists, for each partial region set with respect to image data inputted; computing a weight for each of the plurality of object detection units by using weight computation parameters and based on the image data, the weight being used when the scores outputted by the plurality of object detection units are merged; changing the weight for a predetermined object detection unit, among the computed weights, to 0 and outputting the weights; merging the scores outputted by the plurality of object detection units for each of the partial regions, by using the computed weights including the changed weight; computing a difference between a ground truth label of the image data and the merged score as a loss; and correcting the weight computation parameters so as to reduce the loss. ### Claim 15 (depends on 14) A non-transitory computer-readable recording medium having recorded thereon instructions executable by a processor to perform the method of claim 14. ### Claim 16 A learning method of an object detection device, comprising: outputting, by a plurality of object detection units, a score indicating a probability that a predetermined object exists, for each partial region set with respect to image data inputted; computing weights used for merging the scores outputted by the plurality of object detection units by using weight computation parameters and based on the image data; changing the weight for a predetermined object detection unit, among the computed weights, to 0 and outputting the weights; merging the scores outputted by the plurality of object detection units for each of the partial regions, by using the computed weights including the changed weight; outputting, by an object detection unit of a target model, a score indicating the probability that the predetermined object exists, for each partial region set with respect to the image data; computing a loss indicating a difference of the output of the object detection unit of the target model from a ground truth label of the image data and the merged score; and correcting the parameters of the object detection unit of the target model so as to reduce the loss. ### Claim 17 (depends on 16) A non-transitory computer-readable recording medium having recorded thereon instructions executable by a processor to perform the method of claim 16.