# US 10,769,794 B2 · Multi-sensor object recognition system and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,794 B2 - Document ID: US10769794B2 - Status: Granted - Kind: B2 - Application number: 16/692,466 - Filed: 2019-11-22 - Published: 2020-03-19 - Granted: 2020-09-08 - Earliest claimed: 2017-09-07 - Assignees: Symbol Technologies, Inc. - Applicants: Symbol Technologies, LLC - Inventors: Joseph Lam; Xinyi Gong - Web page: https://patentgrep.com/patent/US10769794B2 ## CPC - G06T 7/136: involving thresholding - G06T 7/12: Edge-based segmentation - G06T 7/13: Edge detection - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 20/00: Scenes; Scene-specific elements (control of digital cameras H04N23/60) - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30241: Trajectory - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method of object detection includes obtaining a set of images depicting overlapping regions of an area containing a plurality of objects. Each image includes input object indicators defined by input bounding boxes, input confidence level values, and object identifiers. The method includes identifying candidate subsets of input object indicators in adjacent images. Each candidate subset has input overlapping bounding boxes in a common frame of reference, and a common object identifier. The method includes adjusting the input confidence levels for each input object indicator in the candidate subsets; selecting clusters of the input object indicators satisfying a minimum input confidence threshold, having a common object identifier, and having a degree of overlap satisfying a predefined threshold; and detecting an object by generating a single output object indicator for each cluster, the output object indicator having an output bounding box, an output confidence level value, and the common object identifier. ## Claims ### Claim 1 A method of object detection in an imaging controller, comprising: obtaining a set of images depicting overlapping regions of an area containing a plurality of objects, each of the set of images including a plurality of input object indicators; identifying candidate subsets of input object indicators in adjacent ones of the set of images, each candidate subset having input bounding boxes that overlap in a common frame of reference, and a common object identifier; adjusting input confidence level values upwards from initial values for each input object indicator in the candidate subsets; selecting clusters of the input object indicators, the input object indicators of each cluster satisfying a minimum input confidence threshold, having a common object identifier, and having a degree of overlap that satisfies a predefined threshold; and detecting an object by generating a single output object indicator for each cluster. ### Claim 2 (depends on 1) The method of claim 1, wherein identifying the candidate subsets includes determining whether at least two input object indicators have a degree of overlap satisfying an adjacency overlap threshold. ### Claim 3 (depends on 1) The method of claim 1, wherein the area contains a plurality of shelves supporting the objects; the set of images depicting a vertical segment of the shelves. ### Claim 4 (depends on 1) The method of claim 1, wherein adjusting the input confidence level values for each object indicator in the candidate subsets includes: determining a degree of occlusion for each input object indicator in a candidate subset based on a comparison between the input bounding box and a reference bounding box corresponding to the object identifier of the input object indicator; and adjusting the input confidence level based on the degree of occlusion. ### Claim 5 (depends on 1) The method of claim 1, further comprising: adjusting the input bounding boxes for each input object indicator in the candidate sub sets. ### Claim 6 (depends on 5) The method of claim 5, wherein adjusting the input bounding boxes includes replacing the input bounding box for each input object in the subset with a reference bounding box corresponding to the common input object identifier. ### Claim 7 (depends on 1) The method of claim 1, wherein selecting the clusters includes: generating a ranked list of each of the input object indicators based on the input confidence level values; and beginning at the highest-ranked input object indicator, determining a degree of overlap for successive pairs of the input bounding boxes. ### Claim 8 (depends on 7) The method of claim 7, wherein determining the degree of overlap includes determining a ratio of the intersection of each successive pair of the input bounding boxes to the union of the pair of the input bounding boxes. ### Claim 9 (depends on 7) The method of claim 7, further comprising: obtaining a further set of images depicting overlapping regions of an adjacent area, the further set of images registered to the common frame of reference; and repeating the identifying candidate subsets, the adjusting, the selecting clusters, and the generating output object indicators, wherein generating the ranked list includes inserting the output object indicators into the ranked list. ### Claim 10 (depends on 1) The method of claim 1 wherein the plurality of input object indicators are defined by respective (i) input bounding boxes, (ii) input confidence level values, and (iii) object identifiers. ### Claim 11 (depends on 1) The method of claim 1 wherein the output object indicator includes (i) an output bounding box, (ii) an output confidence level value, and (iii) the common object identifier. ### Claim 12 A computing device for detecting objects, the computing device comprising: a memory; an imaging controller comprising: an image preprocessor configured to obtain a set of images from the memory depicting overlapping regions of an area containing a plurality of objects, each of the set of images including a plurality of input object indicators; a subset detector configured to identify candidate subsets of input object indicators in adjacent ones of the set of images, each candidate subset having input bounding boxes that overlap in a common frame of reference, and a common object identifier; the subset detector further configured to adjust input confidence level values upwards from initial values for each input object indicator in the candidate subsets; a cluster detector configured to select clusters of the input object indicators, the input object indicators of each cluster satisfying a minimum input confidence threshold, having a common object identifier, and having a degree of overlap that satisfies a predefined threshold; and an output generator configured to detect an object by generating a single output object indicator for each cluster. ### Claim 13 (depends on 12) The computing device of claim 12, wherein the subset detector is further configured to identify the candidate subsets by determining whether at least two input object indicators have a degree of overlap satisfying an adjacency overlap threshold. ### Claim 14 (depends on 12) The computing device of claim 12, wherein the area contains a plurality of shelves supporting the objects; the set of images depicting a vertical segment of the shelves. ### Claim 15 (depends on 12) The computing device of claim 12, wherein the subset detector is further configured to adjust the input confidence level values for each object indicator in the candidate subsets by: determining a degree of occlusion for each input object indicator in a candidate subset based on a comparison between the input bounding box and a reference bounding box corresponding to the object identifier of the input object indicator; and adjusting the input confidence level based on the degree of occlusion. ### Claim 16 (depends on 12) The computing device of claim 12, wherein the subset detector is further configured to adjust the input bounding boxes for each input object indicator in the candidate sub sets. ### Claim 17 (depends on 16) The computing device of claim 16, wherein adjusting the input bounding boxes includes replacing the input bounding box for each input object in the subset with a reference bounding box corresponding to the common input object identifier. ### Claim 18 (depends on 12) The computing device of claim 12, wherein the cluster detector is configured to select the clusters by: generating a ranked list of each of the input object indicators based on the input confidence level values; and beginning at the highest-ranked input object indicator, determining a degree of overlap for successive pairs of the input bounding boxes. ### Claim 19 (depends on 18) The computing device of claim 18, wherein the cluster detector is further configured to determine the degree of overlap by determining a ratio of the intersection of each successive pair of the input bounding boxes to the union of the pair of the input bounding boxes. ### Claim 20 (depends on 18) The computing device of claim 18, wherein the imaging controller is further configured to: obtain a further set of images depicting overlapping regions of an adjacent area, the further set of images registered to the common frame of reference; and repeat the identifying candidate subsets, the adjusting, the selecting clusters, and the generating output object indicators, wherein generating the ranked list includes inserting the output object indicators into the ranked list. ### Claim 21 A non-transitory computer readable storage medium containing a plurality of computer readable instructions executable by an imaging controller to configure the imaging controller to perform a method of object detection comprising: obtaining a set of images depicting overlapping regions of an area containing a plurality of objects, each of the set of images including a plurality of input object indicators; identifying candidate subsets of input object indicators in adjacent ones of the set of images, each candidate subset having input bounding boxes that overlap in a common frame of reference, and a common object identifier; adjusting input confidence level values upwards from initial values for each input object indicator in the candidate subsets; selecting clusters of the input object indicators, the input object indicators of each cluster satisfying a minimum input confidence threshold, having a common object identifier, and having a degree of overlap that satisfies a predefined threshold; and detecting an object by generating a single output object indicator for each cluster.