# US 12,002,217 B1 · Detection box determination based on pixel clustering > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,002,217 B1 - Document ID: US12002217B1 - Status: Granted - Kind: B1 - Application number: 17/509,121 - Filed: 2021-10-25 - Granted: 2024-06-04 - Earliest claimed: 2021-10-25 - Assignees: Zoox, Inc. - Applicants: Zoox, Inc. - Inventors: Stanley Lilian Volta; Allan Zelener - Web page: https://patentgrep.com/patent/US12002217B1 ## CPC - G06T 7/181: involving edge growing; involving edge linking - G06T 7/13: Edge detection - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion - G06V 10/457: {by analysing connectivity, e.g. edge linking, connected component analysis or slices} - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T 2210/12: Bounding box - G06V 2201/07: Target detection - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Techniques for determining detection boxes representing objects in an environment using pixel clustering are disclosed herein. Autonomous vehicle sensors can capture data in an environment that may include separate objects, such as large and/or articulated vehicles. In an example, the data can include a plurality of pixels that are associated with a large object. A vehicle computing system can generate a detection box for each pixel of the plurality of pixels and can determine at least one averaged detection box representing the large object. In an example, the vehicle computing system can control the vehicle based in part on the at least one averaged detection boxes. ## Claims ### Claim 1 A system comprising: one or more processors; and one or more computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising: receiving pixel data for a plurality of pixels corresponding to an object in an environment, wherein the pixel data comprises object classification confidence values and detection box parameters associated with individual pixels of the plurality of pixels; identifying a subset of the plurality of pixels associated with the object classification confidence values above a threshold confidence value; generating, based at least in part on a detection box parameter, a cluster of pixels comprising a first pixel of the subset of the plurality of pixels and a second pixel of the subset of the plurality of pixels; generating an averaged detection box based at least in part on a first detection box parameter associated with the first pixel and a second detection box parameter associated with the second pixel; and controlling a vehicle based at least in part on the averaged detection box. ### Claim 2 (depends on 1) The system of claim 1, wherein: the pixel data comprises classifications associated with the individual pixels of the plurality of pixels; and identifying the subset of the plurality of pixels comprises identifying the subset of the plurality of pixels associated with classifications that are a same classification. ### Claim 3 (depends on 1) The system of claim 1, wherein the detection box parameter comprises at least one of: a yaw value; detection box center coordinates; a detection box center offset distance; a detection box center offset direction; or detection box dimensions. ### Claim 4 (depends on 1) The system of claim 1, wherein: the pixel data comprises object classifications associated with the individual pixels of the plurality of pixels; and generating the cluster of pixels is further based at least in part on the object classifications associated with individual pixels of the subset of the plurality of pixels. ### Claim 5 (depends on 1) The system of claim 1, wherein: the pixel data comprises pixel locations in a grid associated with the individual pixels of the plurality of pixels; and generating the cluster of pixels is further based at least in part on the pixel locations associated with individual pixels of the subset of the plurality of pixels. ### Claim 6 (depends on 5) The system of claim 5, wherein generating the cluster of pixels is further based at least in part on determining that the pixel locations share an edge in the grid. ### Claim 7 A method comprising: receiving pixel data for a plurality of pixels corresponding to an object in an environment, wherein the pixel data comprises confidence values and detection box parameters associated with individual pixels of the plurality of pixels; identifying a subset of the plurality of pixels based at least in part on the confidence values; generating, based at least in part on a detection box parameter, a cluster of pixels comprising a first pixel of the subset of the plurality of pixels and a second pixel of the subset of the plurality of pixels; generating an averaged detection box based at least in part on a first detection box parameter associated with the first pixel and a second detection box parameter associated with the second pixel; and controlling a vehicle based at least in part on the averaged detection box. ### Claim 8 (depends on 7) The method of claim 7, wherein generating the cluster of pixels comprises determining, based at least in part on the detection box parameters, that detection boxes associated with the individual pixels of the cluster of pixels overlap. ### Claim 9 (depends on 8) The method of claim 8, wherein determining that the detection boxes overlap comprises determining an intersection over union (IoU) value for the detection boxes. ### Claim 10 (depends on 7) The method of claim 7, wherein controlling the vehicle based at least in part on the averaged detection box comprises: determining that a number of pixels associated with the cluster of pixels exceeds a threshold number of pixels; and in response to determining that the number of pixels associated with the cluster of pixels exceeds the threshold number of pixels, controlling the vehicle based at least in part on the averaged detection box. ### Claim 11 (depends on 7) The method of claim 7, wherein: the confidence values associated with the individual pixels of the plurality of pixels comprise detection confidence values; and generating the cluster of pixels is further based at least in part on the detection confidence values. ### Claim 12 (depends on 7) The method of claim 7, wherein generating the cluster of pixels comprises: determining, based at least in part on the detection box parameter, detection box orientations for the individual pixels of the plurality of pixels; and generating the cluster of pixels is further based at least in part on the detection box orientations. ### Claim 13 (depends on 7) The method of claim 7, wherein generating the cluster of pixels comprises determining, based at least in part on the detection box parameter, that the individual pixels of the cluster of pixels share an edge in a location grid. ### Claim 14 (depends on 7) The method of claim 7, wherein the detection box parameter comprises at least one of: a yaw value; detection box center coordinates; a detection box center offset distance; a detection box center offset direction; or detection box dimensions. ### Claim 15 One or more computer-readable media storing instructions that, when executed by one or more processors, perform operations comprising: receiving pixel data for a plurality of pixels associated with an object in an environment, wherein the pixel data comprises detection box parameters associated with individual pixels of the plurality of pixels; identifying a subset of the plurality of pixels based at least in part on the pixel data; generating, based at least in part on a detection box parameter, a cluster of pixels comprising a first pixel of the subset of the plurality of pixels and a second pixel of the subset of the plurality of pixels; generating an averaged detection box based at least in part on a first detection box parameter associated with the first pixel and a second detection box parameter associated with the second pixel; and controlling a vehicle based at least in part on the averaged detection box. ### Claim 16 (depends on 15) The one or more computer-readable media of claim 15, wherein: the detection box parameter comprises a detection box yaw value; and generating the cluster of pixels from the subset of the plurality of pixels comprises determining that detection box yaw values for individual pixels of the cluster of pixels are within a range of yaw values. ### Claim 17 (depends on 15) The one or more computer-readable media of claim 15, wherein: the pixel data comprises object classifications associated with the individual pixels of the plurality of pixels; and identifying the subset of the plurality of pixels is further based at least in part on the object classifications associated with the individual pixels of the plurality of pixels. ### Claim 18 (depends on 15) The one or more computer-readable media of claim 15, wherein: the pixel data comprises object classifications associated with the individual pixels of the plurality of pixels; and generating the cluster of pixels is further based at least in part on the object classifications associated with the individual pixels of the plurality of pixels. ### Claim 19 (depends on 15) The one or more computer-readable media of claim 15, wherein: the pixel data comprises pixel locations associated with the individual pixels of the plurality of pixels; and generating the cluster of pixels is further based at least in part on the pixel locations associated with the individual pixels of the plurality of pixels. ### Claim 20 (depends on 15) The one or more computer-readable media of claim 15, wherein: the detection box parameter comprises a detection box center location; and generating the cluster of pixels comprises determining that detection box center locations for individual pixels of the cluster of pixels are within a range of locations.