# US 12,594,934 B2 · Equidistant-temporal aggregation for moving object segmentation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,594,934 B2 - Document ID: US12594934B2 - Status: Granted - Kind: B2 - Application number: 18/340,361 - Filed: 2023-06-23 - Published: 2024-12-26 - Granted: 2026-04-07 - Earliest claimed: 2023-06-23 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Ming-Yuan Yu; Varun Ravi Kumar; Senthil Kumar Yogamani - Web page: https://patentgrep.com/patent/US12594934B2 ## CPC - B60W 30/0956: {the prediction being responsive to traffic or environmental parameters} - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215) - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08) - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2420/408: Radar; Laser, e.g. lidar - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/20221: Image fusion; Image merging - G06T 2207/30241: Trajectory - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract A system for navigation store, in a time-based buffer, a first set of frames acquired by a sensor; stores, in a distance-based buffer, a second set of frames acquired by the sensor; performs moving object segmentation on the first set of frames and the second set of frames to identify at least one moving object in a scene of the frames; predicts a trajectory of the at least one moving object; and performs a navigation function based on the predicted trajectory. ## Claims ### Claim 1 An apparatus for navigation, the apparatus comprising: a memory configured to store a time-based buffer with a first set of frames acquired from a sensor and a distance-based buffer with a second set of frames acquired from the sensor; one or more processors implemented in circuitry, in communication with the memory, and configured to: determine that the sensor has moved a threshold unit of distance; add a new frame acquired from the sensor to the second set of frames in response to determining that the sensor has moved the threshold unit of distance; perform moving object segmentation on the first set of frames and the second set of frames to identify at least one moving object in a scene of the first set of frames and the second set of frames; predict a trajectory of the at least one moving object; and perform a navigation function based on the predicted trajectory. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the one or more processors are further configured to: in response to adding the new frame to the distance-based buffer, remove an oldest frame from the distance-based buffer. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the one or more processors are configured to dynamically determine the threshold unit of distance based on a speed at which the sensor is moving. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the one or more processors are configured to dynamically determine the threshold unit of distance based a GPS location of the sensor. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the one or more processors are further configured to: add new frames to the time-based buffer at a fixed frequency. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the one or more processors are further configured to: determine a time to collision based on the trajectory of the at least one moving object and a trajectory of the sensor; and perform the navigation function based on the time to collision. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the sensor comprises an image acquiring device, and wherein the first set of frames and the second set of frames comprises images acquired by the image acquiring device. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the sensor comprises a point cloud acquiring device, and wherein the first set of frames comprises a first set of point clouds and the second set of frames comprises a second set of point clouds. ### Claim 9 (depends on 8) The apparatus of claim 8, wherein the point cloud acquiring device comprises a LiDAR sensor configured to capture point clouds, and wherein the one or more processors are part of an advanced driver assistance system (ADAS). ### Claim 10 (depends on 8) The apparatus of claim 8, wherein, the memory is further configured to store a second time-based buffer with a first set of images acquired from an image acquiring device and a second distance-based buffer with a second set of images acquired from the image acquiring device; and the one or more processors are configured to perform the moving object segmentation on the first set of point clouds, the second set of point clouds, the first set of images, and the second set of images to identify the at least one moving object in the scene of the point clouds. ### Claim 11 (depends on 10) The apparatus of claim 10, wherein the one or more processors are further configured to: add a new image acquired by the image acquiring device to the second distance-based buffer in response to the image acquiring device having moved a second threshold unit of distance. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein the one or more processors are configured to dynamically determine the second threshold unit of distance based on a speed at which the image acquiring device is moving. ### Claim 13 (depends on 10) The apparatus of claim 10, wherein the one or more processors are further configured to: add new images to the second time-based buffer at a fixed frequency. ### Claim 14 (depends on 10) The apparatus of claim 10, wherein the image acquiring device comprises a camera. ### Claim 15 (depends on 1) The apparatus of claim 1, wherein the one or more processors are external to an advanced driver assistance system (ADAS). ### Claim 16 A method for navigation, the method comprising: storing, in a time-based buffer, a first set of frames acquired by a sensor; storing, in a distance-based buffer, a second set of frames acquired by the sensor; determining that the sensor has moved a threshold unit of distance; adding a new frame acquired from the sensor to the second set of frames in response to determining that the sensor has moved the threshold unit of distance; performing moving object segmentation on the first set of frames and the second set of frames to identify at least one moving object in a scene of the first set of frames and the second set of frames; predicting a trajectory of the at least one moving object; and performing a navigation function based on the predicted trajectory. ### Claim 17 (depends on 16) The method of claim 16, further comprising: in response to adding the new frame to the distance-based buffer, removing an oldest frame from the distance-based buffer. ### Claim 18 (depends on 16) The method of claim 16, further comprising: adding new frames to the time-based buffer at a fixed frequency. ### Claim 19 (depends on 16) The method of claim 16, further comprising: determining a time to collision based on the trajectory of the at least one moving object and a trajectory of the sensor; and performing the navigation function based on the time to collision. ### Claim 20 (depends on 16) The method of claim 16, wherein the sensor comprises an image acquiring device and wherein the frames comprises images acquired by the image acquiring device. ### Claim 21 (depends on 16) The method of claim 16, wherein the sensor comprises a point cloud acquiring device, wherein the first set of frames comprises a first set of point clouds and the second set of frames comprises a second set of point clouds. ### Claim 22 (depends on 21) The method of claim 21, further comprising: storing a second time-based buffer with a first set of images acquired by an image acquiring device; storing a second distance-based buffer with a second set of images acquired by the image acquiring device; and performing the moving object segmentation on the first set of frames, the second set of frames, the first set of images, and the second set of images to identify the at least one moving object in the scene of the point clouds. ### Claim 23 (depends on 22) The method of claim 22, further comprising: adding a new image acquired by the image acquiring device to the second distance-based buffer in response to the image acquiring device having moved a second threshold unit of distance. ### Claim 24 (depends on 23) The method of claim 23, further comprising: dynamically determining the second threshold unit of distance based on a speed of the image acquiring device. ### Claim 25 (depends on 22) The method of claim 22, further comprising: adding new images to the second time-based buffer at a fixed frequency. ### Claim 26 A non-transitory computer-readable storage medium storing instructions that when executed by one or more processors cause the one or more processors to: store, in a time-based buffer, a first set of frames acquired by a sensor; store, in a distance-based buffer, a second set of frames acquired by the sensor; determine that the sensor has moved a threshold unit of distance; add a new frame acquired from the sensor to the second set of frames in response to determining that the sensor has moved the threshold unit of distance; perform moving object segmentation on the first set of frames and the second set of frames to identify at least one moving object in a scene of the first set of frames and the second set of frames; predict a trajectory of the at least one moving object; and perform a navigation function based on the predicted trajectory. ### Claim 27 (depends on 26) The non-transitory computer-readable storage medium of claim 26, storing instructions that when executed by the one or more processors cause the one or more processors to: add new frames to the time-based buffer at a fixed frequency. ### Claim 28 (depends on 1) The apparatus of claim 1, wherein the one or more processors are further configured to: determine a frame rate based on a state of a vehicle to which the sensor is mounted, wherein the frame rate defines a number of frames per a unit of distance travelled by the sensor; and add new frames, acquired from the sensor, to the distance-based buffer at the frame rate. ### Claim 29 (depends on 16) The method of claim 16, further comprising: determining a frame rate based on a state of a vehicle to which the sensor is mounted, wherein the frame rate defines a number of frames per a unit of distance travelled by the sensor; and adding new frames, acquired from the sensor, to the distance-based buffer at the frame rate. ### Claim 30 (depends on 26) The non-transitory computer-readable storage medium of claim 26, storing instructions that when executed by the one or more processors cause the one or more processors to: determine a frame rate based on a state of a vehicle to which the sensor is mounted, wherein the frame rate defines a number of frames per a unit of distance travelled by the sensor; and add new frames, acquired from the sensor, to the distance-based buffer at the frame rate.