# US 11,897,502 B2 · Object tracking by generating velocity grids > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,897,502 B2 - Document ID: US11897502B2 - Status: Granted - Kind: B2 - Application number: 17/219,757 - Filed: 2021-03-31 - Published: 2022-10-06 - Granted: 2024-02-13 - Earliest claimed: 2021-03-31 - Assignees: GM Cruise Holdings LLC - Applicants: GM Cruise Holdings LLC - Inventors: Richard Stenson; Minhao Xu - Web page: https://patentgrep.com/patent/US11897502B2 ## CPC - B60W 60/0011: {involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles} - G01S 17/006: {Theoretical aspects} - B60W 60/001: {Planning or execution of driving tasks} - G01S 17/42: Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46) - G01S 17/58: Velocity or trajectory determination systems; Sense-of-movement determination systems - G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching} - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2420/408: Radar; Laser, e.g. lidar - B60W 2554/4049: Relationship among other objects, e.g. converging dynamic objects - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A computed-implemented method is provided for creating a velocity grid using a simulated environment for use by an autonomous vehicle. The method may include simulating a road scenario with simulated objects, collecting first LiDAR data from simulated LiDAR sensors in the simulated road scenario, wherein the collected first LiDAR data comprises a first plurality of points that are representative of a first simulated object at a first 3D location and a first time. The method may also include transforming the first plurality of points from a simulated-scene frame-of-reference to a first simulated object frame-of-reference, and simulating the first simulated object to move from the first 3D location to a second 3D location within the simulated road scenario between the first time and a second time. ## Claims ### Claim 1 A computed-implemented method for creating a velocity grid using a simulated environment for use by an autonomous vehicle, the method comprising: simulating a road scenario with simulated objects; collecting first LiDAR data from simulated LiDAR sensors in the simulated road scenario, wherein the collected first LiDAR data comprises a first plurality of points that are representative of a first simulated object at a first three-dimensional (3D) location and a first time, a first point representative of a portion of the first simulated object; transforming the first plurality of points from a simulated-scene frame-of-reference to a first simulated object frame-of-reference; simulating the first simulated object to move from the first 3D location to a second 3D location within the simulated road scenario between the first time and a second time; collecting second LiDAR data from the simulated LiDAR sensors in the simulated road scenario at the second time, wherein the collected second LiDAR data comprises a second plurality of points representative of the first simulated object at the second 3D location and a second time, a second point representative of a portion of the first simulated object; transforming the second plurality of points from the first simulated object frame-of-reference back to the simulated-scene frame-of-reference; determining a distance between the first point representative of a portion of the first simulated object and the second point representative of a portion of the first simulated object during the simulated movement of the first simulated object calculating the velocity of the portion on the first simulated object in the simulated-scene frame-of-reference during a simulated movement of the first simulated object from the first 3D location to the second 3D location based upon the determined distance; and encoding the velocity in association with the first point prior to the simulated movement of the simulated object with respect to a second point after the simulated movement of the first simulated object in a 3D velocity grid for the simulated objects. ### Claim 2 (depends on 1) The computed-implemented method of claim 1, the collecting first LiDAR data further comprising: capturing light rays that simulate the LiDAR sensors; and determining a first intersection between the light rays and the first simulated object at the first three-dimensional (3D) location in the simulated road scenario, wherein the first intersection defines the first point in the plurality of points representative of the first simulated object at the first 3D location. ### Claim 3 (depends on 1) The computed-implemented method of claim 1, further comprising identifying the second point in the second plurality of points, the second point mapping to the same portion of the first simulated object as the first point. ### Claim 4 (depends on 1) The computed-implemented method of claim 1, the collecting second LiDAR data further comprising: capturing light rays that simulate the LiDAR sensors; and determining a second intersection between the light rays and the first simulated object at the second three-dimensional (3D) location in the simulated road scenario. ### Claim 5 (depends on 4) The computed-implemented method of claim 4, wherein the second intersection defines the second point in the second plurality of points representative of the first simulated object at the second three-dimensional (3D) location. ### Claim 6 (depends on 1) The computed-implemented method of claim 1, wherein a relative position of the first point on the first simulated object frame-of-reference is maintained during the simulated movement of the first simulated object. ### Claim 7 (depends on 1) The computed-implemented method of claim 1, further comprising training the autonomous vehicle using a machine learning algorithm including the three-dimensional (3D) velocity grid for the simulated objects in the simulated environment. ### Claim 8 A system for creating a velocity grid using a simulated environment for use by an autonomous vehicle, the system comprising: one or more processors; a computer-readable medium comprising instructions stored therein, which when executed by one or more processors, cause the processors to: simulate a road scenario with simulated non-player characters, collect first LiDAR data from simulated LiDAR sensors in the simulated road scenario, wherein the collected first LiDAR data comprises a first plurality of points that are representative of a first simulated object at a first three-dimensional (3D) location and a first time, a first point representative of a portion of the first simulated object, transform the first plurality of points from a simulated-scene frame-of-reference to a first simulated object frame-of-reference, simulate the first simulated object to move from the first 3D location to a second 3D location within the simulated road scenario between the first time and a second time, collect second LiDAR data from the simulated LiDAR sensors in the simulated road scenario at the second time, wherein the collected second LiDAR data comprises a second plurality of points representative of the first simulated object at the second 3D location and a second time, a second point representative of a portion of the first simulated object, transform the second plurality of points from the first simulated object frame-of-reference back to the simulated-scene frame-of-reference, determine a distance between the first point representative of a portion of the first simulated object and the second point representative of a portion of the first simulated object during the simulated movement of the first simulated object calculate the velocity of the portion on the first simulated object in the simulated-scene frame-of-reference during a simulated movement of the first simulated object from the first 3D location to the second 3D location based upon the determined distance, and encode the velocity in association with the first point prior to the simulated movement of the simulated object with respect to a second point after the simulated movement of the first simulated object in a 3D velocity grid for the simulated objects. ### Claim 9 (depends on 8) The system of claim 8, wherein the instructions to collect first LiDAR data further cause the processors to: capture light rays that simulate the LiDAR sensors; and determine a first intersection between the light rays and the first simulated object at the first three-dimensional (3D) location in the simulated road scenario, wherein the first intersection defines the first point in the plurality of points representative of the first simulated object at the first 3D location. ### Claim 10 (depends on 8) The system of claim 8, wherein the instructions further cause the processors to identify the second point in the second plurality of points, the second point mapping to the same portion of the first simulated object as the first point. ### Claim 11 (depends on 8) The system of claim 8, wherein the instructions to collect second LiDAR data further cause the processors to: capture light rays that simulate the LiDAR sensors; and determine a second intersection between the light rays and the first simulated object at the second three-dimensional (3D) location in the simulated road scenario. ### Claim 12 (depends on 11) The system of claim 11, wherein the second intersection defines the second point in the second plurality of points representative of the first simulated object at the second three-dimensional (3D) location. ### Claim 13 (depends on 8) The system of claim 8, wherein a relative position of the first point on the first simulated object frame-of-reference is maintained during the simulated movement of the first simulated object. ### Claim 14 (depends on 8) The system of claim 8, wherein the instructions further cause the processors to train the autonomous vehicle using a machine learning algorithm including the three-dimensional (3D) velocity grid for the simulated objects in the simulated environment. ### Claim 15 A non-transitory computer-readable storage medium comprising instructions stored therein, which when executed by one or more processors, cause the processors to: simulate a road scenario with simulated non-player characters, collect first LiDAR data from simulated LiDAR sensors in the simulated road scenario, wherein the collected first LiDAR data comprises a first plurality of points that are representative of a first simulated object at a first three-dimensional (3D) location and a first time, a first point representative of a portion of the first simulated object, transform the first plurality of points from a simulated-scene frame-of-reference to a first simulated object frame-of-reference, simulate the first simulated object to move from the first 3D location to a second 3D location within the simulated road scenario between the first time and a second time, collect second LiDAR data from the simulated LiDAR sensors in the simulated road scenario at the second time, wherein the collected second LiDAR data comprises a second plurality of points representative of the first simulated object at the second 3D location and a second time, a second point representative of a portion of the first simulated object, transform the second plurality of points from the first simulated object frame-of-reference back to the simulated-scene frame-of-reference, determine a distance between the first point representative of a portion of the first simulated object and the second point representative of a portion of the first simulated object during the simulated movement of the first simulated object; calculate the velocity of the portion on the first simulated object in the simulated-scene frame-of-reference during a simulated movement of the first simulated object from the first 3D location to the second 3D location based upon the determined distance, and encode the velocity in association with the first point prior to the simulated movement of the simulated object with respect to a second point after the simulated movement of the first simulated object in a 3D velocity grid for the simulated objects. ### Claim 16 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the instructions to collect first LiDAR data cause the processors to: capture light rays that simulate the LiDAR sensors; and determine a first intersection between the light rays and the first simulated object at the first three-dimensional (3D) location in the simulated road scenario, wherein the first intersection defines the first point in the plurality of points representative of the first simulated object at the first 3D location. ### Claim 17 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the instructions further cause the processors to identify the second point in the second plurality of points, the second point mapping to the same portion of the first simulated object as the first point. ### Claim 18 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein the instructions to collect second LiDAR data further cause the processors to: capture light rays that simulate the LiDAR sensors; and determine a second intersection between the light rays and the first simulated object at the second three-dimensional (3D) location in the simulated road scenario. ### Claim 19 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein the second intersection defines the second point in the second plurality of points representative of the first simulated object at the second three-dimensional (3D) location. ### Claim 20 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein a relative position of the first point on the first simulated object frame-of-reference is maintained during the simulated movement of the first simulated object.