# US 12,217,514 B2 · Parked car classification based on a velocity estimation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,514 B2 - Document ID: US12217514B2 - Status: Granted - Kind: B2 - Application number: 17/732,393 - Filed: 2022-04-28 - Published: 2023-11-02 - Granted: 2025-02-04 - Earliest claimed: 2022-04-28 - Assignees: Toyota Research Institute, Inc.; Toyota Jidosha Kabushiki Kaisha - Applicants: Toyota Research Institute, Inc. - Inventors: Kuan-Hui Lee; Charles Christopher Ochoa; Arjun Bhargava; Chao Fang - Web page: https://patentgrep.com/patent/US12217514B2 ## CPC - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - B60W 40/04: Traffic conditions - B60W 60/001: {Planning or execution of driving tasks} - G06T 7/248: {involving reference images or patches} - G06V 10/764: using classification, e.g. of video objects - B60W 2420/408: Radar; Laser, e.g. lidar - B60W 2520/10: Longitudinal speed - B60W 2554/20: Static objects - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/20081: Training; Learning - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06V 2201/08: Detecting or categorising vehicles - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method controlling an ego vehicle in an environment includes determining, via a flow model of a parked vehicle recognition system, a flow between a first representation of the environment and a second representation of the environment. The method also includes determining, via a velocity model of the parked vehicle recognition system, a velocity of a vehicle in the environment based on the flow. The method further includes determining, via a parked vehicle classification model of the parked vehicle recognition system, the vehicle is parked based on the velocity of the vehicle and one or more of features associated with the vehicle and/or the environment. The method still further includes planning a trajectory of the ego vehicle based on determining the vehicle is parked. ## Claims ### Claim 1 A method controlling an ego vehicle in an environment, comprising: determining, via a flow model of a parked vehicle recognition system, a flow between a first representation of the environment and a second representation of the environment, the first representation and the second representation being captured by one or more sensors associated with the ego vehicle; determining, via a velocity model of the parked vehicle recognition system, a velocity of a vehicle in the environment based on the flow; determining, via a parked vehicle classification model of the parked vehicle recognition system, the vehicle is parked based on the velocity of the vehicle and one or more of features associated with the vehicle and/or the environment, the one or more features being extracted from one or both of the first representation or the second representation of the environment via a feature extractor associated with the parked vehicle classification model; planning a trajectory of the ego vehicle based on determining the vehicle is parked; and controlling the ego vehicle to follow the trajectory based on the environment. ### Claim 2 (depends on 1) The method of claim 1, wherein the one or more features include vehicle estimated features. ### Claim 3 (depends on 2) The method of claim 2, wherein the vehicle estimated features include one or more of an object type of the vehicle, a first distance from a center point of the vehicle to a road boundary, an estimated absolute speed of the vehicle, a second distance to a nearest intersection from the vehicle, a map location type, a free lane ratio, or an edge distance between an edge of the vehicle and the road boundary. ### Claim 4 (depends on 2) The method of claim 2, wherein the velocity is concatenated with the vehicle estimated features. ### Claim 5 (depends on 1) The method of claim 1, further comprising: obtaining the first representation via a first LiDAR sweep performed at a first time period via the one or more sensors; and obtaining the second representation via a second LiDAR sweep performed at a second time period via the one or more sensors. ### Claim 6 (depends on 1) The method of claim 1, wherein the flow model is associated with a flow loss, the velocity model is associated with a velocity loss, and the parked vehicle classification model is associated with a parked vehicle loss. ### Claim 7 (depends on 6) The method of claim 6, further comprising training the parked vehicle recognition system end-to-end to minimize a sum of the flow loss, the velocity loss, and the parked vehicle loss. ### Claim 8 An apparatus for controlling an ego vehicle in an environment the apparatus comprising: at least one processor; and at least one memory coupled with the at least one processor and storing instructions operable, when executed by the at least one processor, to cause the apparatus: to determine, via a flow model of a parked vehicle recognition system, a flow between a first representation of the environment and a second representation of the environment, the first representation and the second representation being captured by one or more sensors associated with the ego vehicle; to determine, via a velocity model of the parked vehicle recognition system, a velocity of a vehicle in the environment based on the flow; to determine, via a parked vehicle classification model of the parked vehicle recognition system, the vehicle is parked based on the velocity of the vehicle and one or more of features associated with the vehicle and/or the environment, the one or more features being extracted from one or both of the first representation or the second representation of the environment via a feature extractor associated with the parked vehicle classification model; to plan a trajectory of the ego vehicle based on determining the vehicle is parked; and to control the ego vehicle to follow the trajectory based on the environment. ### Claim 9 (depends on 8) The apparatus of claim 8, wherein the one or more features include vehicle estimated features. ### Claim 10 (depends on 9) The apparatus of claim 9, wherein the vehicle estimated features include one or more of an object type of the vehicle, a first distance from a center point of the vehicle to a road boundary, an estimated absolute speed of the vehicle, a second distance to a nearest intersection from the vehicle, a map location type, a free lane ratio, or an edge distance between an edge of the vehicle and the road boundary. ### Claim 11 (depends on 9) The apparatus of claim 9, wherein the velocity is concatenated with the vehicle estimated features. ### Claim 12 (depends on 8) The apparatus of claim 8, wherein execution of the instructions further cause the apparatus to: obtain the first representation via a first LiDAR sweep performed at a first time period via the one or more sensors; and obtain the second representation via a second LiDAR sweep performed at a second time period via the one or more sensors. ### Claim 13 (depends on 8) The apparatus of claim 8, wherein the flow model is associated with a flow loss, the velocity model is associated with a velocity loss, and the parked vehicle classification model is associated with a parked vehicle loss. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein execution of the instructions further cause the apparatus to train the parked vehicle recognition system end-to-end to minimize a sum of the flow loss, the velocity loss, and the parked vehicle loss. ### Claim 15 A non-transitory computer-readable medium having program code recorded thereon for controlling an ego vehicle in an environment the program code executed by at least one processor and comprising: program code to determine, via a flow model of a parked vehicle recognition system, a flow between a first representation of the environment and a second representation of the environment, the first representation and the second representation being captured by one or more sensors associated with the ego vehicle; program code to determine, via a velocity model of the parked vehicle recognition system, a velocity of a vehicle in the environment based on the flow; program code to determine, via a parked vehicle classification model of the parked vehicle recognition system, the vehicle is parked based on the velocity of the vehicle and one or more of features associated with the vehicle and/or the environment, the one or more features being extracted from one or both of the first representation or the second representation of the environment via a feature extractor associated with the parked vehicle classification model; program code to plan a trajectory of the ego vehicle based on determining the vehicle is parked; and program code to control the ego vehicle to follow the trajectory based on the environment. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the one or more features include vehicle estimated features. ### Claim 17 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the vehicle estimated features include one or more of an object type of the vehicle, a first distance from a center point of the vehicle to a road boundary, an estimated absolute speed of the vehicle, a second distance to a nearest intersection from the vehicle, a map location type, a free lane ratio, or an edge distance between an edge of the vehicle and the road boundary. ### Claim 18 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the velocity is concatenated with the vehicle estimated features. ### Claim 19 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the program code further includes: program code to obtain the first representation via a first LiDAR sweep performed at a first time period via the one or more sensors; and program code to obtain the second representation via a second LiDAR sweep performed at a second time period via the one or more sensors. ### Claim 20 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein: the flow model is associated with a flow loss, the velocity model is associated with a velocity loss, and the parked vehicle classification model is associated with a parked vehicle loss; and the program code further includes program code to train the parked vehicle recognition system end-to-end to minimize a sum of the flow loss, the velocity loss, and the parked vehicle loss.