# US 12,589,737 B2 · Vehicle systems and methods for pedestrian road crossing predictions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,589,737 B2 - Document ID: US12589737B2 - Status: Granted - Kind: B2 - Application number: 18/402,834 - Filed: 2024-01-03 - Published: 2025-07-03 - Granted: 2026-03-31 - Earliest claimed: 2024-01-03 - Assignees: GM Global Technology Operations LLC - Applicants: GM Global Technology Operations LLC - Inventors: Oded Bialer; Roy Uziel - Web page: https://patentgrep.com/patent/US12589737B2 ## CPC - B60W 30/09: Taking automatic action to avoid collision, e.g. braking and steering - B60W 10/18: including control of braking systems - B60W 30/0956: {the prediction being responsive to traffic or environmental parameters} - B60W 50/0097: {Predicting future conditions} - B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention - G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215) - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion - G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - B60W 2050/146: {Display means} - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2554/4029: Pedestrians - B60W 2554/4045: Intention, e.g. lane change or imminent movement - G06T 2207/20081: Training; Learning - G06T 2207/30196: Human being; Person - G06T 2207/30241: Trajectory - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A vehicle system includes at least one sensor configured to capture one or more images of a pedestrian positioned near a road, and a control module in communication with the at least one sensor. The control module is configured to determine one or more characteristics associated with the pedestrian positioned near to the road based on the one or more captured images, generate a trajectory prediction for the pedestrian, overlay the trajectory prediction for the pedestrian on a road segmentation of the road, and generate a road crossing prediction for the pedestrian based on the one or more characteristics and the overlayed trajectory prediction. The road crossing prediction forecasts whether the pedestrian will be on or off the road. Other example vehicle systems and methods for forecasting a future presence of a pedestrian on a road are also disclosed. ## Claims ### Claim 1 A vehicle system for a vehicle to forecast a future presence of a pedestrian on a road, the vehicle system comprising: at least one sensor configured to capture one or more images of the pedestrian positioned near the road; and a controller in communication with the at least one sensor, the controller configured to: determine one or more characteristics associated with the pedestrian positioned near the road based on the one or more captured images, the one or more characteristics including a crossing intention of the pedestrian indicative as to whether the pedestrian intends to cross the road and a motion state estimation of the pedestrian indicative of whether the pedestrian is moving or not moving; generate a trajectory prediction for the pedestrian; overlay the trajectory prediction for the pedestrian on a road segmentation of the road; set confidence values for the crossing intention of the pedestrian, the motion state estimation of the pedestrian, and the trajectory prediction overlapping at least a portion of the road; and generate a road crossing prediction forecasting that the pedestrian will be on the road only if a sum of the confidence values exceeds a defined threshold. ### Claim 2 (depends on 1) The vehicle system of claim 1, wherein the controller is configured to determine the crossing intention of the pedestrian with a machine learning module based on one or more behavior features associated with the pedestrian in the one or more captured images. ### Claim 3 (depends on 1) The vehicle system of claim 1, wherein the controller is configured to determine the motion state estimation of the pedestrian with a machine learning module based on the one or more captured images. ### Claim 4 (depends on 1) The vehicle system of claim 1, wherein: the controller is configured to receive position data of the pedestrian to detect and locate the pedestrian; and the controller includes a prediction model configured to generate the trajectory prediction based on the position data. ### Claim 5 (depends on 4) The vehicle system of claim 4, wherein the prediction model is configured to generate the trajectory prediction based on the motion state estimation of the pedestrian. ### Claim 6 (depends on 1) The vehicle system of claim 1, wherein the controller is configured to generate the road crossing prediction forecasting whether the pedestrian will be on or off the road for a period of time. ### Claim 7 (depends on 1) The vehicle system of claim 1, further comprising a sensor configured to detect a velocity of the vehicle, wherein the controller is configured to generate the road crossing prediction for the pedestrian based on the velocity of the vehicle. ### Claim 8 (depends on 1) The vehicle system of claim 1, wherein the confidence values are weighted values. ### Claim 9 (depends on 1) The vehicle system of claim 1, further comprising a vehicle controller in communication with the controller, the vehicle controller configured to receive one or more signals from the controller indicative of the generated road crossing prediction. ### Claim 10 (depends on 9) The vehicle system of claim 9, wherein the vehicle controller is configured to control at least one vehicle control system based on the one or more signals. ### Claim 11 (depends on 10) The vehicle system of claim 10, wherein the vehicle control system includes an autonomous braking system. ### Claim 12 A method for forecasting a future presence of a pedestrian on a road on which a vehicle is located, the method comprising: determining one or more characteristics associated with the pedestrian positioned near to the road based on one or more images captured by at least one sensor, the one or more characteristics including a crossing intention of the pedestrian indicative as to whether the pedestrian intends to cross the road and a motion state estimation of the pedestrian indicative of whether the pedestrian is moving or not moving; generating a trajectory prediction for the pedestrian; overlaying the trajectory prediction for the pedestrian on a road segmentation of the road; setting confidence values for the crossing intention of the pedestrian, the motion state estimation of the pedestrian, and the trajectory prediction overlapping at least a portion of the road; and generating a road crossing prediction forecasting that the pedestrian will be on the road only if a sum of the confidence values exceeds a defined threshold. ### Claim 13 (depends on 12) The method of claim 12, further comprising: generating one or more signals indicative of the generated road crossing prediction; and controlling at least one vehicle control system based on the one or more signals. ### Claim 14 (depends on 13) The method of claim 13, wherein the vehicle control system includes an autonomous braking system. ### Claim 15 A vehicle system for a vehicle to forecast a future presence of a pedestrian on a road, the vehicle system comprising: at least one sensor configured to capture one or more images of the pedestrian positioned near the road; and a controller in communication with the at least one sensor, the controller configured to: determine one or more characteristics associated with the pedestrian positioned near the road based on the one or more captured images, the one or more characteristics including a crossing intention of the pedestrian indicative as to whether the pedestrian intends to cross the road and a motion state estimation of the pedestrian indicative of whether the pedestrian is moving or not moving; generate a trajectory prediction for the pedestrian; overlay the trajectory prediction for the pedestrian on a road segmentation of the road; and generate a road crossing prediction forecasting that the pedestrian will be on the road only if the pedestrian intends to cross, the motion state estimation is moving, a velocity of the vehicle is below a defined threshold, and the overlayed trajectory prediction overlaps at least a portion of the road. ### Claim 16 (depends on 15) The vehicle system of claim 15, further comprising a vehicle controller in communication with the controller, the vehicle controller configured to receive one or more signals from the controller indicative of the generated road crossing prediction. ### Claim 17 (depends on 16) The vehicle system of claim 16, wherein the vehicle controller is configured to control at least one vehicle control system based on the one or more signals. ### Claim 18 (depends on 17) The vehicle system of claim 17, wherein the vehicle control system includes an autonomous braking system. ### Claim 19 (depends on 15) The vehicle system of claim 15, wherein the controller is configured to determine the crossing intention of the pedestrian with a machine learning module based on one or more behavior features associated with the pedestrian in the one or more captured images. ### Claim 20 (depends on 15) The vehicle system of claim 15, wherein the controller is configured to determine the motion state estimation of the pedestrian with a machine learning module based on the one or more captured images.