# US 11,999,385 B2 · Mobile object control system, control method, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,999,385 B2 - Document ID: US11999385B2 - Status: Granted - Kind: B2 - Application number: 17/671,643 - Filed: 2022-02-15 - Published: 2022-09-01 - Granted: 2024-06-04 - Earliest claimed: 2021-03-01 - Assignees: Honda Motor Co., Ltd. - Applicants: Honda Motor Co., Ltd. - Inventors: Yosuke Koike; Misako Yoshimura; Ichiro Baba; Kosuke Nakanishi; Koki Aizawa - Web page: https://patentgrep.com/patent/US11999385B2 ## CPC - B60W 60/0027: {using trajectory prediction for other traffic participants} - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - B60W 60/0011: {involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles} - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2552/45: Pedestrian sidewalk - B60W 2552/50: Barriers - B60W 2554/4029: Pedestrians - B60W 2554/4042: Longitudinal speed - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A mobile object control system includes: a storage device configured to store instructions; and one or more processors, wherein the one or more processors execute the instructions stored in the storage device to: acquire an image in which a road where a mobile object is present is imaged; recognize support information for supporting movement of a specific traffic participant who is a traffic participant having a predetermined attribute appearing on a road surface included in the image; recognize a road structure included in the image or a road structure which is a structure provided on the road; move the mobile object at least based on a trajectory determined using the support information when the support information is recognized; and move the mobile object based on a trajectory determined using the road structure without using the support information when the support information has not been recognized. ## Claims ### Claim 1 A mobile object control system comprising: a storage device configured to store instructions; and one or more processors, wherein the one or more processors execute the instructions stored in the storage device to: acquire an image in which a road where a mobile object is present is imaged; recognize support information for supporting movement of a specific traffic participant who is a traffic participant having a predetermined attribute appearing on a road surface included in the image; recognize a road structure included in the image or a road structure which is a structure provided on the road in the image; move the mobile object at least based on a trajectory determined using the support information when the support information is recognized; and move the mobile object based on a trajectory determined using the road structure without using the support information when the support information has not been recognized. ### Claim 2 (depends on 1) The mobile object control system according to claim 1, wherein the support information is a plurality of Braille blocks provided on a sidewalk of the road, and the one or more processors execute the instructions to: determine the trajectory at least based on a direction in which the plurality of Braille blocks are connected when the support information is recognized. ### Claim 3 (depends on 1) The mobile object control system according to claim 1, wherein the one or more processors execute the instructions to: move the mobile object based on a trajectory determined using an extension direction of a sidewalk included in the road, a structure that separates the sidewalk from a roadway, or a structure that separates the sidewalk from a region different from the sidewalk on an opposite side of the roadway without using the support information when the support information has not been recognized. ### Claim 4 (depends on 3) The mobile object control system according to claim 3, wherein the structure is a guardrail, a curb, a fence, or a plant. ### Claim 5 (depends on 1) The mobile object control system according to claim 1, wherein the traffic participant is a pedestrian, and the one or more processors execute the instructions to: determine the trajectory based on the support information and an attribute of the pedestrian when the support information is recognized and a pedestrian is present in the a position according to where the support information appears or in the vicinity of a position according to where the support information appears. ### Claim 6 (depends on 1) The mobile object control system according to claim 1, wherein the traffic participant is a pedestrian and the one or more processors execute the instructions to: generate a trajectory for moving through positions farther from the pedestrian when the pedestrian is the specific traffic participant than when the pedestrian is a pedestrian different from the specific traffic participant. ### Claim 7 (depends on 1) The mobile object control system according to claim 1, wherein the traffic participant is a pedestrian and the support information is a plurality of Braille blocks provided on a sidewalk of the road, the Braille blocks include a guidance block that indicates a traveling direction in which the specific traffic participant travels and guides movement of the specific traffic participant, and a warning block that warns the specific traffic participant, and the one or more processors execute the instructions to: when it is recognized that the guidance block and a warning block connected to the guidance block are provided in this order on a sidewalk from a position of the mobile object, move the mobile object based on a first trajectory in which the mobile object moves along a guidance direction indicated by the guidance block and the mobile object decelerates or stops at or in the vicinity of the warning block. ### Claim 8 (depends on 7) The mobile object control system according to claim 7, wherein the traffic participant is a pedestrian, and the one or more processors execute the instructions to: cancel deceleration of the mobile object or starting the mobile object from a stopped state when condition (b), (c), or (d) is satisfied after condition (a) is satisfied: (a) the specific traffic participant is present at or in the vicinity of a warning block when the mobile object decelerates or stops and the mobile object moving based on the first trajectory reaches the warning block or the vicinity of the warning block; (b) the specific traffic participant is going straight ahead after passing a warning block; (c) the specific traffic participant is moving in a first direction and has crossed the mobile object; and (d) the specific traffic participant moves in a second direction opposite to the first direction. ### Claim 9 A control method for causing a computer to execute: acquiring an image in which a road where a mobile object is present is imaged; recognizing support information for supporting walking of a specific traffic participant who is a traffic participant having a predetermined attribute appearing on a road surface included in the image; recognizing a road structure included in the image or a road structure which is a structure provided on the road; moving the mobile object at least based on a trajectory determined using the support information when the support information is recognized; and moving the mobile object based on a trajectory determined using the road structure without using the support information when the support information has not been recognized. ### Claim 10 A non-transitory computer storage medium storing a program causing a computer to execute: acquiring an image in which a road where a mobile object is present is imaged; recognizing support information for supporting walking of a specific traffic participant who is a traffic participant having a predetermined attribute appearing on a road surface included in the image; recognizing a road structure included in the image or a road structure which is a structure provided on the road; moving the mobile object at least based on a trajectory determined using the support information when the support information is recognized; and moving the mobile object based on a trajectory determined using the road structure without using the support information when the support information has not been recognized.