# US 2023/0267751 A1 · Vehicle safety assistance system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0267751 A1 - Document ID: US20230267751A1 - Status: Published application - Kind: A1 - Application number: 18/017,701 - Filed: 2021-07-27 - Published: 2023-08-24 - Earliest claimed: 2020-07-27 - Assignees: Techno-Accel Networks Corp.; Murakami Corporation - Applicants: Techno-Accel Networks Corp.; Murakami Corporation - Inventors: Kazutami ARIMOTO; Naoki YAMAUCHI; Kagehisa KAJIWARA; Atsushi HAYAMI - Web page: https://patentgrep.com/patent/US20230267751A1 ## CPC - G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions - A61B 5/6892: {Mats} - A61B 5/6893: {Cars} - B60Q 9/00: Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 {, e.g. haptic signalling} - G01J 1/42: using electric radiation detectors (optical or mechanical part G01J1/04; by comparison with a reference light or electric value G01J1/10) - G01S 13/56: for presence detection {(presence detection using near field arrangements G01V3/00, e.g. G01V3/08, G01V3/12; burglar, theft or intruder alarms with electrical actuation G08B13/22 - G08B13/26)} - G01S 13/867: {Combination of radar systems with cameras} - G01S 13/931: of land vehicles - G06T 7/0012: {Biomedical image inspection} - G06V 10/75: Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries - G06V 10/776: Validation; Performance evaluation - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/593: {Recognising seat occupancy} - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - G08B 21/22: responsive to presence or absence of persons - G08B 21/24: Reminder alarms, e.g. anti-loss alarms - A61B 2503/06: Children, e.g. for attention deficit diagnosis - A61B 2503/40: Animals - B60K 28/02: responsive to conditions relating to the driver - B60R 25/305: {using a camera} - B60R 25/31: of human presence inside or outside the vehicle - G01S 13/862: {Combination of radar systems with sonar systems} - G06T 2207/30196: Human being; Person - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL - G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N) - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract An object of the present invention is to provide a vehicle safety assistance system enabling perception of a state of passengers. A vehicle safety assistance system according to the present invention assists safety of at least one passenger on board a vehicle, and includes: a monitoring unit capable of monitoring the passenger in the vehicle; a recognition unit recognizing a state of the passenger by monitoring information from the monitoring unit; and a control unit carrying out control of operation of the vehicle or notification to the passenger on basis of recognition information from the recognition unit, in which the monitoring unit includes a monitoring camera. ## Claims ### Claim 1 A vehicle safety assistance system assisting safety of at least one passenger on board a vehicle, comprising: a monitoring unit capable of monitoring the at least one passenger in the vehicle; a recognition unit recognizing a state of the at least one passenger by monitoring information from the monitoring unit; and a control unit carrying out control of operation of the vehicle or notification to the at least one passenger on basis of recognition information from the recognition unit, wherein the monitoring unit comprises a monitoring camera. ### Claim 2 (depends on 1) The vehicle safety assistance system according to claim 1, wherein: the monitoring unit comprises a belongings identification means identifying belongings likely to be brought into the vehicle; the monitoring information comprises belongings information identified by the belongings identification means; the recognition information comprises belongings detection information identifying whether the belongings are present in the vehicle and getting-off information identifying whether all of the at least one passenger has gotten off the vehicle; and when it is identified that all of the at least one passenger has gotten off the vehicle on basis of the getting-off information and that the belongings are present in the vehicle on basis of the belongings detection information, the control unit notifies the at least one passenger that the belongings are present in the vehicle. ### Claim 3 (depends on 2) The vehicle safety assistance system according to claim 2, wherein the monitoring unit comprises a monitoring radar. ### Claim 4 (depends on 3) The vehicle safety assistance system according to claim 3, wherein the belongings identification means uses periodic oscillation detected by the monitoring radar. ### Claim 5 (depends on 1) The vehicle safety assistance system according to claim 1, wherein the recognition unit comprises: a quantification means expressing the state of the at least one passenger by a numerical value from the monitoring information through an evaluation function; an optimization means optimizing a threshold value for identifying the state of the at least one passenger according to a magnitude relationship with the numerical value; and a recognition information identification means identifying the state of the at least one passenger as the recognition information from the numerical value and the threshold value. ### Claim 6 (depends on 5) The vehicle safety assistance system according to claim 5, wherein: the monitoring unit comprises: a database on which a reference image of a passenger captured beforehand is recorded; and a passenger identification means identifying the passenger by comparatively searching for a captured image of the passenger captured by the monitoring camera with the reference image; the passenger identification means is configured to be capable of calculating a degree of matching between the captured image and the reference image; and the optimization means of the recognition unit uses the degree of matching. ### Claim 7 (depends on 5) The vehicle safety assistance system according to claim 5, wherein: the monitoring unit comprises a photometer measuring luminosity of a periphery of the vehicle; and the optimization means of the recognition unit uses the luminosity. ### Claim 8 (depends on 1) The vehicle safety assistance system according to claim 1, wherein: the monitoring unit comprises a vital sensor acquiring biological information of the at least one passenger; and the monitoring information comprises the biological information obtained from the vital sensor. ### Claim 9 (depends on 1) The vehicle safety assistance system according to claim 1, wherein: the monitoring unit comprises an ultrasonic sensor capable of detecting a person approaching the vehicle from outside of the vehicle, and an outboard camera or an outboard radar for sensing the person; the monitoring information comprises outboard information obtained from the ultrasonic sensor, and the outboard camera or the outboard radar; the recognition information comprises suspicious person identification information identifying, from the outboard information, whether the person is a suspicious person; and the control unit uses the suspicious person identification information. ### Claim 10 (depends on 2) The vehicle safety assistance system according to claim 2, wherein the recognition unit comprises: a quantification means expressing the state of the at least one passenger by a numerical value from the monitoring information through an evaluation function; an optimization means optimizing a threshold value for identifying the state of the at least one passenger according to a magnitude relationship with the numerical value; and a recognition information identification means identifying the state of the at least one passenger as the recognition information from the numerical value and the threshold value. ### Claim 11 (depends on 3) The vehicle safety assistance system according to claim 3, wherein the recognition unit comprises: a quantification means expressing the state of the at least one passenger by a numerical value from the monitoring information through an evaluation function; an optimization means optimizing a threshold value for identifying the state of the at least one passenger according to a magnitude relationship with the numerical value; and a recognition information identification means identifying the state of the at least one passenger as the recognition information from the numerical value and the threshold value. ### Claim 12 (depends on 4) The vehicle safety assistance system according to claim 4, wherein the recognition unit comprises: a quantification means expressing the state of the at least one passenger by a numerical value from the monitoring information through an evaluation function; an optimization means optimizing a threshold value for identifying the state of the at least one passenger according to a magnitude relationship with the numerical value; and a recognition information identification means identifying the state of the at least one passenger as the recognition information from the numerical value and the threshold value. ### Claim 13 (depends on 10) The vehicle safety assistance system according to claim 10, wherein: the monitoring unit comprises: a database on which a reference image of a passenger captured beforehand is recorded; and a passenger identification means identifying the passenger by comparatively searching for a captured image of the passenger captured by the monitoring camera with the reference image; the passenger identification means is configured to be capable of calculating a degree of matching between the captured image and the reference image; and the optimization means of the recognition unit uses the degree of matching. ### Claim 14 (depends on 11) The vehicle safety assistance system according to claim 11, wherein: the monitoring unit comprises: a database on which a reference image of a passenger captured beforehand is recorded; and a passenger identification means identifying the passenger by comparatively searching for a captured image of the passenger captured by the monitoring camera with the reference image; the passenger identification means is configured to be capable of calculating a degree of matching between the captured image and the reference image; and the optimization means of the recognition unit uses the degree of matching. ### Claim 15 (depends on 12) The vehicle safety assistance system according to claim 12, wherein: the monitoring unit comprises: a database on which a reference image of a passenger captured beforehand is recorded; and a passenger identification means identifying the passenger by comparatively searching for a captured image of the passenger captured by the monitoring camera with the reference image; the passenger identification means is configured to be capable of calculating a degree of matching between the captured image and the reference image; and the optimization means of the recognition unit uses the degree of matching. ### Claim 16 (depends on 10) The vehicle safety assistance system according to claim 10, wherein: the monitoring unit comprises a photometer measuring luminosity of a periphery of the vehicle; and the optimization means of the recognition unit uses the luminosity. ### Claim 17 (depends on 11) The vehicle safety assistance system according to claim 11, wherein: the monitoring unit comprises a photometer measuring luminosity of a periphery of the vehicle; and the optimization means of the recognition unit uses the luminosity. ### Claim 18 (depends on 12) The vehicle safety assistance system according to claim 12, wherein: the monitoring unit comprises a photometer measuring luminosity of a periphery of the vehicle; and the optimization means of the recognition unit uses the luminosity.