Back to results
US 2026/0285330 A1Published application
System
AssigneeSoftBank Group Corp.
InventorDaisuke YOSHIDA
Life of this patent
Estimating the term…
Abstract
A system includes a processor that is configured to collect voice data of a driver and use a prompt to instruct a generative artificial intelligence model to analyze a reaction speed, slurring, and a voice pattern based on the voice data, capture eye movements of the driver and analyze a gaze movement and a blinking frequency using an image processing technique based on the eye movements, and integrate analyzed voice data and eye movement data and determine presence or absence of alcohol consumption of the driver and a degree of influence of the alcohol consumption using a machine learning technique.
Claims20
A system comprising:
- circuitry configured to:
- acquire acoustic information of an operator from an audio input device and execute speech recognition processing to convert the acoustic information into character information and time information, and calculate a reaction time and a speech clarity value based on the character information and the time information;
- acquire, via a communication interface coupled to a packet-switched network, image information including an eye region of the operator from an image acquisition device, perform image processing on the image information to detect the eye region, and calculate a blink frequency and a gaze direction variation amount based on position changes of an eyelid and a pupil;
- synchronize the reaction time, the speech clarity value, the blink frequency, and the gaze direction variation amount on a common time axis, and generate, for each of predetermined time intervals, a feature vector comprising the reaction time, the speech clarity value, the blink frequency, and the gaze direction variation amount;
- input the feature vector into a machine learning model and output a determination result regarding a presence or absence of an impairment condition of the operator and a degree of influence of the impairment condition;
- format evaluation information comprising the determination result and the feature vector into a text representation, attach a prompt sentence that instructs a generative neural network model to generate an explanation of the evaluation information, transmit the text representation and the prompt sentence to the generative neural network model, and obtain an explanatory text regarding a state of the operator;
- and
- generate control information indicating whether activation of a power unit is permitted or prohibited in accordance with the determination result, and transmit the control information as a notification data packet to a safety mechanism so as to control the safety mechanism such that the power unit is not activated solely by an operation of the operator.
5 more elements
A system comprising:
- circuitry configured to:
- acquire, via a communication interface coupled to a packet-switched network, acoustic information from an audio input device and image information from an image acquisition device of a moving body, execute speech recognition processing to calculate a reaction time and a speech clarity value from the acoustic information, and perform image processing to calculate a blink frequency and a gaze direction variation amount from the image information;
- synchronize the reaction time, the speech clarity value, the blink frequency, and the gaze direction variation amount on a common time axis to generate a feature vector, and input the feature vector into a machine learning model to output a determination result comprising a classification label and a confidence score regarding an impairment condition;
- extract emotion-related feature values from the acoustic information and the image information, input a prompt sentence comprising the emotion-related feature values to a generative neural network model comprising a transformer-based architecture, obtain an emotional state evaluation, and correct the determination result based on the emotional state evaluation;
- format the determination result and the feature vector into a text representation, input the text representation with a prompt sentence to the generative neural network model, and obtain an explanatory text comprising detected indicators, an assessed severity, and a recommended action;
- and
- generate control information indicating whether activation of a power unit is permitted or prohibited, transmit the control information as a notification data packet to a safety mechanism, and transmit the explanatory text to a terminal device of the operator.
4 more elements
A method comprising:
- acquiring acoustic information of an operator from an audio input device and executing speech recognition processing to calculate a reaction time and a speech clarity value;
- acquiring, via a communication interface coupled to a packet-switched network, image information including an eye region of the operator from an image acquisition device, performing image processing to calculate a blink frequency and a gaze direction variation amount;
- synchronizing the reaction time, the speech clarity value, the blink frequency, and the gaze direction variation amount on a common time axis, and generating a feature vector for each of predetermined time intervals;
- inputting the feature vector into a machine learning model and outputting a determination result regarding a presence or absence of an impairment condition and a degree of influence;
- formatting evaluation information comprising the determination result and the feature vector, attaching a prompt sentence instructing a generative neural network model to generate an explanatory text, and obtaining the explanatory text;
- and
- generating control information indicating whether activation of a power unit is permitted or prohibited in accordance with the determination result, and transmitting the control information as a notification data packet to a safety mechanism.
4 more elements
Dependencies are inferred from the claim text.