US 11,886,644 B2Granted

Human machine interface and associated method of interaction

AssigneeThe Boeing Company

InventorAmir Afrasiabi

Life of this patent

Estimating the term…

Loading patent…

Abstract

A human machine interface (HMI), a method of interacting with an HMI and a corresponding computer program product facilitate user interaction with an HMI. In a method, a face of a person attempting to interact with the HMI is detected and a gesture made by the person relative to the HMI is also detected. The method analyzes information regarding the face of the person that has been detected to determine whether the person is wearing personal protection equipment. In an instance in which the person is determined to not be wearing personal protection equipment, the method suspends a response to the gesture made by the person. However, in an instance in which the person is determined to be wearing personal protection equipment, the method analyzes the gesture made by the person relative to the HMI and converts the gesture to a corresponding command to the system associated with the HMI.

Claims20

Dependent claims

A method for interacting with a human machine interface (HMI) of a system, the method comprising:

detecting a face of a first person attempting to interact with the HMI and detecting a gesture made by the first person relative to the HMI;
analyzing information regarding the face of the first person that has been detected to determine whether the first person is wearing personal protection equipment;
in an instance in which the first person is determined to not be wearing personal protection equipment, suspending a response to the gesture made by the first person relative to the HMI;
in an instance in which the first person is determined to be wearing personal protection equipment, analyzing the gesture made by the first person relative to the HMI and converting the gesture to a corresponding command to the system with which the HMI is associated;
in an instance in which the response to the gesture made was suspended:
subsequently detecting a subsequent face of a subsequent person attempting to interact with the HMI within a predetermined time period;
analyzing information regarding the subsequent face of the subsequent person that has been subsequently detected to determine if it is the first person;
analyzing information regarding the subsequent face of the subsequent person that has been subsequently detected to determine whether the subsequent person is now wearing the personal protection equipment;
and
in an instance in which the subsequent person is determined to be the first person and determined to now be wearing personal protection equipment based upon subsequent detection of the subsequent face of the subsequent person, terminating suspension of the response to the gesture and analyzing the gesture made by the first person relative to the HMI and converting the gesture to the corresponding command to the system with which the HMI is associated;
and
in an instance in which the subsequent person is determined to not be the first person, maintaining suspension of the response to the gesture.

9 more elements

A human machine interface (HMI) of a system, the human machine interface comprising processing circuitry configured to:

detect a face of a first person attempting to interact with the HMI and detect a gesture made by the first person relative to the HMI;
analyze information regarding the face of the first person that has been detected to determine whether the first person is wearing personal protection equipment;
in an instance in which the first person is determined to not be wearing personal protection equipment, suspend a response to the gesture made by the first person relative to the HMI;
and
in an instance in which the first person is determined to be wearing personal protection equipment, analyze the gesture made by the first person relative to the HMI and convert the gesture to a corresponding command to the system with which the HMI is associated;
subsequently detect a subsequent face of a subsequent person attempting to interact with the HMI within a predetermined time period;
analyze information regarding the subsequent face of the subsequent person that has been subsequently detected to determine if it is the first person;
analyze information regarding the subsequent face of the subsequent person that has been subsequently detected to determine whether the subsequent person is now wearing the personal protection equipment;
and
in an instance in which the subsequent person is determined to be the first person and determined to now be wearing personal protection equipment based upon subsequent detection of the subsequent face of the subsequent person, terminate suspension of the response to the gesture and analyze the gesture made by the first person relative to the HMI and convert the gesture to the corresponding command to the system with which the HMI is associated;
and
in an instance in which the subsequent person is determined to not be the first person, maintain suspension of the response to the gesture.

9 more elements

A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions to:

detect a face of a first person attempting to interact with the HMI and detect a gesture made by the first person relative to the HMI;
analyze information regarding the face of the first person that has been detected to determine whether the first person is wearing personal protection equipment;
in an instance in which the first person is determined to not be wearing personal protection equipment, suspend a response to the gesture made by the first person relative to the HMI;
and
in an instance in which the first person is determined to be wearing personal protection equipment, analyze the gesture made by the first person relative to the HMI and convert the gesture to a corresponding command to the system with which the HMI is associated;
subsequently detect a subsequent face of a subsequent person attempting to interact with the HMI within a predetermined time period;
analyze information regarding the subsequent face of the subsequent person that has been subsequently detected to determine if it is the first person;
analyze information regarding the subsequent face of the subsequent person that has been subsequently detected to determine whether the subsequent person is now wearing the personal protection equipment;
and
in an instance in which the subsequent person is determined to be the first person and determined to now be wearing personal protection equipment based upon subsequent detection of the subsequent face of the subsequent person, terminate suspension of the response to the gesture and analyze the gesture made by the first person relative to the HMI and convert the gesture to the corresponding command to the system with which the HMI is associated;
and
in an instance in which the subsequent person is determined to not be the first person, maintain suspension of the response to the gesture.

9 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/550,213Filed 14 Dec 2021 · us provisional application
Priority: 14 Dec 2020
Published: US 2022/0187923 A116 Jun 2022
Granted: 30 Jan 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification10 codes

G06F 3/017
Input arrangements or combined input and output arrangements for interaction between user and computer: gesture based interaction, e.g. based on a…
B64D 11/00151
Arrangements for entertainment or communications, e.g. radio, television: permanently mounted seat back monitors
B64D 11/00155
Arrangements for entertainment or communications, e.g. radio, television: individual entertainment or communication system remote controls…
G06F 3/012
Arrangements for interaction with the human body, e.g. for user immersion in virtual reality: head tracking input arrangements