# US 11,077,844 B2 · System and method for increasing passenger satisfaction in a vehicle having an automated driving system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,077,844 B2
- Document ID: US11077844B2
- Status: Granted
- Kind: B2
- Application number: 16/389,330
- Filed: 2019-04-19
- Published: 2020-10-22
- Granted: 2021-08-03
- Earliest claimed: 2019-04-19
- Assignees: GM Global Technology Operations LLC
- Applicants: GM Global Technology Operations LLC
- Inventors: Joseph F. Szczerba
- Web page: https://patentgrep.com/patent/US11077844B2

## CPC

- B60W 30/025: {related to comfort of drivers or passengers}
- A61B 5/02055: {Simultaneously evaluating both cardiovascular condition and temperature}
- A61B 5/18: for vehicle drivers {or machine operators}
- B60R 1/27: providing all-round vision, e.g. using omnidirectional cameras
- B60W 40/08: related to drivers or passengers
- B60W 50/0098: {Details of control systems ensuring comfort, safety or stability not otherwise provided for}
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- B60W 60/0013: {specially adapted for occupant comfort}
- G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)}
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens}
- A61B 5/01: Measuring temperature of body parts {; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue} (clinical contact thermometers G01K13/20)
- A61B 5/02438: {with portable devices, e.g. worn by the patient}
- A61B 5/0533: Measuring galvanic skin response
- A61B 5/1123: {Discriminating type of movement, e.g. walking or running (A61B5/1116, A61B5/112 take precedence)}
- A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)}
- A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles}
- A61B 5/681: {Wristwatch-type devices}
- A61B 5/6893: {Cars}
- B60W 2050/0088: {Adaptive recalibration}
- B60W 2050/146: {Display means}
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2540/22: Psychological state; Stress level or workload
- B60W 2720/106: Longitudinal acceleration
- B60W 2754/30: Longitudinal distance
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system for increasing satisfaction of a passenger in a vehicle having an automated driving system structured to operate the autonomous vehicle may include a first sensor structured to detect a driving environment of the vehicle, the first sensor being operably coupled to the automated driving system; a processor operably coupled to the first sensor and the automated driving system, the processor being structured to calculate a vehicle path plan based on the driving environment of the vehicle and local infrastructure information; and a display operably coupled to the first sensor, the processor, and the automated driving system. The processor may be structured to control the display to display a graphical representation of the driving environment of the vehicle and the vehicle path plan.

## Claims

### Claim 1

A system for increasing satisfaction of a passenger in a vehicle having an automated driving system structured to operate the vehicle, the system comprising:
a first sensor structured to detect a driving environment of the vehicle, the first sensor being operably coupled to the automated driving system;
a processor operably coupled to the first sensor and the automated driving system, the processor being structured to calculate a vehicle path plan based on the driving environment of the vehicle and local infrastructure information;
a display operably coupled to the first sensor, the processor, and the automated driving system, wherein the processor is structured to control the display to display a graphical representation of the driving environment of the vehicle and the vehicle path plan;
a second sensor structured to detect a first property of the passenger, the second sensor being operably connected to the processor, wherein the processor is structured to calculate a passenger satisfaction index based on the first property of the passenger, to control the automated driving system to modify driving behavior of the vehicle in response to the passenger satisfaction index satisfying a first condition, and to control the display to notify the passenger of a change in driving behavior of the vehicle.

### Claim 2 (depends on 1)

The system of claim 1, wherein the graphical representation comprises a current speed of the vehicle, an acceleration status of the vehicle, an indication of nearby objects, and the local infrastructure information.

### Claim 3 (depends on 1)

The system of claim 1, wherein the graphical representation comprises an indication of an upcoming maneuver based on the vehicle path plan.

### Claim 4 (depends on 1)

The system of claim 1, further comprising a communication node structured to communicate with an external device;
wherein the communication node is operably coupled to the processor;
wherein the communication node is structured to receive the local infrastructure information from the external device.

### Claim 5 (depends on 1)

The system of claim 1, wherein the first sensor comprises a LIDAR system; and
the processor is structured to control the display to display a thumbnail of an output of the LIDAR system.

### Claim 6 (depends on 5)

The system of claim 5, wherein the processor is structured to control the display to display an enlarged view of the output LIDAR system in response to an input from the passenger.

### Claim 7 (depends on 1)

The system of claim 1, wherein the first condition comprises a predetermined level of passenger dissatisfaction; and
the processor is structured to, in response to the passenger satisfaction index satisfying the predetermined level of passenger dissatisfaction, control the automated driving system to increase deliberateness of the driving behavior of the vehicle.

### Claim 8 (depends on 1)

The system of claim 1, wherein the first property comprises a passenger frustration index IF or a passenger trust index IT.

### Claim 9 (depends on 8)

The system of claim 8, wherein the passenger frustration index IF is a function of a road monitoring duration value, a secondary activity duration value, a multi-task activity transaction value, a side window glance value, or a facial gesture value.

### Claim 10 (depends on 8)

The system of claim 8, wherein the passenger frustration index IF is a function of a galvanic skin response value, a skin temperature value, a verbal valence value, or a facial gesture value.

### Claim 11 (depends on 1)

The system of claim 1, further comprising a communication node structured to communicate with an external device; and
wherein the communication node is operably coupled to the processor;
wherein the second sensor is a smart device worn by the passenger;
wherein the communication node is structured to communicate with the smart device to receive the first property; and
wherein the first property comprises a passenger galvanic skin response, a passenger skin temperature, or a passenger heart rate.

### Claim 12 (depends on 1)

The system of claim 1, further comprising a communication node structured to communicate with an external device; and
wherein the communication node is operably coupled to the processor;
wherein the communication node is structured to receive traffic data, weather data, passenger social data, passenger calendar data, or destination data from the external device; and
wherein the processor is structured to modify the passenger satisfaction index based on the traffic data, the weather data, the passenger social data, the passenger calendar data, or the destination data.

### Claim 13

A method for increasing satisfaction of a passenger in a vehicle having an automated driving system structured to operate the vehicle, the method comprising:
detecting, with a first sensor, a driving environment of the vehicle;
obtaining local infrastructure information;
calculating, with a processor, a vehicle path plan based on the driving environment of the vehicle and the local infrastructure information;
displaying, on a display, a graphical representation of the driving environment of the vehicle and the vehicle path plan;
detecting, with a second sensor, a first property of the passenger;
calculating, with the processor, a passenger satisfaction index based on the first property of the passenger;
controlling the automated driving system to modify driving behavior of the vehicle in response to the passenger satisfaction index satisfying a first condition; and
displaying, on the display, a notification of a change in driving behavior of the vehicle.

### Claim 14 (depends on 13)

The method of claim 13, wherein the displaying a graphical representation comprises displaying a current speed of the vehicle, an acceleration status of the vehicle, an indication of nearby objects, and the local infrastructure information.

### Claim 15 (depends on 13)

The method of claim 13, wherein the displaying the graphical representation comprises indicating an upcoming maneuver based on the vehicle path plan.

### Claim 16 (depends on 13)

The method of claim 13, wherein the first condition comprises a predetermined level of passenger dissatisfaction; and
the method further comprises, in response to the passenger satisfaction index satisfying the predetermined level of dissatisfaction, controlling the automated driving system to increase deliberateness of the driving behavior of the vehicle.

### Claim 17 (depends on 13)

The method of claim 13, further comprising communicating with the second sensor via a communication node of the vehicle;
wherein the second sensor comprises a smart device worn by the passenger;
wherein the first property comprises a passenger galvanic skin response, a passenger skin temperature, or a passenger heart rate.

### Claim 18 (depends on 13)

The method of claim 13, further comprising receiving, via a communication node of the vehicle, traffic data, weather data, passenger social data, passenger calendar data, or destination data from an external device;
wherein the passenger satisfaction index is modified based on the traffic data, the weather data, the passenger social data, the passenger calendar data, or the destination data.