# US 2023/0058508 A1 · System amd method for providing situational awareness interfaces for a vehicle occupant

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0058508 A1
- Document ID: US20230058508A1
- Status: Published application
- Kind: A1
- Application number: 17/445,450
- Filed: 2021-08-19
- Published: 2023-02-23
- Earliest claimed: 2021-08-19
- Assignees: GM Global Technology Operations LLC
- Applicants: GM Global Technology Operations LLC
- Inventors: Lawrence A. Bush; Joseph F. Szczerba; Roy J. Mathieu
- Web page: https://patentgrep.com/patent/US20230058508A1

## CPC

- G01C 21/3641: {Personalized guidance, e.g. limited guidance on previously travelled routes}
- B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
- B60K 35/22: Display screens
- B60K 35/25: using haptic output
- B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
- B60K 35/80: Arrangements for controlling instruments
- B60W 40/105: Speed
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- B60W 50/16: Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
- B60W 60/00: Drive control systems specially adapted for autonomous road vehicles
- B60W 60/0015: {specially adapted for safety}
- B60W 60/005: {Handover processes (Handing over between remote control and on-board control or handing over between remote control arrangements G05D1/227)}
- G01C 21/3453: {Special cost functions, i.e. other than distance or default speed limit of road segments}
- G01C 21/3632: {Guidance using simplified or iconic instructions, e.g. using arrows (G01C21/365 takes precedence)}
- G01C 21/3676: {Overview of the route on the road map}
- G01C 21/3697: {Output of additional, non-guidance related information, e.g. low fuel level (G01C21/3679 takes precedence)}
- B60K 2360/166: Navigation
- B60K 2360/171: Vehicle or relevant part thereof displayed
- B60K 2360/175: Autonomous driving
- B60W 2050/146: {Display means}
- B60W 2555/60: Traffic rules, e.g. speed limits or right of way
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT

## Abstract

A method for providing a situational awareness interface for an occupant of an automated vehicle is provided. The method includes: generating a vehicle route graphic that displays a street level map of an area that includes a current location of the vehicle and an intended destination of the vehicle along with a visual depiction of a planned route and one or more alternate routes for the vehicle; determining a probability of requiring assistance or interaction and an estimated time of arrival for each of the planned route and the one or more alternate routes; providing a visual indication of the probability of requiring assistance or interaction and the estimated time of arrival for each of the planned route and the one or more alternate routes on the vehicle route graphic; and signaling a display device to display the vehicle route graphic.

## Claims

### Claim 1

A supervisory control system for providing a situational awareness interface for an occupant of an automated vehicle, the system comprising a controller configured to:
generate a bird's eye view graphic that provides an indication of vehicle maneuver intent;
generate a vehicle route graphic that displays a street level map of an area that includes a current location of the vehicle and an intended destination of the vehicle along with a visual depiction of a planned route and one or more alternate routes for the vehicle from the current location of the vehicle to the intended destination;
determine a probability of requiring assistance or interaction and an estimated time of arrival for each of the planned route and the one or more alternate routes;
provide a visual indication of the probability of requiring assistance or interaction and the estimated time of arrival for each of the planned route and the one or more alternate routes on the vehicle route graphic;
configure the visual depiction of the planned route and the one or more alternate routes to be selectable, wherein when a route is selected, the controller instructs the vehicle to navigate in accordance with the selected route; and
signal a display device to display the situational awareness interface which includes the bird's eye view graphic and the vehicle route graphic.

### Claim 2 (depends on 1)

The supervisory control system of claim 1, wherein the controller is further configured to:
identify a readiness state for a plurality of vehicle systems for autonomous travel;
generate a ready-state graphic that provides a visual indication of the readiness state for the plurality of vehicle systems; and
signal the display device to display the ready-state graphic as part of the situational awareness interface.

### Claim 3 (depends on 1)

The supervisory control system of claim 1, wherein the controller is further configured to:
generate an occupant control interface that includes an assistance interface for enabling the occupant to stop the vehicle or summon help; and
provide the occupant control interface in the situational awareness interface.

### Claim 4 (depends on 3)

The supervisory control system of claim 3, wherein the controller is further configured to:
provide an in-vehicle security and emergency services system interface in the occupant control interface for enabling the occupant to access a remote operator for an in-vehicle security and emergency services system.

### Claim 5 (depends on 1)

The supervisory control system of claim 1, wherein the controller is further configured to signal the display device to display an informational graphic informing that a remote operator will take control of the vehicle as part of the situational awareness interface.

### Claim 6 (depends on 1)

The supervisory control system of claim 1, wherein the controller is further configured to:
determine a route from the current location of the vehicle to the intended destination that reduces a likelihood of needing operator intervention; and
provide the route that reduces the likelihood of needing operator intervention as the planned route for the vehicle or the one or more alternate routes for the vehicle.

### Claim 7 (depends on 1)

The supervisory control system of claim 1, wherein the controller is further configured to:
identify a state of an upcoming traffic control device in a direction of vehicle travel from on-board vehicle input sensor data or off board input sensor data;
generate a traffic control device icon representative of the upcoming traffic control device that indicates the state of the upcoming traffic control device; and
overlay the traffic control device icon on the bird's eye view graphic.

### Claim 8 (depends on 1)

The supervisory control system of claim 1, wherein the controller is further configured to:
identify a speed limit in a direction of vehicle travel from on-board vehicle input sensor data or off board input sensor data;
generate a speed limit icon that indicates the speed limit in the direction of vehicle travel; and
overlay the speed limit icon on the bird's eye view graphic.

### Claim 9 (depends on 1)

The supervisory control system of claim 1, wherein the controller is further configured to:
identify a direction of vehicle travel from on-board vehicle input sensor data or off board input sensor data;
generate a vehicle travel direction icon that indicates the direction of vehicle travel; and
overlay the vehicle travel direction icon on the bird's eye view graphic.

### Claim 10 (depends on 1)

The supervisory control system of claim 1, wherein the controller is further configured to:
retrieve LIDAR data from on-board vehicle input sensor data;
generate a LIDAR map graphic that indicates objects sensed via the LIDAR data; and
overlay the LIDAR map graphic on the bird's eye view graphic.

### Claim 11 (depends on 1)

The supervisory control system of claim 1, wherein the controller is further configured to:
retrieve vehicle speed data that indicates current vehicle speed from on-board vehicle input sensor data;
generate a vehicle speed graphic that indicates the current vehicle speed; and
overlay the vehicle speed graphic on the bird's eye view graphic.

### Claim 12 (depends on 7)

The supervisory control system of claim 7, wherein the controller is further configured to:
generate a vehicle acceleration graphic that indicates whether the vehicle is accelerating; and
overlay the vehicle acceleration graphic on the bird's eye view graphic.

### Claim 13 (depends on 1)

The supervisory control system of claim 1, wherein to generate a bird's eye view graphic that provides an indication of vehicle maneuver intent, the controller is configured to:
generate a bird's eye view graphic that provides an elevated view of a roadway surrounding the vehicle;
overlay a direction of travel graphic on the bird's eye view graphic centered at a location consistent with retrieved location data wherein the direction of travel graphic has a predetermined length and width centered around the location;
position, based on the location data, a vehicle icon that is representative of the vehicle on the bird's eye view graphic overlaying the direction of travel graphic;
retrieve potential obstacle location data for each potential obstacle identified within a predetermined distance surrounding the vehicle; and
for each identified potential obstacle, position an obstacle icon representative of the potential obstacle on the bird's eye view graphic.

### Claim 14 (depends on 13)

The supervisory control system of claim 13, wherein the controller is further configured to:
assess whether vehicle travel into an area represented by an area of the direction of travel graphic could result in unsafe vehicle operation; and
apply a specific color to the area of the direction of travel graphic when vehicle travel into the area represented by the area of the direction of travel graphic could result in unsafe vehicle operation.

### Claim 15 (depends on 1)

The supervisory control system of claim 1, wherein the controller is further configured to control a plurality of haptic actuators to provide haptic feedback to convey a sense of direction to an occupant seated in a seating apparatus while riding in the vehicle.

### Claim 16 (depends on 1)

The supervisory control system of claim 1, wherein the controller is further configured to control a plurality of speakers in a directional speaker system in a vehicle to convey a sense of direction to an occupant seated in a seating apparatus while riding in the vehicle.

### Claim 17

A supervisory control system for providing a situational awareness interface for an occupant of an automated vehicle, the system comprising a controller configured to:
generate a vehicle route graphic that displays a street level map of an area that includes a current location of the vehicle and an intended destination of the vehicle along with a visual depiction of a planned route and one or more alternate routes for the vehicle from the current location of the vehicle to the intended destination;
determine a probability of requiring assistance or interaction and an estimated time of arrival for each of the planned route and the one or more alternate routes;
provide a visual indication of the probability of requiring assistance or interaction and the estimated time of arrival for each of the planned route and the one or more alternate routes on the vehicle route graphic;
configure the visual depiction of the planned route and the one or more alternate routes to be selectable, wherein when a route is selected, the controller instructs the vehicle to navigate in accordance with the selected route; and
signal a display device to display the situational awareness interface which includes the vehicle route graphic.

### Claim 18

A method for providing a situational awareness interface for an occupant of an automated vehicle, the method comprising:
generating a bird's eye view graphic that provides an indication of vehicle maneuver intent; generating a vehicle route graphic that displays a street level map of an area that includes a current location of the vehicle and an intended destination of the vehicle along with a visual depiction of a planned route and one or more alternate routes for the vehicle from the current location of the vehicle to the intended destination;
determining a probability of requiring assistance or interaction and an estimated time of arrival for each of the planned route and the one or more alternate routes;
providing a visual indication of the probability of requiring assistance or interaction and the estimated time of arrival for each of the planned route and the one or more alternate routes on the vehicle route graphic;
configuring the visual depiction of the planned route and the one or more alternate routes to be selectable, wherein when a route is selected, the vehicle is instructed to navigate in accordance with the selected route; and
signaling a display device to display the situational awareness interface which includes the bird's eye view graphic and the vehicle route graphic.

### Claim 19 (depends on 18)

The method of claim 18, further comprising:
identifying a readiness state for a plurality of vehicle systems for autonomous travel;
generating a ready-state graphic that provides a visual indication of the readiness state for the plurality of vehicle systems; and
signaling the display device to display the ready-state graphic as part of the situational awareness interface.

### Claim 20 (depends on 18)

The method of claim 18, further comprising:
determining a route from the current location of the vehicle to the intended destination that reduces a likelihood of needing operator intervention; and
providing the route that reduces the likelihood of needing operator intervention as the planned route for the vehicle or the one or more alternate routes for the vehicle.
