# US 2024/0201489 A1 · Adaptive Heads-Up Display Using a User Monitoring System and Remote Systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0201489 A1 - Document ID: US20240201489A1 - Status: Published application - Kind: A1 - Application number: 18/156,134 - Filed: 2023-01-18 - Published: 2024-06-20 - Earliest claimed: 2022-12-15 - Assignees: GM Global Technology Operations LLC - Applicants: GM Global Technology Operations LLC - Inventors: Wenyuan Qi - Web page: https://patentgrep.com/patent/US20240201489A1 ## CPC - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - B60K 35/00: Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles - B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor - B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver - B60K 35/29: Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions - G02B 27/01: Head-up displays - G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)} - B60K 35/23: Head-up displays [HUD] (optical aspects of head-up displays G02B27/01) - B60K 35/60: Instruments characterised by their location or relative disposition in or on vehicles (arrangements of lighting devices on dashboards B60Q3/10) - B60K 35/654: {the user being the driver} - B60K 35/81: for controlling displays - B60K 2360/334: Projection means - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - G02B 2207/101: Nanooptics - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES ## Abstract Methods and systems for a vehicle for adaptively controlling messages displayed on a heads-up display. A plurality of requests is received by a first controller to display a message associated with each of the plurality of requests on a heads-up display from one or more operational applications. The plurality of requests are filtered by an arbitration gateway based on user attention data gathered from a user monitoring system and a user behavior history. A priority is assigned to each of the plurality of filtered requests based on a user interest prediction model, at least a portion of the plurality of the filtered requests are selected based on the assigned priority, and the messages associated with the selected portion of the plurality of filtered requests are displayed on the heads-up display. ## Claims ### Claim 1 A method of adaptively controlling messages displayed on a heads-up display, comprising: receiving a plurality of requests to display a message associated with each of the plurality of requests on a heads-up display from one or more operational applications; filtering the plurality of requests by an arbitration gateway based on user attention data gathered from a user monitoring system and a user behavior history; assigning a priority to each of the plurality of filtered requests based on a user interest prediction model; selecting at least a portion of the plurality of the filtered requests based on the assigned priority; and displaying the messages associated with the selected portion of the plurality of filtered requests on the heads-up display. ### Claim 2 (depends on 1) The method of claim 1, further comprising dropping the message associated with at least one of the plurality of filtered requests that was not selected based on the assigned priority. ### Claim 3 (depends on 2) The method of claim 2, further comprising displaying the dropped message associated with at least one of the plurality of filtered requests on the heads-up display that was not selected based on the assigned priority. ### Claim 4 (depends on 2) The method of claim 2, further comprising providing at least one of a sound and a haptic response when the dropped message is related to safety. ### Claim 5 (depends on 1) The method of claim 1, further comprising gathering user attention data using an attention sensor, wherein the attention sensor includes at least one of a time-of-flight sensor and a camera. ### Claim 6 (depends on 1) The method of claim 1, further comprising calculating the user interest prediction model based upon a surrounding environmental event and the user behavior history. ### Claim 7 (depends on 1) The method of claim 1, further comprising filtering the plurality of requests by calculating a filtering tuple for each of the plurality of requests. ### Claim 8 (depends on 7) The method of claim 7, further comprising calculating the filtering tuple by characterizing each of the plurality of requests according to the following elements: an ID, a priority, an emergency level, a user attention status, a user behavior history, a heads-up display attribute, and raw data. ### Claim 9 (depends on 8) The method of claim 8, further comprising assigning the priority of each of the plurality of filtered requests by weighting the filtering tuple for each of the plurality of filtered requests; weighting a prioritizing tuple for each of the plurality of filtered requests; and adding the weighted filtering tuple and the weighted prioritizing tuple. ### Claim 10 (depends on 9) The method of claim 9, further comprising calculating the prioritizing tuple by characterizing each of the plurality of requests according to the following elements: an event_ID, a priority, an emergency level, a predictive user interest, a heads-up display attribute, a lead time, and raw event data. ### Claim 11 An on-board adaptive heads-up display system for a vehicle, comprising: a first controller; a heads-up display connected the first controller; and a user monitoring system connected to the first controller; wherein the first controller is configured to execute instructions to: receive a plurality of requests to display a message associated with each of the plurality of requests on a heads-up display from one or more operational applications; filter the plurality of requests by an arbitration gateway based on user attention data gathered from the user monitoring system and a user behavior history; assign a priority to each of the plurality of filtered requests based on a user interest prediction model; select at least a portion of the plurality of the filtered requests based on the assigned priority; and display the messages associated with the selected portion of the plurality of the filtered requests on the heads-up display. ### Claim 12 (depends on 11) The on-board adaptive heads-up display system for a vehicle of claim 11, wherein the controller is further configured to execute instructions to drop the message associated with the at least one of the plurality of filtered requests that was not selected based on the assigned priority. ### Claim 13 (depends on 12) The on-board adaptive heads-up display system for a vehicle of claim 12, wherein the controller is further configured to execute instructions to provide at least one of a sound and a haptic response when the dropped message is related to safety. ### Claim 14 (depends on 11) The on-board adaptive heads-up display system for a vehicle of claim 11, wherein the controller is further configured to execute instructions to gather user attention data using an attention sensor, wherein the attention sensor includes at least one of a time-of-flight sensor and a camera. ### Claim 15 (depends on 14) The on-board adaptive heads-up display system for a vehicle of claim 14, wherein the controller is further configured to execute instructions to configure a heads-up display attribute of the messages using user inputs received from a user input device. ### Claim 16 (depends on 11) The on-board adaptive heads-up display system for a vehicle of claim 11, wherein controller is further configured to execute instructions to filter the plurality of requests using a filtering tuple to characterize each of the plurality of requests by the arbitration gateway including the following elements: an ID, a priority, an emergency level, a user attention status, a user behavior history, a heads-up display attributes, and raw data. ### Claim 17 (depends on 16) The on-board adaptive heads-up display system for a vehicle of claim 16, wherein controller is further configured to execute instructions to assign a priority to the plurality of filtered requests by weighting the filtering tuple for each of the plurality of filtered requests; weighting a prioritizing tuple based on the user interest prediction model; and adding the weighted filtering tuple and the weighted prioritizing tuple to calculate a prioritizing score. ### Claim 18 (depends on 17) The on-board adaptive heads-up display system for a vehicle of claim 17, wherein controller is further configured to execute instructions to calculate the prioritizing tuple by characterizing each of the plurality of filtered requests according to the following elements: an event_ID, a priority, an emergency level, a predictive user interest, a heads-up display attribute, a lead time, and raw event data. ### Claim 19 An adaptive heads-up display system for a vehicle, comprising: an on-board adaptive heads-up display system including a first controller, a heads-up display connected the first controller, a user monitoring system connected to the first controller, and a first communication system connected to the first controller; and a remote system including a remote controller and a remote communication system connected to the remote controller, wherein the remote controller is configured to wirelessly communicate information to the first communication system, wherein the first controller is configured to execute instructions to: receive a plurality of requests to display a message associated with each of the plurality of requests on a heads-up display from one or more operational applications; filter the plurality of requests by an arbitration gateway based on user attention data gathered from the user monitoring system and a user behavior history; assign a priority to each of the plurality of filtered requests based on a user interest prediction model calculated by the remote system; select at least a portion of the plurality of the filtered requests based on the assigned priority; and display the messages associated with the selected portion of the plurality of the filtered requests on the heads-up display. ### Claim 20 (depends on 19) The adaptive heads-up display system for a vehicle of claim 19, wherein the remote controller is configured to execute instructions to: calculate a user prediction model based upon prior behavior gathered from the user monitoring system and data regarding a surrounding environment.