# US 2024/0189715 A1 · Determining user interaction with user interface elements

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0189715 A1
- Document ID: US20240189715A1
- Status: Published application
- Kind: A1
- Application number: 18/063,491
- Filed: 2022-12-08
- Published: 2024-06-13
- Earliest claimed: 2022-12-08
- Assignees: Dell Products L.P.
- Applicants: Dell Products L.P.
- Inventors: Ryan Nicholas Comer; Jake Mitchell Leland; Erik Summa; Jyoti Jasmine Luu
- Web page: https://patentgrep.com/patent/US20240189715A1

## CPC

- A63F 13/537: using indicators, e.g. showing the condition of a game character on screen
- A63F 13/533: for prompting the player, e.g. by displaying a game menu
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06T 11/00: Two-dimensional [2D] image generation
- A63F 2300/303: for displaying additional data, e.g. simulating a Head Up Display
- A63F 2300/308: Details of the user interface
- G06F 2203/04804: Transparency, e.g. transparent or translucent windows
- G06T 2200/24: involving graphical user interfaces [GUIs]
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods described herein may provide a system that adjusts a user interface (UI) of a game in a non-obtrusive way to a user. The system may determine the least obtrusive area(s) on the screen for a UI element game overlay to be presented, at least in part by determining an importance level for area on the screen containing a UI element. The importance level for an area on the screen may be determined using a generated heat map that indicates the importance values of various areas of the screen. The heat map may be generated with inputs including one or more of user interactions with UI elements, game context, UI element rate of change, user matching, and user preferences. The system may temporarily hide UI elements from display that are determined to be below a threshold importance level.

## Claims

### Claim 1

A method, comprising:
detecting a plurality of user interface (UI) elements included in a UI of an application displayed on a display of a device;
receiving indications of user interactions with the plurality of UI elements; and
generating a heat map based on the user interactions with the plurality of UI elements, the heat map indicating a frequency of user interaction with each of a plurality of areas of the display of the device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving an indication of information to provide on the display overlaid on the application;
determining an area of the display that is free of overlap with one of the plurality of UI elements and sized to accommodate a UI overlay comprising the information; and
displaying the UI overlay in the determined area and overlayed over the application.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
determining an area of the display having a frequency of user interaction below a predetermined threshold value based on the generated heat map; and
displaying a UI overlay in the determined area and overlayed over the application.

### Claim 4 (depends on 3)

The method of claim 3, further comprising determining a transparency value for the UI overlay based on the frequency of user interaction of the determined area.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
identifying a UI element disposed in an area of the display having a frequency of user interaction below a predetermined threshold value based on the generated heat map; and
hiding the identified UI element from presentation on the display.

### Claim 6 (depends on 5)

The method of claim 5, wherein hiding the UI element includes determining regions of the display for which a UI layer is hidden, the UI layer including the plurality of UI elements.

### Claim 7 (depends on 5)

The method of claim 5, further comprising displaying the hidden UI element in response to receiving an indication of a user interaction with the area of the display including the hidden UI element.

### Claim 8 (depends on 1)

The method of claim 1, wherein detecting the plurality of UI elements includes detecting coordinates of each of the plurality of UI elements on the display.

### Claim 9 (depends on 1)

The method of claim 1, wherein generating the heat map is further based on activity levels of each of the plurality of UI elements, the activity levels measuring a rate of change of a respective UI element.

### Claim 10 (depends on 1)

The method of claim 1, wherein generating the heat map is further based on rules set by the user regarding at least one area of the plurality of areas of the display of the device.

### Claim 11

An information handling system, comprising:
a memory;
a processor coupled to the memory, wherein the processor is configured to perform steps comprising:
detecting a plurality of user interface (UI) elements included in a UI of an application displayed on a display of a device;
receiving indications of user interactions with the plurality of UI elements; and
generating a heat map based on the user interactions with the plurality of UI elements, the heat map indicating a frequency of user interaction with each of a plurality of areas of the display of the device.

### Claim 12 (depends on 11)

The information handling system of claim 11, the steps further comprising:
determining an area of the display that is free of overlap with one of the plurality of UI elements and sized to accommodate a UI overlay; and
displaying the UI overlay in the determined area and overlayed over the application.

### Claim 13 (depends on 11)

The information handling system of claim 11, the steps further comprising:
determining an area of the display having a frequency of user interaction below a predetermined threshold value based on the generated heat map; and
displaying a UI overlay in the determined area and overlayed over the application.

### Claim 14 (depends on 13)

The information handling system of claim 13, the steps further comprising determining a transparency value for the UI overlay based on the frequency of user interaction of the determined area.

### Claim 15 (depends on 11)

The information handling system of claim 11, the steps further comprising:
identifying a UI element disposed in an area of the display having a frequency of user interaction below a predetermined threshold value based on the generated heat map; and
hiding the identified UI element from presentation on the display.

### Claim 16 (depends on 15)

The information handling system of claim 15, wherein hiding the UI element includes determining regions of the display for which a UI layer is hidden, the UI layer including the plurality of UI elements.

### Claim 17 (depends on 15)

The information handling system of claim 15, the steps further comprising displaying the hidden UI element in response to receiving an indication of a user interaction with the area of the display including the hidden UI element.

### Claim 18

A computer program product, comprising:
a non-transitory computer readable medium comprising code for performing steps comprising:
detecting a plurality of user interface (UI) elements included in a UI of an application displayed on a display of a device;
receiving indications of user interactions with the plurality of UI elements; and
generating a heat map based on the user interactions with the plurality of UI elements, the heat map indicating a frequency of user interaction with each of a plurality of areas of the display of the device.

### Claim 19 (depends on 18)

The computer program product of claim 18, wherein the medium further comprises code for performing steps comprising:
receiving an indication of information to provide on the display overlaid on the application;
determining an area of the display that is free of overlap with one of the plurality of UI elements and sized to accommodate a UI overlay comprising the information; and
displaying the UI overlay in the determined area and overlayed over the application.

### Claim 20 (depends on 18)

The computer program product of claim 18, wherein the medium further comprises code for performing steps comprising:
identifying a UI element disposed in an area of the display having a frequency of user interaction below a predetermined threshold value based on the generated heat map; and
hiding the identified UI element from presentation on the display.
