# US 11,442,608 B1 · Preserving engagement state based on contextual signals > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,442,608 B1 - Document ID: US11442608B1 - Status: Granted - Kind: B1 - Application number: 17/444,656 - Filed: 2021-08-06 - Granted: 2022-09-13 - Earliest claimed: 2021-08-06 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Brett Aladdin Barros; Theophile Vincent Goguely - Web page: https://patentgrep.com/patent/US11442608B1 ## CPC - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method for dynamically changing a graphical user interface element occurs in response to detecting that a temporal user interface element displayed on a user interface of user device. The method includes receiving, at the user device, a contextual signal characterizing a state of a user. The method further includes determining, by the user device, that the contextual signal characterizing the state of the user is indicative of the user intending to interact with a temporal user interface element. The methods also include, in response to determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with a temporal user interface element, modifying a respective state of the temporal user interface element displayed on the user interface of the user device. ## Claims ### Claim 1 A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations comprising: displaying a temporal user-interface element on a user interface of a user device for a timeout duration, the temporal user-interface element comprising a prompt for a user to initiate fulfillment of a perceived command detected in streaming audio captured by the user device by providing a user input indication indicating selection of the temporal user-interface element while the temporal user-interface element is displayed during the timeout duration; in response to detecting that the temporal user-interface element is displayed on the user interface of the user device: receiving, at the user device, a contextual signal characterizing a state of a user; determining, by the user device, that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element; and in response to determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element, modifying the timeout duration of the temporal user-interface element displayed on the user interface of the user device. ### Claim 2 (depends on 1) The method of claim 1, wherein the state of the user comprises an engagement state indicative of the user attempting to engage or engaging with the temporal user-interface element displayed on the user interface of the user device. ### Claim 3 (depends on 2) The method of claim 2, wherein modifying the timeout duration of the temporal user-interface element comprises increasing the timeout duration of the temporal user-interface element. ### Claim 4 (depends on 2) The method of claim 2, wherein modifying the timeout duration of the temporal user-interface element comprises suspending the timeout duration of the temporal user-interface element. ### Claim 5 (depends on 1) The method of claim 1, wherein the state of the user comprises a disengagement state indicative of the user disengaging with the temporal user-interface element displayed on the user interface of the user device. ### Claim 6 (depends on 5) The method of claim 5, wherein, in response to determining that the contextual signal characterizing the state of the user comprises the disengagement state, modifying the timeout duration of the temporal user-interface element comprises removing the temporal user-interface element prior to an expiration of the timeout duration for the temporal user-interface element. ### Claim 7 (depends on 5) The method of claim 5, wherein, in response to determining that the contextual signal characterizing the state of the user comprises the disengagement state, modifying the timeout duration of the temporal user-interface element comprises decreasing the timeout duration of the temporal user-interface element. ### Claim 8 (depends on 1) The method of claim 1, wherein the contextual signal comprises a user-proximity signal indicating a proximity of the user relative to the user device. ### Claim 9 (depends on 1) The method of claim 1, wherein the contextual signal comprises a presence detection signal indicating a presence of the user within a field of view of a sensor associated with the user device. ### Claim 10 (depends on 1) The method of claim 1, wherein the contextual signal comprises an attention detection signal indicating an attention of the user with respect to the user device. ### Claim 11 (depends on 1) The method of claim 1, wherein: the contextual signal comprises a presence detection signal indicating a presence of the user within a field of view of a sensor associated with the user device; determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element comprises determining that the presence detection signal indicates that the presence of the user within the field of view of the sensor has changed from not present to present; and modifying the timeout duration of the temporal user-interface element displayed on the user interface comprises increasing the timeout duration or suspending the timeout duration of the temporal user-interface element. ### Claim 12 (depends on 1) The method of claim 1, wherein: the contextual signal comprises a user-proximity signal indicating a proximity of the user relative to the user device; determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element comprises determining that the user-proximity signal indicates that the proximity of the user relative to the user device has changed to be closer to the user device; and modifying the timeout duration of the temporal user-interface element displayed on the user interface comprises increasing the timeout duration or suspending the timeout duration of the temporal user-interface element. ### Claim 13 (depends on 1) The method of claim 1, wherein: the contextual signal comprises an attention detection signal indicating an attention of the user with respect to the user device; determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element comprises determining that the attention detection signal indicates that the attention of the user has changed to focus on the user device; and modifying the timeout duration of the temporal user-interface element displayed on the user interface comprises increasing the timeout duration or suspending the timeout duration of the temporal user-interface element. ### Claim 14 A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations comprising: in response to detecting that a temporal user-interface element is displayed on a user interface of a user device: receiving, at the user device, a contextual signal characterizing a state of a user; determining, by the user device, that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element; and in response to determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element: determining that the respective state of the temporal user-interface element has failed to have been previously modified a threshold number of times within a period of time; and modifying a respective state of the temporal user-interface element displayed on the user interface of the user device. ### Claim 15 A system comprising: data processing hardware; and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising: displaying a temporal user-interface element on a user interface of a user device for a timeout duration, the temporal user-interface element comprising a prompt for a user to initiate fulfillment of a perceived command detected in streaming audio captured by the user device by providing a user input indication indicating selection of the temporal user-interface element while the temporal user-interface element is displayed during the timeout duration; in response to detecting that the temporal user-interface element is displayed on the user interface of the user device: receiving, at the user device, a contextual signal characterizing a state of a user; determining, by the user device, that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element; and in response to determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element, modifying the timeout duration of the temporal user-interface element displayed on the user interface of the user device. ### Claim 16 (depends on 15) The system of claim 15, wherein the state of the user comprises an engagement state indicative of the user attempting to engage or engaging with the temporal user-interface element displayed on the user interface of the user device. ### Claim 17 (depends on 16) The system of claim 16, wherein modifying the timeout duration of the temporal user-interface element comprises increasing the timeout duration of the temporal user-interface element. ### Claim 18 (depends on 16) The system of claim 16, wherein modifying the timeout duration of the temporal user-interface element comprises suspending the timeout duration of the temporal user-interface element. ### Claim 19 (depends on 15) The system of claim 15, wherein the state of the user comprises a disengagement state indicative of the user disengaging with the temporal user-interface element displayed on the user interface of the user device. ### Claim 20 (depends on 19) The system of claim 19, wherein, in response to determining that the contextual signal characterizing the state of the user comprises the disengagement state, modifying the timeout duration of the temporal user-interface element comprises removing the temporal user-interface element prior to an expiration of the timeout duration for the temporal user-interface element. ### Claim 21 (depends on 19) The system of claim 19, wherein, in response to determining that the contextual signal characterizing the state of the user comprises the disengagement state, modifying the timeout duration of the temporal user-interface element comprises decreasing the timeout duration of the temporal user-interface element. ### Claim 22 (depends on 15) The system of claim 15, wherein the contextual signal comprises a user-proximity signal indicating a proximity of the user relative to the user device. ### Claim 23 (depends on 15) The system of claim 15, wherein the contextual signal comprises a presence detection signal indicating a presence of the user within a field of view of a sensor associated with the user device. ### Claim 24 (depends on 15) The system of claim 15, wherein the contextual signal comprises an attention detection signal indicating an attention of the user with respect to the user device. ### Claim 25 (depends on 15) The system of claim 15, wherein: the contextual signal comprises a presence detection signal indicating a presence of the user within a field of view of a sensor associated with the user device; determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element comprises determining that the presence detection signal indicates that the presence of the user within the field of view of the sensor has changed from not present to present; and modifying the timeout duration of the temporal user-interface element displayed on the user interface comprises increasing the timeout duration or suspending the timeout duration of the temporal user-interface element. ### Claim 26 (depends on 15) The system of claim 15, wherein: the contextual signal comprises a user-proximity signal indicating a proximity of the user relative to the user device; determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element comprises determining that the user-proximity signal indicates that the proximity of the user relative to the user device has changed to be closer to the user device; and modifying the timeout duration of the temporal user-interface element displayed on the user interface comprises increasing the timeout duration or suspending the timeout duration of the temporal user-interface element. ### Claim 27 (depends on 15) The system of claim 15, wherein: the contextual signal comprises an attention detection signal indicating an attention of the user with respect to the user device; determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element comprises determining that the attention detection signal indicates that the attention of the user has changed to focus on the user device; and modifying the timeout duration of the temporal user-interface element displayed on the user interface comprises increasing the timeout duration or suspending the timeout duration of the temporal user-interface element. ### Claim 28 A system comprising: data processing hardware; and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising: in response to detecting that a temporal user-interface element is displayed on a user interface of a user device: receiving, at the user device, a contextual signal characterizing a state of a user: determining, by the user device, that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element; and in response to determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with the temporal user-interface element: determining that the respective state of the temporal user-interface element has failed to have been previously modified a threshold number of times within a period of time; and modifying a respective state of the temporal user-interface element displayed on the user interface of the user device.