# US 11,994,675 B2 · Head-mounted device for tracking screen time > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,994,675 B2 - Document ID: US11994675B2 - Status: Granted - Kind: B2 - Application number: 17/663,444 - Filed: 2022-05-16 - Published: 2023-11-16 - Granted: 2024-05-28 - Earliest claimed: 2022-05-16 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Ruofei Du; Maksym Dzitsiuk - Web page: https://patentgrep.com/patent/US11994675B2 ## CPC - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 1/163: {Wearable computers, e.g. on a belt} - G06F 1/3231: Monitoring the presence, absence or movement of users - G08B 25/016: {Personal emergency signalling and security systems (emergency non-personal manually actuated alarm activators G08B25/12)} - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G16H 40/63: for local operation - G16H 50/30: for calculating health indices; for individual health risk assessment - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract A head-mounted device, such as augmented reality glasses, can be configured to track screen time. The head-mounted device can advantageously track screen time across a variety of devices whether or not they are configured to such measurements. Various modes for tracking the screen time may be used based on accuracy, power consumption, and privacy. The head-mounted device may track in a mode based on a battery level, a device setting or a user input. The head-mounted device may accumulate screen times in a database and in some implementations, can link screen times with devices in the database based on device information gathered by the head-mounted device during the tracking process. The head-mounted device may be configured to generate and display an alert based on one or more rules about screen time. ## Claims ### Claim 1 A computer-implemented method, comprising: capturing an eye image of an eye using an eye-tracking camera of a head-mounted device, the eye image being an image of a surface of the eye of a user wearing the head-mounted device; analyzing the eye image to identify a screen reflection, the screen reflection corresponding to a screen that is positioned in front of the user and reflecting off the surface of the eye of the user; starting a screen timer after the screen reflection is identified; tracking the screen reflection over time; and stopping the screen timer when the screen reflection is no longer identified. ### Claim 2 (depends on 1) The computer-implemented method according to claim 1, wherein the tracking the screen reflection over time includes repeating the capturing and the analyzing at a cycle period to periodically capture subsequent eye images to track the screen reflection over time. ### Claim 3 (depends on 2) The computer-implemented method according to claim 2, wherein the cycle period is greater than 2 seconds. ### Claim 4 (depends on 2) The computer-implemented method according to claim 2, wherein the cycle period is less than 2 seconds. ### Claim 5 (depends on 1) The computer-implemented method according to claim 1, further comprising detecting that the head-mounted device is in a lower-power mode for screen time tracking, wherein the detecting the lower-power mode includes: detecting a battery level of the head-mounted device; and determining that the battery level is below a threshold. ### Claim 6 (depends on 1) The computer-implemented method according to claim 1, wherein analyzing the eye image to identify the screen reflection includes: applying the eye image to an input of a neural network, the eye image including the eye and the screen reflection; and receiving a screen image at an output of the neural network, the screen image including the screen reflection. ### Claim 7 (depends on 1) The computer-implemented method according to claim 1, further including: detecting that the head-mounted device is in a higher-power mode for screen time tracking; capturing a field-of-view image using a front facing camera of the head-mounted device; analyzing the field-of-view image to identify the screen; starting the screen timer after the screen is identified; periodically capturing subsequent field-of-view images to track the screen over time; and stopping the screen timer when the screen is no longer identified in the subsequent field-of-view images. ### Claim 8 (depends on 7) The computer-implemented method according to claim 7, wherein detecting that the head-mounted device is in the higher-power mode for the screen time tracking includes: detecting a battery level of the head-mounted device; and determining that the battery level is above a threshold. ### Claim 9 (depends on 7) The computer-implemented method according to claim 7, further including: detecting that the head-mounted device is in a no camera mode for the screen time tracking; capturing orientation data of the head-mounted device using at least one orientation sensor of the head-mounted device; capturing relative location data of a device from an ultra-wideband (UWB) signal received at the head-mounted device; analyzing the orientation data and the relative location data to determine a viewing condition; starting the screen timer after the viewing condition is identified; periodically capturing subsequent orientation data and relative location data to track the viewing condition over time; and stopping the screen timer when the viewing condition can no longer be tracked. ### Claim 10 (depends on 9) The computer-implemented method according to claim 9, further including: recording a screen time in a database based on the screen timer; and generating an alert when the screen time satisfies a criterion. ### Claim 11 (depends on 10) The computer-implemented method according to claim 10, further including: classifying instances of screen times as corresponding to devices based on device information captured by the head-mounted device. ### Claim 12 A head-mounted device comprising: an eye-tracking camera; and a processor in communication with the eye-tracking camera, the processor configured to by software instructions to: receive an eye image of an eye from the eye-tracking camera, the eye image being an image of a surface of the eye of a user wearing the head-mounted device; analyze the eye image to identify a screen reflection, the screen reflection corresponding to a screen that is position in front of the user and reflecting off the surface of the eye of the user; start a screen timer after the screen reflection is identified; track the screen reflection over time; and stop the screen timer when the screen reflection is no longer identified. ### Claim 13 (depends on 12) The head-mounted device according to claim 12, wherein the processor is further configured by software instructions to: repeatedly receive and analyze at a cycle period to periodically capture subsequent eye images to track the screen reflection over time. ### Claim 14 (depends on 12) The head-mounted device according to claim 12, wherein the processor is further configured by software instructions to: determine that a battery level of the head-mounted device is below a threshold; and detect that the head-mounted device is in a lower-power mode for screen time tracking based on the determination. ### Claim 15 (depends on 12) The head-mounted device according to claim 12, wherein the processor is further configured by software instructions to: apply the eye image to an input of a neural network, the eye image include the eye and the screen reflection; and receiving a screen image at an output of the neural network, the screen image including the screen reflection. ### Claim 16 (depends on 12) The head-mounted device according to claim 12, wherein the processor is further configured by software instructions to: detect that the head-mounted device is in a higher-power mode for screen time tracking; capture a field-of-view image using a front facing camera of the head-mounted device; analyze the field-of-view image to identify the screen; start the screen timer after the screen is identified; periodically capturing subsequent field-of-view images to track the screen over time; and stop the screen timer when the screen is no longer identified in the subsequent field-of-view images. ### Claim 17 (depends on 16) The head-mounted device according to claim 16, wherein the processor is further configured by software instructions to: determine that a battery level of the head-mounted device is above a threshold; and detect that the head-mounted device is in a higher-power mode for the screen time tracking based on the determination. ### Claim 18 (depends on 16) The head-mounted device according to claim 16, wherein the processor is further configured by software instructions to: detect that the head-mounted device is in a no camera mode for the screen time tracking; receive orientation data of the head-mounted device from at least one orientation sensor of the head-mounted device; receive relative location data of a device from an ultra-wideband (UWB) signal received at the head-mounted device; analyze the orientation data and the relative location data to determine a viewing condition; start the screen timer after the viewing condition is identified; periodically receive subsequent orientation data and relative location data to track the viewing condition over time; and stop the screen timer when the viewing condition can no longer be tracked. ### Claim 19 (depends on 18) The head-mounted device according to claim 18, wherein the processor is further configured by software instructions to: record a screen time in a database based on the screen timer; and generate an alert when the screen timer satisfies a criterion. ### Claim 20 (depends on 19) The head-mounted device according to claim 19. wherein the processor is further configured by software instructions to: classify instances of screen times as corresponding to devices based on device information captured by the head-mounted device.