US 12,222,512 B2 · Displaying content based on state information

Abstract

A head-mounted device may determine contextual information by analyzing sensor data. The head-mounted device may use computer vision analysis to determine contextual information from images of the physical environment around the head-mounted device. Instead or in addition, the head-mounted device may determine contextual information by receiving state information directly from external equipment within the physical environment. Based on the received state information, the head-mounted device may display content, play audio, change a device setting on the head-mounted device, the external equipment, and/or additional external equipment, and/or may open an application. The head-mounted device may receive the state information in accordance with identifying the external equipment in images of the physical environment. The head-mounted device may receive the state information and then obtain and analyze sensor data to determine contextual information in response to the received state information being insufficient.

Claims21

Dependent claims

An electronic device comprising:

one or more sensors comprising one or more cameras;
communication circuitry;
one or more processors;
and
memory storing instructions configured to be executed by the one or more processors, the instructions for:
capturing, via the one or more cameras, one or more images;
analyzing the one or more images, wherein analyzing the one or more images comprises detecting an external electronic device in the one or more images;
obtaining, via the communication circuitry, state information from the external electronic device;
and
in accordance with a determination that the state information does not satisfy a criterion:
capturing, via the one or more cameras, one or more additional images;
estimating state information for the external electronic device based on the one or more additional images;
and
presenting content that is based on the estimated state information for the external electronic device.

11 more elements

A method of operating an electronic device that comprises one or more sensors and communication circuitry, the one or more sensors comprising one or more cameras and the method comprising:

capturing, via the one or more cameras, one or more images;
analyzing the one or more images, wherein analyzing the one or more images comprises detecting an external electronic device in the one or more images;
obtaining, via the communication circuitry, state information from the external electronic device;
and
in accordance with a determination that the state information does not satisfy a criterion:
capturing, via the one or more cameras, one or more additional images;
estimating state information for the external electronic device based on the one or more additional images;
and
presenting content that is based on the estimated state information for the external electronic device.

6 more elements

A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device that comprises one or more sensors and communication circuitry, the one or more sensors comprising one or more cameras and the one or more programs including instructions for:

capturing, via the one or more cameras, one or more images;
analyzing the one or more images, wherein analyzing the one or more images comprises detecting an external electronic device in the one or more images;
obtaining, via the communication circuitry, state information from the external electronic device;
and
in accordance with a determination that the state information does not satisfy a criterion:
capturing, via the one or more cameras, one or more additional images;
estimating state information for the external electronic device based on the one or more additional images;
and
presenting content that is based on the estimated state information for the external electronic device.

6 more elements

Dependencies are inferred from the claim text.

Displaying content based on state information

Life of this patent

Displaying content based on state information

Life of this patent

Abstract

Claims21