# US 2019/0318168 A1 · Scene classification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0318168 A1
- Document ID: US20190318168A1
- Status: Published application
- Kind: A1
- Application number: 16/370,231
- Filed: 2019-03-29
- Published: 2019-10-17
- Earliest claimed: 2018-04-13
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Peter MEIER; Michael J. ROCKWELL
- Web page: https://patentgrep.com/patent/US20190318168A1

## CPC

- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G06V 20/35: {Categorising the entire scene, e.g. birthday party or wedding scene}
- G06V 20/20: in augmented reality scenes
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An exemplary process for identifying a type of a physical environment amongst a plurality of types of physical environments is provided. The process includes obtaining, using the one or more cameras, image data corresponding to a physical environment. The process further includes identifying at least one portion of an entity in the physical environment based on the image data; determining, based on the identified at least one portion of the entity, whether the entity is an entity of a first type; determining a type of the physical environment if the entity is an entity of the first type; and presenting one or more virtual objects and a representation of the entity.

## Claims

### Claim 1

An electronic device, comprising:
one or more processors;
one or more cameras; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
obtaining, using the one or more cameras, image data corresponding to a physical environment;
based on the image data, identifying at least one portion of an entity in the physical environment;
determining, based on the identified at least one portion of the entity, whether the entity is an entity of a first type;
in accordance with a determination that the entity is an entity of the first type, determining a type of the physical environment based on the entity; and
presenting one or more virtual objects and a representation of the entity corresponding to the determined type of the physical environment.

### Claim 2 (depends on 1)

The electronic device of claim 1, further comprising:
prior to determining, based on the identified at least one portion of the entity, whether the entity is an entity of the first type:
determining whether the at least one portion of the entity corresponds to at least one portion of a previously-identified entity; and
in accordance with a determination that the at least one portion of the entity does not correspond to at least one portion of a previously-identified entity, storing data indicative of the at least one portion of the entity.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein identifying the at least one portion of the entity in the physical environment comprises:
identifying a first portion of the entity without identifying the entire entity; and
foregoing identifying a second portion of the entity.

### Claim 4 (depends on 1)

The electronic device of claim 1, wherein determining, based on the identified at least one portion of the entity, whether the entity is an entity of the first type comprises:
determining, based on the identified at least one portion of the entity in the physical environment, one or more properties of the entity; and
determining, based on the one or more properties of the entity, whether the entity is an entity of the first type.

### Claim 5 (depends on 4)

The electronic device of claim 4, wherein determining, based on the identified at least one portion of the entity, the one or more properties of the entity comprises:
classifying the entity based on the identified at least one portion of the entity;
obtaining, based on the classification of the entity, one or more properties of the entity from a plurality of properties.

### Claim 6 (depends on 4)

The electronic device of claim 4, wherein the one of more properties of the entity comprise at least one of:
mobility of the entity;
an indication of whether the entity is a building structure; and
consistency of the appearance of the entity.

### Claim 7 (depends on 4)

The electronic device of claim 4, wherein the determining, based on the one or more properties of the entity, whether the entity is an entity of the first type comprises:
determining whether a combination of the one or more properties of the entity exceeds a confidence threshold.

### Claim 8 (depends on 1)

The electronic device of claim 1, wherein determining the type of the physical environment based on the entity comprises:
determining whether the entity corresponds to at least one of a plurality of types of the physical environments; and
in accordance with a determination that the entity corresponds to at least one of the plurality of types of the physical environments, selecting one of the plurality of types of the physical environments.

### Claim 9 (depends on 8)

The electronic device of claim 8, further comprising:
determining one or more additional types of the physical environments;
presenting the determined one or more additional types of the physical environments; and
receiving, from a user, a selection of one of the determined types of the physical environments.

### Claim 10 (depends on 1)

The electronic device of claim 1, further comprising:
receiving input representing a selection of a virtual object of the one or more presented virtual objects; and
performing one or more tasks in accordance with the selected virtual object.

### Claim 11

A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for:
obtaining, using the one or more cameras, image data corresponding to a physical environment;
based on the image data, identifying at least one portion of an entity in the physical environment;
determining, based on the identified at least one portion of the entity, whether the entity is an entity of a first type; and
in accordance with a determination that the entity is an entity of the first type, determining a type of the physical environment based on the entity; and
presenting one or more virtual objects and a representation of the entity corresponding to the determined type of the physical environment.

### Claim 12 (depends on 11)

The computer-readable storage medium of claim 11, wherein identifying the at least one portion of the entity in the physical environment comprises:
identifying a first portion of the entity without identifying the entire entity; and
foregoing identifying a second portion of the entity.

### Claim 13 (depends on 11)

The computer-readable storage medium of claim 11, wherein determining, based on the identified at least one portion of the entity, whether the entity is an entity of the first type comprises:
determining, based on the identified at least one portion of the entity in the physical environment, one or more properties of the entity; and
determining, based on the one or more properties of the entity, whether the entity is an entity of the first type.

### Claim 14 (depends on 13)

The computer-readable storage medium of claim 13, wherein determining, based on the identified at least one portion of the entity, the one or more properties of the entity comprises:
classifying the entity based on the identified at least one portion of the entity;
obtaining, based on the classification of the entity, one or more properties of the entity from a plurality of properties.

### Claim 15 (depends on 13)

The computer-readable storage medium of claim 13, wherein the one of more properties of the entity comprise at least one of:
mobility of the entity;
an indication of whether the entity is a building structure; and
consistency of the appearance of the entity.

### Claim 16 (depends on 13)

The computer-readable storage medium of claim 13, wherein the determining, based on the one or more properties of the entity, whether the entity is an entity of the first type comprises:
determining whether a combination of the one or more properties of the entity exceeds a confidence threshold.

### Claim 17 (depends on 11)

The computer-readable storage medium of claim 11, wherein determining the type of the physical environment based on the entity comprises:
determining, whether the entity corresponds to at least one of a plurality of types of the physical environments; and
in accordance with a determination that the entity corresponds to at least one of the plurality of types of the physical environments, selecting one of the plurality of types of the physical environments.

### Claim 18 (depends on 17)

The computer-readable storage medium of claim 17, further comprising:
determining one or more additional types of the physical environments;
presenting the determined one or more additional types of the physical environments; and
receiving, from a user, a selection of one of the determined types of the physical environments.

### Claim 19 (depends on 11)

The computer-readable storage medium of claim 11, further comprising:
receiving input representing a selection of a virtual object of the one or more presented virtual objects; and
performing one or more tasks in accordance with the selected virtual object.

### Claim 20

A method comprising:
at an electronic device with one or more processors, memory, and one or more cameras:
obtaining, using the one or more cameras, image data corresponding to a physical environment;
based on the image data, identifying at least one portion of an entity in the physical environment;
determining, based on the identified at least one portion of the entity, whether the entity is an entity of a first type;
in accordance with a determination that the entity is an entity of the first type, determining a type of the physical environment based on the entity; and
presenting one or more virtual objects and a representation of the entity corresponding to the determined type of the physical environment.

### Claim 21 (depends on 20)

The method of claim 20, wherein identifying the at least one portion of the entity in the physical environment comprises:
identifying a first portion of the entity without identifying the entire entity; and
foregoing identifying a second portion of the entity.

### Claim 22 (depends on 20)

The method of claim 20, wherein determining, based on the identified at least one portion of the entity, whether the entity is an entity of the first type comprises:
determining, based on the identified at least one portion of the entity in the physical environment, one or more properties of the entity; and
determining, based on the one or more properties of the entity, whether the entity is an entity of the first type.

### Claim 23 (depends on 22)

The method of claim 22, wherein determining, based on the identified at least one portion of the entity, the one or more properties of the entity comprises:
classifying the entity based on the identified at least one portion of the entity;
obtaining, based on the classification of the entity, one or more properties of the entity from a plurality of properties.

### Claim 24 (depends on 22)

The method of claim 22, wherein the one of more properties of the entity comprise at least one of:
mobility of the entity;
an indication of whether the entity is a building structure; and
consistency of the appearance of the entity.

### Claim 25 (depends on 22)

The method of claim 22, wherein the determining, based on the one or more properties of the entity, whether the entity is an entity of the first type comprises:
determining whether a combination of the one or more properties of the entity exceeds a confidence threshold.

### Claim 26 (depends on 20)

The method of claim 20, wherein determining the type of the physical environment based on the entity comprises:
determining, whether the entity corresponds to at least one of a plurality of types of the physical environments; and
in accordance with a determination that the entity corresponds to at least one of the plurality of types of the physical environments, selecting one of the plurality of types of the physical environments.

### Claim 27 (depends on 20)

The method of claim 20, further comprising:
receiving input representing a selection of a virtual object of the one or more presented virtual-reality objects; and
performing one or more tasks in accordance with the selected virtual object.