# US 10,482,661 B2 · Displaying of augmented reality objects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,482,661 B2
- Document ID: US10482661B2
- Status: Granted
- Kind: B2
- Application number: 15/057,592
- Filed: 2016-03-01
- Published: 2017-09-07
- Granted: 2019-11-19
- Earliest claimed: 2016-03-01
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Taku Aratsu; Toshiyuki Komoda; Yohei Umehara; Satoshi Yokoyama
- Web page: https://patentgrep.com/patent/US10482661B2

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An embodiment of the present invention provides a computer-implemented method for displaying one or more augmented reality (AR) objects on a transparent display device, comprising: associating data of one or more transparent areas corresponding to one or more real objects with a first layer for seeing one or more real objects on the transparent display device, wherein the data has information on a location and shape of the one or more real objects on the transparent display device, associating one or more AR objects with a second layer for displaying one or more AR objects on the transparent display device; and overlaying the first layer with the second layer to display the one or more AR objects on the transparent display device, wherein the one or more real objects are seen through the one or more transparent areas on the transparent display device by transparent area the transparent display device.

## Claims

### Claim 1

A computer-implemented method for displaying one or more augmented reality (AR) objects on a transparent display device, the method comprising:
associating data of one or more transparent areas corresponding to one or more real objects with a first layer;
associating one or more AR objects with a second layer;
determining a concentration level of one or more users as it relates to at least one AR object of the one or more AR objects;
based on determining that the concentration level does not exceed a threshold, increasing a playback speed associated with the at least one AR object;
based on determining that determining that the concentration level exceeds a threshold, decreasing a playback speed associated with the at least one AR object; and
overlaying the first layer with the second layer to display the one or more AR objects on a transparent display device based on a priority, wherein the one or more real objects are seen through the one or more transparent areas on the transparent display device by a user utilizing the transparent display device.

### Claim 2 (depends on 1)

The method according to claim 1, the method further comprising:
preparing a drawing area in a priority buffer, wherein the drawing area corresponds to a display area on the transparent display device, and wherein each priority associated with each object of the one or more real objects and the one or more AR objects is given to the drawing area further based on information on a location and a shape of the object;
selecting one object having a highest priority among the one or more real objects and the one or more AR objects;
determining whether the location and shape of the selected object overlaps with a part of the drawing area to which a priority was already given;
based on determining that the location and shape of the selected object overlaps with a part of the drawing area to which a priority was already given; determining whether the selected object is an AR object;
based on determining that the selected object is an AR object, displaying the selected object on the transparent display device; and
based on determining that the selected object is not an AR object, the selected object is seen through the transparent area corresponding to the selected object on the transparent display device by the user.

### Claim 3 (depends on 2)

The method according to claim 2, the method further comprising:
based on determining that the location and shape of the selected object does not overlap with a part of the drawing area to which a priority was already given, giving to the drawing area a priority associated with the selected object so as to correspond to the location and shape corresponding to the selected object;
determining whether the selected object is an AR object;
based on determining that the selected object is an AR object, displaying the selected object on the transparent display device; and
based on determining that the selected object is not an AR object, the selected object is seen through the transparent area corresponding to the selected object on the transparent display device utilized by the user.

### Claim 4 (depends on 2)

The method according to claim 2, the method further comprising:
based on determining that the location and shape of the selected object overlaps with a part of the drawing area to which a priority was already given, determining whether the priority associated with the selected object is lower than the priority which was already given to the priority buffer;
based on determining that the priority associated with the selected object is lower than the priority which was already given to the priority buffer, not displaying the selected object on the transparent display device.

### Claim 5 (depends on 2)

The method according to claim 2, the method further comprising:
based on determining that the location and shape of the selected object overlaps with a part of the drawing area to which a priority was already given, determining whether the priority associated with the selected object is the same as the priority which was already given to the drawing area;
based on determining that the priority associated with the selected object is the same as the priority which was already given to the drawings area, determining whether the selected object is an AR object;
based on determining that the selected object is an AR object, determining whether an object associated with the priority which was already given to the drawing area is a real object; and
based on determining that an object associated with the priority which was already given to the drawing area is a real object, displaying the AR object on the transparent area corresponding to the real object.

### Claim 6 (depends on 2)

The method according to claim 2, the method further comprising:
based on determining that the location and shape of the selected object overlaps with a part of the drawing area to which a priority was already given, determining whether the priority associated with the selected object is the same as the priority which was already given to the drawing area;
based on determining that the priority associated with the selected object is the same as the priority which was already given to the drawing area, determining whether a content type of the selected object is the same as a content type of an object associated with the priority which was already given to the drawing area; and
based on determining that a content type of the selected object is the same as a content type of an object associated with the priority which was already given to the drawing area, displaying the selected object on the object associated with the priority which was already given to the drawing area.

### Claim 7 (depends on 1)

The method according to claim 1, the method further comprising:
selecting an AR content of at least one AR object of the one or more AR objects, based on at least one characteristic of the user before the association of the data with the first layer.

### Claim 8 (depends on 1)

The method according to claim 1, wherein determining the priority for each object of the one or more real objects and the one or more AR objects is further based on:
determining a degree of concentration of the user utilizing the transparent display device;
determining a degree of concentration of each of a plurality of users; and
based on the degree of concentration of the user and each of the plurality of users, deciding a concentration level of the user and the users.

### Claim 9 (depends on 8)

The method according to claim 8, wherein determining the priority for each object of the one or more real objects and the one or more AR objects is further based on at least one of:
detecting a direction of an eye gaze of the user utilizing the transparent display device.

### Claim 10 (depends on 9)

The method according to claim 9, the method further comprising:
before the association of the data with the first layer and based on the priority for each object of the one or more real objects and the one or more AR objects, performing at least one of:
determining whether to display the AR object;
changing a display size of the AR object;
changing a displaying location of the AR object;
changing an AR content of the AR object;
changing the number of the AR objects to be displayed;
changing a speed of playing the AR object; and
changing a frequency of switching the AR objects.

### Claim 11 (depends on 1)

The method according to claim 1, the method further comprising:
determining whether a display location of at least one of: a first AR object and first transparent area is approaching a display location on the transparent display device of at least one of: a second AR object and second transparent area; and
based on determining that a display location of at least one of: a first AR object and first transparent area is approaching a display location on the transparent display device of at least one of: a second AR object and second transparent area, changing an AR content of at least one of the first and second AR objects.

### Claim 12 (depends on 11)

The method according to claim 11, wherein the determination is performed by identifying an overlapping of a transparent region adjacent to at least one of: the first AR object and first transparent area with a transparent region adjacent to at least one of: the second AR object or second transparent area.

### Claim 13 (depends on 1)

The method according to claim 1, the method further comprising:
determining whether a display location of an AR object is approaching a displaying location of a real object on the transparent display device; and
based on determining that a display location of an AR object is approaching a displaying location of a real objects on the transparent display device, replacing an AR content of the AR object with an AR content associated with the real object.

### Claim 14 (depends on 13)

The method according to claim 13, wherein the determination is performed by identifying an overlapping of a transparent region adjacent to the AR object with a transparent region adjacent to the real object.

### Claim 15 (depends on 13)

The method according to claim 13, the method further comprising:
displaying the AR content after the replacement on a transparent area corresponding to the real object.

### Claim 16 (depends on 13)

The method according to claim 13, the method further comprising:
displaying the AR content after the replacement so that a displaying area of the AR content after the change does not overlap with a transparent area corresponding to the real object.

### Claim 17 (depends on 8)

The method of claim 8, wherein determining a degree of concentration of the user utilizing the transparent display device further comprises:
recording a video of the user; and
analyzing facial expressions of the user.