US 9,754,417 B2Granted

Methods and systems for displaying virtual objects

AssigneesCanon Information and Imaging Solutions, Inc., Canon U.S.A., Inc.

InventorsDon Hideyasu Matsubayashi, Masaharu Enomoto

Life of this patent

Estimating the term…

Loading patent…

Abstract

Methods and systems for displaying a virtual object capture, via an image capturing device, images of a physical scene that includes a first marker and a second marker, wherein the first marker and the second marker are physical markers; associate a first virtual object with the first marker and a second virtual object with the second marker; track the position of the first marker and the position of the second marker in the captured images of the physical scene using a tracking device; detect an interaction between the first marker and the second marker based on the tracked position of the first marker and the tracked position of the second marker, wherein the interaction is detected when the first marker and the second marker are within a predetermined proximity of each other; and associate, in response to detecting that the interaction, a third virtual object with the first marker.

Claims14

Dependent claims

A system for displaying a virtual object, the system comprising:

one or more non-transitory computer-readable media;
and
one or more processors that are coupled to the one or more computer-readable media and that are configured to cause the system to
obtain images of a physical scene that includes a first marker and a second marker, wherein the first marker and the second marker are physical markers;
generate images of an enhanced scene that show a first virtual object with the first marker and that show a second virtual object with the second marker;
track a position of the first marker and a position of the second marker in the images of the physical scene;
detect an interaction between the first marker and the second marker based on the tracked position of the first marker and the tracked position of the second marker, wherein the interaction is detected when the position of the first marker is within a predetermined distance of the position of the second marker;
in response to detecting the interaction, select another virtual object based on a topmost marker of the first marker and the second marker when one of the first marker and the second marker at least partially overlaps the other, wherein the other virtual object that is selected is a third virtual object if the first marker is the topmost marker, wherein the other virtual object that is selected is a fourth virtual object if the second marker is the topmost marker, and wherein the first virtual object, the second virtual object, the third virtual object, and the fourth virtual object are different from each other;
and
generate revised images of the enhanced scene that replace both the first virtual object and the second virtual object with the other virtual object,
wherein the third virtual object is a first combination of the first virtual object and the second virtual object, and
wherein the fourth virtual object is a second combination of the first virtual object and the second virtual object.

9 more elements

A system for displaying a virtual object, the system comprising:

one or more non-transitory computer-readable media;
and
one or more processors that are coupled to the one or more computer-readable media and that are configured to cause the system to
obtain images of a physical scene that includes a first marker and a second marker, wherein the first marker and the second marker are physical markers;
generate images of an enhanced scene that show a first virtual object with the first marker and that show a second virtual object with the second marker;
track a position of the first marker and a position of the second marker in the images of the physical scene;
detect an interaction between the first marker and the second marker based on the tracked position of the first marker and the tracked position of the second marker, wherein the interaction is detected when the position of the first marker is within a predetermined distance of the position of the second marker;
in response to detecting the interaction, select another virtual object based on a most-recently-moved marker, as indicated by the tracked position of the first marker and the tracked position of the second marker, wherein the other virtual object that is selected is a third virtual object if the first marker is the most-recently-moved marker, wherein the other virtual object that is selected is a fourth virtual object if the second marker is the most-recently-moved marker, and wherein the first virtual object, the second virtual object, the third virtual object, and the fourth virtual object are different from each other;
and
generate revised images of the enhanced scene that replace both the first virtual object and the second virtual object with the other virtual object,
wherein the third virtual object is a first combination of the first virtual object and the second virtual object, and
wherein the fourth virtual object is a second combination of the first virtual object and the second virtual object.

9 more elements

One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:

obtaining images of a physical scene that includes a first marker and a second marker, wherein the first marker and the second marker are physical markers;
generating images of an enhanced scene that show a first virtual object with the first marker and that show a second virtual object with the second marker;
tracking a position of the first marker and a position of the second marker in the images of the physical scene;
detecting an interaction between the first marker and the second marker based on the tracked position of the first marker and the tracked position of the second marker, wherein the interaction is detected when the position of the first marker is within a predetermined distance of the position of the second marker;
in response to detecting the interaction, selecting another virtual object based on a topmost marker when one of the first marker and the second marker at least partially overlaps the other, wherein a third virtual object is selected as the other virtual object if the first marker is the topmost marker, wherein a fourth virtual object is selected as the other virtual object if the second marker is the topmost marker, and wherein the first virtual object, the second virtual object, the third virtual object, and the fourth virtual object are different from each other;
and
generating revised images of the enhanced scene that replace both the first virtual object and the second virtual object with the other virtual object,
wherein the third virtual object is a first combination of the first virtual object and the second virtual object, and
wherein the fourth virtual object is a second combination of the first virtual object and the second virtual object.

6 more elements

A method comprising:

obtaining images of a physical scene that includes a first marker and a second marker, wherein the first marker and the second marker are physical markers, and wherein the images were captured by an image capturing device;
generating images of an enhanced scene that show a first virtual object with the first marker and that show a second virtual object with the second marker;
tracking a position of the first marker and a position of the second marker in the images of the physical scene;
detecting an interaction between the first marker and the second marker based on the tracked position of the first marker and the tracked position of the second marker, wherein the interaction is detected when the position of the first marker is within a predetermined distance of the position of the second marker;
in response to detecting the interaction, selecting another virtual object based on a most-recently-moved marker, wherein a third virtual object is selected as the other virtual object if the first marker is the most-recently-moved marker, wherein a fourth virtual object is selected as the other virtual object if the second marker is the most-recently-moved marker, and wherein the first virtual object, the second virtual object, the third virtual object, and the fourth virtual object are different from each other;
and
generating revised images of the enhanced scene that replace both the first virtual object and the second virtual object with the other virtual object,
wherein the third virtual object is a first combination of the first virtual object and the second virtual object, and
wherein the fourth virtual object is a second combination of the first virtual object and the second virtual object.

6 more elements

Dependencies are inferred from the claim text.