US 11,380,070 B2 · Real-time augmentation of a virtual object onto a real-world object

Abstract

This disclosure describes techniques that enable an augmentation controller to generate a real-time augmented multimedia stream. More specifically, the augmentation controller may receive a real-time multimedia stream from a client device, along with a representation of a related, virtual object. The real-time multimedia stream may be analyzed to identify a real-world object (i.e. a vehicle) along with a set of associated feature descriptors (i.e. vehicle wheels). Similarly, the augmentation controller may identify feature descriptors of the virtual object (i.e. wheelrim) and in doing so, overlay a representation of the virtual object on the real-world object based on an alignment of their respective feature descriptors. In this way, the augmentation controller may transmit to the client device, an augmented multimedia stream of the real-time multimedia stream with the virtual object overlaid onto the real-world object.

Claims20

Dependent claims

A system, comprising:

one or more processors;
memory coupled to the one or more processors, the memory including one or more modules that are executable by the one or more processors to:
receive, from a client device, a real-time multimedia stream that includes an image of a real-world object;
analyze the image of the real-world object to identify a first set of feature descriptors;
identify a virtual object to overlay onto a feature descriptor of the first set of feature descriptors;
analyze the virtual object to identify a second set of feature descriptors;
generate an augmented multimedia stream that overlays a representation of the virtual object onto the image of the real-world object, based at least in part on an overlay of the first set of feature descriptors and the second with the second set of feature descriptor;
and
transmit, to the client device, the augmented multimedia stream.

6 more elements

A computer-implemented method, comprising:

under control of one or more processors:
receiving, from a client device, an indication of a virtual object that is to be augmented onto a real-world object on a real-time multimedia stream;
identifying a first set of feature descriptors associated with the virtual object;
parsing the real-time multimedia stream to identify the real-world object;
analyzing the real-world object to identify a second set of feature descriptors;
generating a scaled representation of the virtual object, based at least in part on the second set of feature descriptors;
and
generating an augmented multimedia stream that overlays and anchors the scaled representation of the virtual object in real-time over the real-world object within the real-time multimedia stream, based at least in part on an alignment of the first set of feature descriptors and the second set of feature descriptors.

5 more elements

One or more non-transitory computer-readable media storing computer-executable instructions that, when executed on one or more processors, cause the one or more processors to perform acts comprising:

identifying a virtual object that is to be overlaid and anchored onto a real-world object in a real-time multimedia stream;
determining a first set of feature descriptors associated with the virtual object that act as anchor fixed on the virtual object that anchor to the real-world object;
determining a second set of feature descriptors associated with the real-world object that act as fixed points on the real-world object to interface with the first set of feature descriptors;
generating an augmented multimedia stream by overlaying the first set of feature descriptors of a representation of the virtual object to the second set of feature descriptors of the real-world object;
and
transmitting the augmented multimedia stream, in real-time, to a client device.

3 more elements

Dependencies are inferred from the claim text.

Real-time augmentation of a virtual object onto a real-world object

Life of this patent

Real-time augmentation of a virtual object onto a real-world object

Life of this patent

Abstract

Claims20