US 10,950,038 B2 · Realistically illuminated virtual objects embedded within immersive environments

Abstract

Matching an illumination of an embedded virtual object (VO) with current environment illumination conditions provides an enhanced immersive experience to a user. To match the VO and environment illuminations, illumination basis functions are determined based on preprocessing image data, captured as a first combination of intensities of direct illumination sources illuminates the environment. Each basis function corresponds to one of the direct illumination sources. During the capture of runtime image data, a second combination of intensities illuminates the environment. An illumination-weighting vector is determined based on the runtime image data. The determination of the weighting vector accounts for indirect illumination sources, such as surface reflections. The weighting vector encodes a superposition of the basis functions that corresponds to the second combination of intensities. The method illuminates the VO based on the weighting vector. The resulting illumination of the VO matches the second combination of the intensities and surface reflections.

Claims20

Dependent claims

A computer-readable storage medium having instructions stored thereon for providing an immersive experience to a user, which, when executed by a processor of a computing device cause the computing device to perform actions comprising:

determining a set of basis functions, wherein each of the basis functions of the set of basis functions includes a rendering of an environment that is illuminated by a corresponding illumination source of a set of illumination sources;
determining a weighting vector based on the set of basis functions, a set of albedo values that correspond to surface locations of the environment that are mapped to image data and a set of illumination values of the image data, wherein the weighting vector encodes a superposition of the renderings of the environment included in the set of basis functions that corresponds to a combination of intensities of the set of illumination sources;
and
illuminating a virtual object embedded in the environment based on the weighting vector and the set of basis functions.

1 more element

A method for providing an immersive experience to a user, comprising:

steps for partitioning initial illumination conditions of an environment into a set of indirect illumination sources and a set of direct illumination sources, wherein each direct illumination source of the set of direct illumination sources includes a light emitter positioned within the environment and the set of indirect illumination sources includes optical reflections of light emitted by the set of direct illumination sources from a plurality of reflective surfaces included in the environment;
steps for generating a set of basis images, wherein each basis image of the set of basis images depicts the environment illuminated by light emitted from a corresponding light emitter of the set of direct illumination sources;
steps for determining a weighting vector based on a set of albedo values that correspond to surface locations of the environment that are mapped to image data and a set of illumination values of the image data;
and
steps for illuminating a virtual object embedded in the environment based on the weighting vector and the set of basis images.

2 more elements

A computing system, comprising:

a processor device;
and
a computer-readable storage medium, coupled with the processor device, having instructions stored thereon for providing an immersive experience to a user, which, when executed by the processor device, provide the system with an immersive environment engine configured to perform actions comprising:
determining a set of basis functions, wherein each of the basis functions of the set of basis functions includes a rendering of an environment that is illuminated by a corresponding illumination source of a set of illumination sources;
determining a weighting vector, based on a set of albedo values that correspond to surface locations of the environment that are mapped to image data and a set of illumination values of the image data, wherein the weighting vector encodes a superposition of the renderings of the environment included in the set of basis functions that corresponds to a combination of intensities of the set of illumination sources;
and
illuminating a virtual object embedded in the environment based on the weighting vector and the set of basis functions.

4 more elements

Dependencies are inferred from the claim text.

Realistically illuminated virtual objects embedded within immersive environments

Life of this patent

Realistically illuminated virtual objects embedded within immersive environments

Life of this patent

Abstract

Claims20