Augmented lighting environments

A computer readable storage medium having stored thereon computer-executable instructions, the computer readable storage medium excluding signals per se, the instructions comprising instructions that, when executed by a processor in a computing system, at least cause the computing system to:

• identify in data received from a sensor at least one local light source in a physical lighting environment;
• estimate at least one characteristic of a shadow;
• estimate a first light characteristic of the at least one local light source based on the at least one characteristic of the shadow;
• estimate a second light characteristic of at least one ambient light source based on light in the physical lighting environment not attributable to a local light source;
• synthesize a lighting environment based at least on the first light characteristic and the second characteristic;
• and
• render a virtual object in an augmented reality display, wherein the lighting of the virtual object is based on the synthesized lighting environment.

A device for use in determining a physical lighting environment, the device comprising:

• a processor;
• memory having stored therein instructions executable by the device, the instructions comprising:
• instructions to identify at least one local light source based on the data generated by a first light sensor related to the physical lighting environment,
• instructions to measure a first light characteristic of the at least one local light source,
• instructions to attribute light in the physical lighting environment to at least one ambient light source based on light in the physical lighting environment not attributable to the at least one local light source,
• instructions to estimate a second light characteristic of the at least one ambient light source based on light in the physical lighting environment not attributable to a local light source,
• instructions to form a synthesized lighting environment based at least on the first light characteristic and the second light characteristic, and
• instructions to render a virtual object in an augmented reality display, wherein the lighting of the virtual object is based on the synthesized lighting environment;
• and
• a second light sensor configured to generate data relating to the physical environment, the second light sensor being oriented to face in a different direction than the first light sensor.

A method of updating the rendering of a virtual object in an augmented reality display, wherein light characteristics of light sources in a physical environment were previously determined for use in rendering the virtual object, the method comprising:

• receiving a first image and at least one parameter from a user-facing light sensor;
• correcting the first image based on the at least one parameter of the user-facing light sensor;
• receiving a second image and at least one parameter from a scene-facing light sensor;
• correcting the second image based on the at least one parameter of the scene-facing light sensor;
• constructing an environment lighting map based on the corrected first image and the corrected second image;
• determining light sources in the environment lighting map, wherein the light sources comprise at least one local light source and at least one ambient light source;
• updating the previously-determined light characteristics of the light sources based on characteristics of the light sources determined in the environment lighting map;
• and
• rendering a virtual object in an augmented reality display based on the updated light characteristics, the rendering comprising referencing the environment lighting map for reflection and refraction of the physical environment on the virtual object.

A computer readable storage medium comprising instructions for updating the rendering of a virtual object in an augmented reality display, wherein light characteristics of light sources in a physical environment were previously determined for use in rendering the virtual object, the computer readable storage medium excluding signals per se, the instructions comprising instructions that, when executed by a processor in a computing system, at least cause the computing system to:

• receive a first image and at least one parameter from a user-facing light sensor;
• correct the first image based on the at least one parameter of the user-facing light sensor;
• receive a second image and at least one parameter from a scene-facing light sensor;
• correct the second image based on the at least one parameter of the scene-facing light sensor;
• construct an environment lighting map based on the corrected first image and the corrected second image;
• determine light sources in the environment lighting map, wherein the light sources comprise at least one local light source and at least one ambient light source;
• update the previously-determined light characteristics of the light sources, wherein the updating is based on the characteristics of the light sources determined in the environment lighting map;
• construct a synthesized lighting environment based on the characteristics of the light sources determined in the environment lighting map;
• and
• render a virtual object in an augmented reality display based on the updated light characteristics and based on the synthesized lighting environment.

A device for updating the rendering of a virtual object in an augmented reality display, wherein light characteristics of light sources in a physical environment were previously determined for use in rendering the virtual object, the device comprising:

• a user-facing light sensor configured to take a first image;
• a scene-facing light sensor configured to take a second image;
• and
• a processor configured to:
• correct the first image based on at least one parameter of the user-facing light sensor,
• correct the second image based on at least one parameter of the scene-facing light sensor,
• construct an environment lighting map based on the corrected first image and the corrected second image,
• determine light sources in the environment lighting map, wherein the light sources comprise at least one local light source and at least one ambient light source,
• determine that a new local light source in the environment lighting map was not determined previously,
• attribute light in the environment lighting map to the new local light source by reducing an amount of light attributed to the at least one ambient light source,
• update the previously-determined light characteristics of the light sources, wherein the updating is based on the characteristics of the light sources determined in the environment lighting map, and
• render a virtual object in an augmented reality display based on the updated light characteristics.