# US 2019/0378327 A1 · Surround ambient light sensing, processing and adjustment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378327 A1
- Document ID: US20190378327A1
- Status: Published application
- Kind: A1
- Application number: 16/546,668
- Filed: 2019-08-21
- Published: 2019-12-12
- Earliest claimed: 2015-07-21
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Timo KUNKEL; Patrick GRIFFIS; Gregory John WARD
- Web page: https://patentgrep.com/patent/US20190378327A1

## CPC

- G06T 15/506: {Illumination models}
- G06F 3/147: using display panels
- G06T 15/04: Texture mapping
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G09G 3/00: Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- H04N 5/58: in dependence upon ambient light
- G06T 2215/16: Using real world measurements to influence rendering
- G09G 2320/0261: in the context of movement of objects on the screen or movement of the observer relative to the screen
- G09G 2320/028: by changing the viewing angle properties, e.g. widening the viewing angle, adapting the viewing angle to the view direction
- G09G 2340/12: Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
- G09G 2360/144: the light being ambient light
- H04N 7/15: Conference systems
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Directional image sensor data may be acquired with one or more directional image sensors. A light source and illumination image may be generated based on the directional image sensor data. A number of operations may be caused to be performed for an image based at least in part on light source information in the light source image. The operations may include display management operations, device positional operations, augmented reality superimposition operations, ambient light control operations, etc.

## Claims

### Claim 1

A method, comprising:
acquiring directional image sensor data with one or more directional image sensors;
generating a light source image based on the directional image sensor data;
causing one or more operations to be performed for an image based at least in part on light source information in the light source image;
wherein the method is performed by one or more computing devices;
wherein the light source image is generated in an image rendering environment, and further comprising:
determining, based at least in part on the light source image, ambient light conditions incident on an image rendering surface;
predicting how the ambient light conditions incident on the image rendering surface in combination with properties of the image rendering surface affect image qualities for images rendered on the image rendering surface;
based on the predicting, setting one or more of black levels, contrast settings, white points or primary colors differently in different spatial portions of the image rendering surface in dependence on different ambient light conditions in the different spatial portions of the image rendering surface.

### Claim 2 (depends on 1)

The method of claim 1 wherein the one or more directional image sensors are configured to capture or sense directional light and to generate directional image sensor data with a determined spatial resolution.

### Claim 3 (depends on 1)

The method of claim 1 wherein the one or more directional image sensors are located outside an image rendering surface.

### Claim 4 (depends on 1)

The method of claim 1 wherein the light source image is generated in real time as the directional image sensor data is being collected, and wherein at least one of the one or more operations is performed in real time or in near real time with generating the light source image based on the directional image sensor data.

### Claim 5 (depends on 1)

The method of claim 1 wherein the one or more operations are performed with one or more of: a mobile phone device, a video game device, a television, a tablet computer, a heads up display (HUD) device, a dashboard display device, a display device operating in conjunction with one or more computers, a display device in a vehicle, a movie studio system, a home theater system or a color grading system.

### Claim 6 (depends on 1)

The method of claim 1 wherein the one or more operations comprise one or more of: display management operations to render the image in an image rendering environment, device positional operations to reposition an image rendering surface of an image rendering device, augmented reality operations to superimpose an object or person into the image, or control operations to control ambient light conditions in an image rendering environment.

### Claim 7 (depends on 1)

The method of claim 1 wherein the one or more directional image sensors comprises one or more of: built-in cameras, external cameras, directional image sensors disposed with an accessory to a computing device, non-camera directional image sensors, CMOS-based directional image sensors, CCD-based directional image sensors, nanotube-based directional image sensors, LED-based image sensors, solar cell arrays, quantum dot photoconductors, photodiodes, polymer-based image sensors, or combinations of two or more types of directional image sensors.

### Claim 8 (depends on 1)

The method of claim 1 wherein the light source image represents one of: low spatial resolution light source images, medium spatial resolution light source images, high spatial resolution light source images, spherical images, non-spherical images, or 3D images.

### Claim 9 (depends on 1)

The method of claim 1 wherein the light source image comprises one of: pixel data or non-pixel-data.

### Claim 10 (depends on 1)

The method of claim 1 wherein the light source image comprises one or more of: light directions, light color maps, or light intensity maps.

### Claim 11 (depends on 1)

The method of claim 1 further comprising computing, based at least in part on the light source image, one or more of: dominant light sources, dominant light directions, dominant light illumination sizes, dominant light reflection sizes, light colors, light intensities, or a position of a viewer's head.

### Claim 12 (depends on 1)

The method of claim 1 further comprising computing, in reference to a viewer, one or more directional light maps based at least in part on the light source information.

### Claim 13 (depends on 12)

The method of claim 12 wherein the directional light maps comprise a directional light map for one of: a reference image rendering surface, a surface other than a reference image rendering surface, or a spatial volume.

### Claim 14 (depends on 12)

The method of claim 12 wherein the directional light maps comprise a directional light map that is computed based on one or more of: a location of the viewer, a reference surface, optical properties of a display screen, or the light source information.

### Claim 15 (depends on 1)

The method of claim 1 wherein the light source map is represented in a coordinate system that is one of: a relative coordinate system in relation to a reference image rendering surface, or a coordinate system in relation to a universal coordinate system independent of a reference image rendering surface.

### Claim 16 (depends on 1)

The method of claim 1 further comprising repositioning or reorienting the light source map in the coordinate system based on locational sensor data collected by one or more of: motion sensors, position sensors, orientation sensors, gyroscopes, electronic compasses, accelerometers, or global positioning system (GPS) modules.

### Claim 17 (depends on 1)

The method of claim 1 wherein the directional image sensor data is captured with different exposure values—EV—and interleaved for capturing both bright and dark image portions, thereby identifying directions and positions of light sources.

### Claim 18 (depends on 17)

The method of claim 17 wherein the directional image sensor data is captured every even times with a first EV suitable for capturing bright image portions, and wherein the directional image sensor data is captured every odd times with a second EV suitable for capturing dark image portions.

### Claim 19 (depends on 1)

The method of claim 1 wherein the light source image is generated in an image rendering environment, and further comprising:
determining, based at least in part on the light source image, first ambient light conditions on an image rendering surface and second ambient light conditions on a surface other than the image rendering surface;
determining that the second ambient light conditions on the surface are better than the first ambient light conditions on the image rendering surface;
automatically repositioning a display screen of an image rendering device to the surface with the second ambient light conditions;
rendering one or more images on the display screen of the image rendering device at the surface with the second ambient light conditions.

### Claim 20

A method, comprising:
acquiring directional image sensor data with one or more directional image sensors;
generating a light source image based on the directional image sensor data;
causing one or more operations to be performed for an image based at least in part on light source information in the light source image;
wherein the method is performed by one or more computing devices;
wherein the light source image is generated in an image rendering environment, and further comprising:
determining, based at least in part on the light source image, light conditions in at least one portion of a background or surround of an image rendering surface;
predicting how the light condition in the at least one portion of the background or surround affect image qualities for images rendered on the image rendering surface;
based on the predicting, setting one or more of black levels, contrast settings, white points or primary colors in one or more spatial portions of the image rendering surface in dependence on the light condition in the at least one portion of the background or surround of the image rendering surface.
