# US 2016/0366444 A1 · Metadata describing nominal lighting conditions of a reference viewing environment for video playback

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0366444 A1
- Document ID: US20160366444A1
- Status: Published application
- Kind: A1
- Application number: 15/146,800
- Filed: 2016-05-04
- Published: 2016-12-15
- Earliest claimed: 2015-06-09
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Gary J. Sullivan
- Web page: https://patentgrep.com/patent/US20160366444A1

## CPC

- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- G09G 5/02: characterised by the way in which colour is displayed {(details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607)}
- G09G 5/10: Intensity circuits
- H04N 19/134: characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N 19/162: User input
- H04N 19/186: the unit being a colour or a chrominance component
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/85: using pre-processing or post-processing specially adapted for video compression
- H04N 21/2353: {specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata}
- H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
- H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)}
- G09G 2320/0673: for control of gamma adjustment, e.g. selecting another gamma curve
- G09G 2360/144: the light being ambient light
- G09G 2370/04: Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

Innovations in the areas of generating, parsing, and using metadata that describes nominal lighting conditions of a reference viewing environment for video playback are presented herein. In various examples described herein, metadata includes parameters that describe the nominal lighting conditions (e.g., level of ambient light, color characteristics of ambient light) of a reference viewing environment. By conveying a representation of the nominal lighting conditions of the reference viewing environment (e.g., one assumed when mastering image content), a transmitter system can enable a receiver system to adapt its local display of the image content. Upon receiving image content and the metadata, the receiver system can identify characteristics of the actual viewing environment, use the metadata to determine whether the actual viewing environment matches the reference viewing environment, and, if not, adjust sample values of the image content, adjust a display device, or adjust lighting conditions of the actual viewing environment.

## Claims

### Claim 1

A computing system comprising:
a metadata generator configured to generate metadata that describes one or more nominal lighting conditions of a reference viewing environment for playback of video, wherein the metadata includes one or more of:
one or more parameters that specify a nominal level of ambient light in the reference viewing environment, the one or more parameters that specify the nominal level of ambient light including an indicator of light per unit of area; and
one or more parameters that specify a nominal color characteristic of the ambient light in the reference viewing environment, the one or more parameters that specify the nominal color characteristic of ambient light including coordinates in a multi-dimensional mapping of values in a color space; and
a buffer configured to store the metadata for output with encoded data for the video.

### Claim 2 (depends on 1)

The computing system of claim 1, further comprising:
a buffer configured to receive input video for the video;
a pre-processor configured to selectively adjust at least some characteristics of the input video based on one or more of (a) characteristics of a reference display device, and (b) the one or more nominal lighting conditions of the reference viewing environment; and
a video encoder configured to encode the selectively adjusted input video to produce the encoded data for the video.

### Claim 3 (depends on 1)

The computing system of claim 1, wherein the indicator of light per unit of area is a value in units of lux, and wherein the coordinates in the multi-dimensional mapping of values in the color space are normalized x and y chromaticity coordinates of the ambient light in the reference viewing environment.

### Claim 4 (depends on 1)

The computing system of claim 1, wherein the one or more parameters that specify the nominal color characteristic of the ambient light further include a value in a color temperature scale.

### Claim 5 (depends on 1)

The computing system of claim 1, wherein the metadata further includes one or more parameters that specify one or more of:
a nominal level of light reflected by a reference display device in the reference viewing environment; and
one or more color characteristics of the light reflected by the reference display device in the reference viewing environment.

### Claim 6 (depends on 1)

The computing system of claim 1, wherein the metadata further includes one or more parameters that specify, for the one or more nominal lighting conditions of the reference viewing environment, adjustments, if any, to reach a final lighting state from an initial lighting state of a hypothetical viewing environment.

### Claim 7 (depends on 1)

The computing system of claim 1, wherein the metadata further includes multiple sets of parameters, and wherein each of the multiple sets of parameters:
is associated with a different hypothetical viewing environment having an initial lighting state; and
includes one or more parameters that specify, for the one or more nominal lighting conditions of the reference viewing environment, adjustments, if any, to reach a final lighting state from the initial lighting state of the hypothetical viewing environment for that set of parameters.

### Claim 8 (depends on 1)

The computing system of claim 1, wherein the reference viewing environment includes multiple regions, and wherein the metadata includes, for each of the multiple regions, one or more parameters that specify those of the one or more nominal lighting conditions that apply for the region, the multiple regions including a first region and a second region, the first region being behind an expected position of a viewer in the reference viewing environment, the second region being in front of the expected position of the viewer in the reference viewing environment.

### Claim 9 (depends on 1)

The computing system of claim 1, wherein the metadata is organized as part of:
a supplemental enhancement information (“SEI”) message, wherein the SEI message is part of a video elementary bitstream for the video;
a video usability information (“VUI”) message, wherein the VUI message is part of the video elementary bitstream for the video;
a field of a container format; or
a message according to a transport protocol.

### Claim 10

In a computing system, a method comprising:
receiving metadata that describes one or more nominal lighting conditions of a reference viewing environment for playback of video, wherein the metadata includes one or more of:
one or more parameters that specify a nominal level of ambient light in the reference viewing environment, the one or more parameters that specify the nominal level of ambient light including an indicator of light per unit of area; and
one or more parameters that specify a nominal color characteristic of the ambient light in the reference viewing environment, the one or more parameters that specify the nominal color characteristic of ambient light including coordinates in a multi-dimensional mapping of values in a color space; and
parsing the metadata.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
determining one or more actual lighting conditions of an actual viewing environment; and
to compensate for differences between the one or more actual lighting conditions and the one or more nominal lighting conditions, adjusting one or more of:
a characteristic of at least some sample values of the video;
a characteristic of an actual display device in the actual viewing environment; and
at least one of the one or more actual lighting conditions of the actual viewing environment.

### Claim 12 (depends on 11)

The method of claim 11, wherein the one or more actual lighting conditions include an actual level of ambient light in the actual viewing environment, and wherein the adjusting comprises:
if the nominal level of ambient light is brighter than the actual level of ambient light, decreasing brightness of the at least some sample values of the video; and
if the nominal level of ambient light is darker than the actual level of ambient light, increasing brightness of the at least some sample values of the video.

### Claim 13 (depends on 11)

The method of claim 11, wherein the one or more actual lighting conditions include an actual color characteristic of ambient light in the actual viewing environment, and wherein the adjusting comprises scaling or attenuating the at least some sample values of the video to compensate for differences between the actual color characteristic and the nominal color characteristic.

### Claim 14 (depends on 11)

The method of claim 11, wherein the metadata includes multiple sets of parameters, and wherein each of the multiple sets of parameters:
is associated with a different hypothetical viewing environment having an initial lighting state; and
includes one or more parameters that specify, for the one or more nominal lighting conditions of the reference viewing environment, adjustments, if any, to reach a final lighting state from the initial lighting state of the hypothetical viewing environment for that set of parameters.

### Claim 15 (depends on 14)

The method of claim 14, further comprising:
based at least in part on the one or more actual lighting conditions of the actual viewing environment, selecting one of the multiple sets of parameters whose different hypothetical viewing environment is closest to the actual viewing environment; and
for the adjusting, performing adjustments, if any, for the selected set of parameters to reach the final lighting state.

### Claim 16 (depends on 11)

The method of claim 11, wherein the determining the one or more actual lighting conditions includes measuring at least one of the one or more actual lighting conditions.

### Claim 17 (depends on 11)

The method of claim 11, further comprising:
receiving user input that specifies a value for at least one of the one or more actual lighting conditions; and
storing the specified value in a storage area;
wherein the determining the one or more actual lighting conditions includes looking up the specified value in the storage area.

### Claim 18 (depends on 10)

The method of claim 10, further comprising:
receiving encoded data for the video; and
decoding the encoded data for the video.

### Claim 19 (depends on 10)

The method of claim 10, wherein the metadata is organized as part of a supplemental enhancement information (“SEI”) message, wherein the SEI message is part of a video elementary bitstream for the video, and wherein persistence scope for the SEI message is one of:
coded layer-wise video sequence (“CLVS”), such that the SEI message persists, for a current layer in decoding order, from a current picture until end of the CLVS;
coded video sequence, such that the SEI message persists from the current picture until end of the coded video sequence; and
some other portion of the video, such that the SEI message persists until end of the portion of the video.

### Claim 20

A computing system comprising:
a buffer configured to store metadata that describes one or more nominal lighting conditions of a reference viewing environment for playback of video, wherein the metadata includes one or more of:
one or more parameters that specify a nominal level of ambient light in the reference viewing environment, the one or more parameters that specify the nominal level of ambient light including an indicator of light per unit of area; and
one or more parameters that specify a nominal color characteristic of the ambient light in the reference viewing environment, the one or more parameters that specify the nominal color characteristic of ambient light including coordinates in a multi-dimensional mapping of values in a color space;
a metadata parser configured to parse the metadata;
a buffer configured to store encoded data for video; and
a video decoder configured to decode the encoded data for the video.
