# US 8,570,319 B2 · Perceptually-based compensation of unintended light pollution of images for projection display systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,570,319 B2
- Document ID: US8570319B2
- Status: Granted
- Kind: B2
- Application number: 12/828,137
- Filed: 2010-06-30
- Published: 2011-07-21
- Granted: 2013-10-29
- Earliest claimed: 2010-01-19
- Assignees: Disney Enterprises, Inc.
- Applicants: Jeroen van Baar; Steven Poulakos; Rasmus Tamstorf; Markus Gross
- Inventors: Jeroen van Baar; Steven Poulakos; Rasmus Tamstorf; Markus Gross
- Web page: https://patentgrep.com/patent/US8570319B2

## CPC

- H04N 13/122: Improving the three-dimensional [3D] impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues (H04N13/128 takes precedence)
- G09G 3/003: {to produce spatial visual effects}
- G09G 2320/0285: using tables for spatial correction of display data
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

Where images are displayed such that unintended light is also included and that light cannot be fully subtracted from the displayed image, an image processor compensates by compensating for the remaining light using perceptual models. In some cases, the perceptual model includes a plurality of user sensitivities and computation is performed, at least in part, based on expected user sensitivity to light pollution in portions of the images. The perceptual model might include a contrast sensitivity function, threshold-vs-intensity, saliency prediction and visual masking. The model might include artist indications of view attention. A light pollution likelihood map might be used to reduce the amount of computation needed, using a likelihood measure to determine an amount of computation to perform.

## Claims

### Claim 1

A method for generating images to be viewable on a display using a computer that are generated based on scene geometry obtained from computer-readable storage and animation data representing changes over time of elements of the scene geometry or other image sources, wherein the images so generated are expected to be used where cross-over is present, cross-over being the illumination of one portion of an image by light intended for another portion of the image or another image, the method comprising:
determining a desired one or more images to be displayed;
determining, from the desired one or more images to be displayed and/or characteristics of a display system that is to display the images, what image cross-over might be present;
determining one or more compensated images representing an inverse of the images and cross-over such that the compensated images are calculated to result in approximately the desired images once displayed;
where the compensated images include negative light values:
a) setting the negative light values to a nonnegative value; and
b) using a perceptual model comprising a perceptual weighting function, adjusting the compensated images to reduce perception of the results of the setting step; and
storing the resulting images as the images to be displayed.

### Claim 2 (depends on 1)

The method of claim 1, wherein the cross-over is cross-over from a stereoscopic pair of images.

### Claim 3 (depends on 1)

The method of claim 1, wherein the cross-over is cross-over from concave screen screen-to-screen reflections.

### Claim 4 (depends on 1)

The method of claim 1, wherein the cross-over is cross-over from an active 3D stereo system using shutter mechanisms in order to present separate images to a left eye and a right eye, whereby shutter times for each eye overlap.

### Claim 5 (depends on 1)

The method of claim 1, wherein the perceptual model includes a plurality of user sensitivities and wherein a computational effort associated with adjusting portions of the images is determined, at least in part, based on expected user sensitivity to pollution in those portions of the images.

### Claim 6 (depends on 1)

The method of claim 1, wherein the perceptual model comprises a contrast sensitivity function, threshold-vs-intensity, saliency prediction and visual masking.

### Claim 7 (depends on 1)

The method of claim 1, further comprising receiving input data related to artist indication of view attention and a computational effort associated with adjusting portions of the images is determined, at least in part, based on expected user sensitivity to light pollution in those portions of the images.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
determining a light pollution likelihood map, which indicates for a plurality of portions of an image or images a likelihood measure for the light pollution portion; and
using the likelihood measure for a portion of the plurality of portions to determine an amount of computation to perform.

### Claim 9 (depends on 8)

The method of claim 8, further comprising using likelihood map coefficients as weights in a weighted optimization.

### Claim 10 (depends on 1)

The method of claim 1, wherein the adjustments to the compensated images are limited from frame to frame such that transitions over time are smoothed.

### Claim 11

A method for generating a pair of modified images to be viewable on a 3D display from a pair of input images, the generating done using a computer based on scene geometry obtained from computer-readable storage and animation data representing changes over time of elements of the scene geometry, such that perception of cross-over illumination from one image of the pair to the display of the other image of the pair is reduced in the pair of modified images relative to display of the pair of input images, the method comprising:
reading in the pair of input images;
determining, from the pair of input images to be displayed and/or characteristics of a display system that is to display images, what image cross-over illumination might be present from one of the pair of input images to a viewer eyepiece for the other of the pair of input images;
subtractively compensating, for at least some of the cross-over illumination, resulting in an indication of residual cross-over illumination; and
perceptually compensating, for at least some of the residual cross-over illumination, to form the pair of modified images, wherein perceptually compensating comprises determining with a perceptual weighting function modification values for the pair of input images that would be expected to reduce perception of the residual cross-over illumination.

### Claim 12 (depends on 11)

The method of claim 11, wherein perceptually compensating comprises:
determining an initial visibility of the residual cross-over illumination according to perceptual metrics determined from parameters representing, threshold-vs-intensity, visual masking between frequency bands, saliency, contrast thresholds, disparity and motion information;
generating a weight map according to the initial visibility of the residual cross-over illumination; and
generating the pair of modified images by reducing a difference according to a perceptual norm between the pair of input images and the pair of input images with the residual cross-over illumination.

### Claim 13 (depends on 11)

The method of claim 11, wherein the 3D display is a projection system.

### Claim 14 (depends on 11)

The method of claim 11, wherein the 3D display is a 3D LCD display.

### Claim 15 (depends on 11)

The method of claim 11, wherein the method also compensates for the perception of cross-over illumination due to scattering of light between displays.

### Claim 16

An image processor that receives a plurality of images to be viewable with a display system expected to be used where cross-over is present, cross-over being the illumination of one portion of an image by light intended for another portion of the image or another image, the image processor comprising:
inputs for parameters relating to the desired one or more images to be displayed and/or characteristics of the display system;
logic for determining, from the input parameters, what image cross-over might be present;
logic for computing one or more compensated images representing an inverse of the images and cross-over such that the compensated images are calculated to result in approximately the desired images once displayed, including logic for compensating for negative light values by treating the negative light values as nonnegative values and using a stored perceptual model comprising a perceptual weighting function to adjust the compensated images to reduce perception of nonnegative light artifacts; and
storage or transmission hardware for storing or transmitting the resulting images as the images to be displayed.

### Claim 17 (depends on 16)

The image processor of claim 16, wherein the cross-over is cross-over from a stereoscopic pair of images.

### Claim 18 (depends on 16)

The image processor of claim 16, wherein the cross-over is cross-over from concave screen screen-to-screen reflections.

### Claim 19 (depends on 16)

The image processor of claim 16, wherein the cross-over is cross-over from an active 3D stereo system using shutter mechanisms in order to present separate images to a left eye and a right eye, whereby shutter times for each eye overlap.

### Claim 20 (depends on 16)

The image processor of claim 16, wherein the perceptual model includes a plurality of user sensitivities and wherein a computational effort associated with adjusting portions of the images is based, at least in part, on expected user sensitivity to pollution in those portions of the images.

### Claim 21 (depends on 16)

The image processor of claim 16, wherein the perceptual model comprises a contrast sensitivity function, threshold-vs-intensity, saliency prediction and visual masking.

### Claim 22 (depends on 16)

The image processor of claim 16, further comprising logic for processing input data related to artist indication of view attention and a computational effort associated with adjusting portions of the images based, at least in part, on expected user sensitivity to light pollution in those portions of the images.

### Claim 23 (depends on 16)

The image processor of claim 16, further comprising:
storage for a light pollution likelihood map that indicates, for a plurality of portions of an image or images, a likelihood measure for the light pollution portion; and
logic for using the likelihood measure for a portion of the plurality of portions to determine an amount of computation to perform.

### Claim 24 (depends on 23)

The image processor of claim 23, wherein likelihood map coefficients are weights in a weighted optimization.

### Claim 25 (depends on 16)

The image processor of claim 16, wherein the adjustments to the compensated images are limited from frame to frame such that transitions over time are smoothed.