# US 2015/0084978 A1 · Local definition of global image transformations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0084978 A1
- Document ID: US20150084978A1
- Status: Published application
- Kind: A1
- Application number: 14/554,783
- Filed: 2014-11-26
- Published: 2015-03-26
- Earliest claimed: 2011-04-08
- Assignees: Dolby Laboratories Licensing Corporation
- Inventors: Anders BALLESTAD; Gerwin DAMBERG
- Web page: https://patentgrep.com/patent/US20150084978A1

## CPC

- G06T 5/92: based on global image properties
- G06T 5/40: using histogram techniques
- G06V 40/162: {using pixel segmentation or colour matching}
- G06T 2207/10024: Color image
- G06T 2207/20012: Locally adaptive
- G06T 2207/30201: Face
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A global image adjustment, such as dynamic range adjustment is established based on image characteristics. The image adjustment is based more heavily on pixel values in image areas identified as being important by one or more saliency mapping algorithms. In one application to dynamic range adjustment, a saliency map is applied to create a weighed histogram and a transformation is determined from the weighted histogram. Artifacts typical of local adjustment schemes may be avoided.

## Claims

### Claim 1

(canceled)

### Claim 2

A method for adjusting image data defining an image, the method comprising:
generating a saliency map for the image wherein generating the saliency map comprises one or more of:
assigning saliency to pixels based on the locations of the pixels in the image; and
receiving information regarding the saliency of various parts of the image by way of a user interface;
establishing a global transformation based on the image data wherein establishing the global transformation comprises:
ignoring parts of the image data corresponding to image areas indicated by the saliency map as having low saliency; or
weighing parts of the image data corresponding to image areas indicated by the saliency map as having low saliency less heavily than other parts of the image data corresponding to image areas indicated by the saliency map as having higher saliency;
and one or more of:
applying the global transformation to the image data to yield altered image data and displaying the altered image data on a display;
applying the global transformation to the image data to yield altered image data and storing the altered image data on a non-transitory medium; and
storing a definition of the global transformation on a non-transitory medium.

### Claim 3 (depends on 2)

A method according to claim 2 wherein generating the saliency map comprises identifying objects in the image and setting saliency of the pixels associated with the objects based on locations of the pixels in the image.

### Claim 4 (depends on 3)

A method according to claim 3 wherein generating the saliency map is based at least in part on metadata that indicates the locations of objects and/or light sources in the image.

### Claim 5 (depends on 3)

A method according to claim 3 wherein generating the saliency map comprises weighting pixels corresponding to those of the objects located closer to a center of an image more heavily than pixels corresponding to those of the objects located near a periphery of the image and weighting pixels corresponding to those of the objects located near a periphery of the image more heavily than pixels not associated with any of the objects.

### Claim 6 (depends on 2)

A method according to claim 2 wherein generating the saliency map comprises weighting pixels located closer to a center of an image more heavily than pixels at a periphery of the image.

### Claim 7 (depends on 2)

A method according to claim 2 wherein generating the saliency map comprises receiving a user selection of a region of interest by way of the user interface.

### Claim 8 (depends on 7)

A method according to claim 7 comprising receiving user input regarding a saliency of the region of interest by way of the user interface.

### Claim 9 (depends on 8)

A method according to claim 8 comprising receiving commands to increase or decrease saliency of the selected region of interest by way of the user interface.

### Claim 10 (depends on 7)

A method according to claim 7 comprising determining saliency for pixels in the region of interest according to one or more saliency algorithms and receiving commands to increase or decrease saliency of the selected region of interest by way of the user interface.

### Claim 11 (depends on 7)

A method according to claim 7 comprising applying an object detection algorithm to the image data and identifying one or more regions of interest for possible selection by a user based on an output of the object detection algorithm.

### Claim 12 (depends on 2)

A method according to claim 2 wherein generating the saliency map comprises performing an object detection algorithm on the image to detect one or more objects in the image and setting areas in the saliency map corresponding to the detected objects to have increased saliency.

### Claim 13 (depends on 2)

A method according to claim 2 wherein generating the saliency map comprises applying a plurality of saliency algorithms to the image data and combining saliency maps produced by the plurality of saliency algorithms.

### Claim 14 (depends on 13)

A method according to claim 13 wherein combining the saliency maps produced by the plurality of saliency algorithms comprises computing a weighted sum or weighted average of the saliency maps produced by the plurality of saliency algorithms.

### Claim 15 (depends on 13)

A method according to claim 13 wherein the plurality of saliency algorithms includes at least one saliency algorithm in which saliency is determined based on local contrast.

### Claim 16 (depends on 15)

A method according to claim 15 wherein in the saliency algorithm in which saliency is determined based on local contrast, the local contrast of the image at each of a plurality of spatial frequency scales is weighted according to a relative sensitivity of the human visual system to contrast at the spatial frequency scale.

### Claim 17 (depends on 16)

A method according to claim 16 wherein computing the local contrast at the plurality of spatial frequency scales comprises computing a Gaussian image-pyramid approximation to local contrast.

### Claim 18 (depends on 2)

A method according to claim 2 wherein generating the saliency map comprises generating a map in which saliency is a function of distance from a central area or pixel of the image.

### Claim 19 (depends on 18)

A method according to claim 18 wherein the function of the distance from the central area or pixel is a Gaussian function.

### Claim 20 (depends on 2)

A method according to claim 2 wherein generating the saliency map comprises obtaining metadata specifying one or more special areas within the image and setting the special areas to have increased saliency in the saliency map.

### Claim 21 (depends on 2)

A method according claim 2 wherein establishing the global transformation comprises computing a weighted histogram of the image data.

### Claim 22 (depends on 21)

A method according to claim 21 wherein establishing the global transformation comprises generating a tone mapping function based on the weighted histogram.

### Claim 23 (depends on 21)

A method according to claim 21 wherein pixels having saliency values below a saliency threshold are the ignored in computing the weighted histogram.

### Claim 24 (depends on 23)

A method according to claim 23 comprising setting a value for the saliency threshold based on an input received by way of the user interface.

### Claim 25 (depends on 2)

A method according to claim 2 wherein the image data comprises video data, and the method comprises generating the saliency map at a video source and transmitting the saliency map as metadata associated with the video data.
