# US 2016/0255356 A1 · Evaluation Measure for Images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0255356 A1
- Document ID: US20160255356A1
- Status: Published application
- Kind: A1
- Application number: 14/759,553
- Filed: 2015-05-21
- Published: 2016-09-01
- Earliest claimed: 2014-07-01
- Applicants: Telefonaktiebolaget L M Ericsson (publ)
- Inventors: Jacob Ström; Jonatan Samuelsson
- Web page: https://patentgrep.com/patent/US20160255356A1

## CPC

- H04N 19/426: using memory downsizing methods
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/90: Determination of colour characteristics
- H04N 19/103: Selection of coding mode or of prediction mode
- H04N 19/136: Incoming video signal characteristics or properties
- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- G06T 2207/20208: High dynamic range [HDR] image processing
- G06T 2207/30168: Image quality inspection
- H04L 1/00: Arrangements for detecting or preventing errors in the information received {(correcting synchronisation H04L7/00)}
- H04N 19/00: Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Tone mapped versions (11) are created of an image (10) and of a reconstructed image (20) obtained following compression of the image (10) for each tone mapper setting value among a set of multiple such tone mapper setting values. A respective error value is calculated over a non-excluded subset of pixels in the tone mapped versions (11, 21). The non-excluded subset of pixels does not meet at least one exclusion criterion. A mean squared error is calculated as an average for the respective non-excluded subset of pixels in the multiple tone mapped versions (11, 21). An evaluation measure is calculated based on this mean squared error. The evaluation measure is suitable as objective evaluation measure for HDR images and video.

## Claims

### Claim at position 1

1-28. (canceled)

### Claim 29

A method for determining an evaluation measure for an image, the method comprising:
performing, for each tone mapper setting value among a set of multiple tone mapper setting values, each tone mapper setting value representing a respective shutter speed, the steps of:
creating a tone mapped version of said image;
creating a tone mapped version of a version of said image obtained following compression of said image; and
calculating an error over a non-excluded subset of pixels in said tone mapped version of said image and in said tone mapped version of said version of said image obtained following compression of said image, said non-excluded subset of pixels does not meet at least one exclusion criterion;
calculating a mean squared error as an average error for said respective non-excluded subset of pixels in said multiple tone mapped versions of said image and said multiple tone mapped versions of said version of said image obtained following compression of said image; and
calculating said evaluation measure based on said mean squared error.

### Claim 30 (depends on 29)

The method of claim 29, wherein
creating said tone mapped version comprises calculating a tone mapped version of said image as:
RLDRc(x, y)=clamp(0, T, round (T×(2c×RHDR(x, y))(1/γ)))
GLDRc(x, y)=clamp(0, T, round (T×(2c×GHDR(x, y))(1/γ)))
BLDRc(x, y)=clamp(0, T, round (T×(2c×BHDR(x, y))(1/γ)))
wherein RHDR(x, y), GHDR(x, y), BHDR(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said image,
RLDRC(x, y), GLDRC(x, y), BLDRc(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said tone mapped version of said image, c represents said tone mapper setting value, γ represents a display gamma, T is a saturation value, clamp(0, T , a) is a clamping function outputting 0 if a<0, T if a>T and a if 0<a≦T, and round( ) is a rounding function; and
creating said tone mapped version comprises calculating a tone mapped version of said version of said image obtained following compression of said image as:
RLDR,Cc(x, y)=clamp(0, T, round (T×(2c×RHDR,C(x, y))(1/γ)))
GLDR,Cc(x, y)=clamp(0, T, round (T×(2c×GHDR,C(x, y))(1/γ)))
BLDR,Cc(x, y)=clamp(0, T, round (T×(2c×BHDR,C(x, y))(1/γ)))
wherein RHDR,C(x, y), GHDR,C(x, y), BHDR,C(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said version of said image obtained following compression of said image, and RLDR,CC(x, y), GLDR,CC(x, y), BLDR,Cc(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said tone mapped version of said version of said image obtained following compression of said image.

### Claim 31 (depends on 29)

The method of claim 29, wherein calculating said error comprises calculating said error representing a difference in pixel values of said tone mapped version of said image and said tone mapped version of said version of said image obtained following compression of said image for said non-excluded subset of pixels.

### Claim 32 (depends on 29)

The method of claim 29, wherein said at least one exclusion criterion comprises:
a black pixel; and
a pixel having at least one pixel component equal to or exceeding a saturation value.

### Claim 33 (depends on 32)

The method of claim 32, wherein calculating said error comprises calculating a difference in pixel value of said tone mapped version of said image and said tone mapped version of said version of said image obtained following compression of said image for each pixel position except for pixel positions having a black pixel or having at least one pixel component equal to or exceeding a saturation value in said tone mapped version of said image.

### Claim 34 (depends on 29)

The method of claim 29, wherein said set of multiple tone mapper setting values is [−36, 15].

### Claim 35 (depends on 29)

The method of claim 29, further comprising:
identifying a largest component value V of said image;
selecting a smallest integer value cmin of said tone mapper setting in said set of multiple tone mapper setting values so that 2c×V(1/γ)>0; and
selecting a largest integer value cmax of said tone mapper setting in said set of multiple tone mapper setting values so that 2c×V(1/γ)<T, wherein T is a saturation value and said set of multiple tone mapper setting values is [cmin, cmax].

### Claim 36 (depends on 29)

The method of claim 29, wherein calculating said mean squared error comprises calculating said mean squared error as:
MSE
=
1
Σ
c


S
c


∑
c


∑
S
c

1
3

(
(
Δ


R
c
)
2
+
(
Δ


G
c
)
2
+
(
Δ


B
c
)
2
)
wherein Sc represents said non-excluded subset of pixels for said tone mapper setting value c, ∥Sc∥ represents the number of pixels in said non-excluded subset of pixels for said tone mapper setting value c, ΔRc=RLDRc−RLDR,Cc, ΔGc=GLDRc−GLDR,Cc, ΔBc=BLDRc−BLDR,Cc, RLDRC(x, y), GLDRC(x, y), BLDRc(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said tone mapped version of said image, and
RLDR,CC(x, y), GLDR,CC(x, y), BLDR,Cc(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said tone mapped version of said version of said image obtained following compression of said image.

### Claim 37 (depends on 29)

The method of claim 29, wherein calculating said evaluation measure comprises calculating said evaluation measure as a peak signal-to-noise ratio of said mean squared error.

### Claim 38 (depends on 37)

The method of claim 37, wherein calculating said peak signal-to-noise ratio comprises calculating said peak signal-to-noise ratio as:
mPSNR
=
10
×
log
10

(
T
2
MSE
)
wherein MSE represents said mean squared error and T represents a saturation value.

### Claim 39

An image encoding method comprising:
compressing at least a portion of an image according to multiple compression modes to obtain multiple compressed versions of said at least a portion of said image;
determining, for each compression mode of said multiple compression modes, a respective evaluation measure for said image; and
selecting a compressed version of said at least a portion of said image as an encoded representation of said at least a portion of said image based on said multiple evaluation measures;
wherein said determining of the respective evaluation for said image comprises:
performing, for each tone mapper setting value among a set of multiple tone mapper setting values, each tone mapper setting value representing a respective shutter speed, the steps of:
creating a tone mapped version of said image;
creating a tone mapped version of a version of said image obtained following compression of said image; and
calculating an error over a non-excluded subset of pixels in said tone mapped version of said image and in said tone mapped version of said version of said image obtained following compression of said image, said non-excluded subset of pixels does not meet at least one exclusion criterion;
calculating a mean squared error as an average error for said respective non-excluded subset of pixels in said multiple tone mapped versions of said image and said multiple tone mapped versions of said version of said image obtained following compression of said image; and
calculating said evaluation measure based on said mean squared error.

### Claim 40 (depends on 39)

The method of claim 39, wherein selecting said compressed version comprises selecting, as said encoded representation of said at least a portion of said image, a compressed version of said at least a portion of said image having a highest evaluation measure among said multiple compressed versions of said at least a portion of said image.

### Claim 41

An encoder for determining an evaluation measure for an image, said encoder comprising processing circuitry configured to:
create, for each tone mapper setting value among a set of multiple tone mapper setting values, each tone mapper setting value represents a respective shutter speed, a tone mapped version of said image;
create, for each tone mapper setting value among said set of multiple tone mapper setting values, a tone mapped version of a version of said image obtained following compression of said image;
calculate, for each tone mapper setting value among said set of multiple tone mapper setting values, an error over a non-excluded subset of pixels in said tone mapped version of said image and said tone mapped version of said version of said image obtained following compression of said image, said non-excluded subset of pixels does not meet at least one exclusion criterion;
calculate a mean squared error as an average error for said respective non-excluded subset of pixels in said multiple tone mapped versions of said image and said multiple tone mapped versions of said version of said image obtained following compression of said image; and
calculate said evaluation measure based on said mean squared error.

### Claim 42 (depends on 41)

The encoder of claim 41, wherein the processing circuitry is configured to calculate a tone mapped version of said image as:
RLDRc(x, y)=clamp(0, T, round (T×(2c×RHDR(x, y))(1/γ)))
GLDRc(x, y)=clamp(0, T, round (T×(2c×GHDR(x, y))(1/γ)))
BLDRc(x, y)=clamp(0, T, round (T×(2c×BHDR(x, y))(1/γ)))
wherein RHDR(x, y), GHDR(x, y), BHDR(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said image,
RLDRC(x, y), GLDRC(x, y), BLDRc(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said tone mapped version of said image, c represents said tone mapper setting value, γ represents a display gamma, T is a saturation value, clamp(0, T, a) is a clamping function outputting 0 if a<0, T if a>T and a if 0≦a≦T, and round( ) is a rounding function; and
said encoder is configured to calculate a tone mapped version of said version of said image obtained following compression of said image as:
RLDR,Cc(x, y)=clamp (0, T, round(T×(2c×RHDR,C(x, y))(1/γ)))
GLDR,Cc(x, y)=clamp (0, T, round(T×(2c×GHDR,C(x, y))(1/γ)))
BLDR,Cc(x, y)=clamp (0, T, round(T×(2c×BHDR,C(x, y))(1/γ)))
wherein RHDR,C(x, y), GHDR,C(x, y), BHDR,C(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said version of said image obtained following compression of said image, and RLDR,CC(x, y), GLDR,CC(x, y), BLDR,Cc(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said tone mapped version of said version of said image obtained following compression of said image.

### Claim 43 (depends on 41)

The encoder of claim 41, wherein the processing circuitry is configured to calculate said error representing a difference in pixel values of said tone mapped version of said image and said tone mapped version of said version of said image obtained following compression of said image for said non-excluded subset of pixels.

### Claim 44 (depends on 41)

The encoder of claim 41, wherein said at least one exclusion criterion comprises:
a black pixel; and
a pixel having at least one pixel component equal to or exceeding a saturation value.

### Claim 45 (depends on 44)

The encoder of claim 44, wherein the processing circuitry is configured to calculate a difference in pixel value of said tone mapped version of said image and said tone mapped version of said version of said image obtained following compression of said image for each pixel position except for pixel positions having a black pixel or having at least one pixel component equal to or exceeding a saturation value in said tone mapped version of said image.

### Claim 46 (depends on 41)

The encoder of claim 41, wherein
the processing circuitry is configured to identify a largest component value V of said image;
the processing circuitry is configured to select a smallest integer value cmin of said tone mapper setting in said set of multiple tone mapper setting values so that 2c×V(1/γ)>T; and
the processing circuitry is configured to select a largest integer value cmax of said tone mapper setting in said set of multiple tone mapper setting values so that 2c×V(1/γ)<T, wherein T is a saturation value and said set of multiple tone mapper setting values is [cmin, cmax].

### Claim 47 (depends on 41)

The encoder of claim 41, wherein the processing circuitry is configured to calculate said mean squared error as:
MSE
=
1
Σ
c


S
c


∑
c


∑
S
c

1
3

(
(
Δ


R
c
)
2
+
(
Δ


G
c
)
2
+
(
Δ


B
c
)
2
)
wherein Sc represents said non-excluded subset of pixels for said tone mapper setting value c, ∥Sc∥ represents the number of pixels in said non-excluded subset of pixels for said tone mapper setting value c, ΔRc=RLDRc−RLDR,Cc, ΔGc=GLDRc−GLDR,Cc, ΔBc=BLDRc−BLDR,Cc, RLDRC(x, y), GLDRC(x, y), BLDRc(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said tone mapped version of said image, and
RLDR,CC(x, y), GLDR,CC(x, y), BLDR,Cc(x, y) represent a first, a second and a third pixel component of pixel position (x, y) in said tone mapped version of said version of said image obtained following compression of said image.

### Claim 48 (depends on 41)

The encoder of claim 41, wherein the processing circuitry is configured to calculate said evaluation measure as a peak signal-to-noise ratio of said mean squared error.

### Claim 49 (depends on 48)

The encoder of claim 48, wherein the processing circuitry is configured to calculate said peak signal-to-noise ratio as:
mPSNR
=
10
×
log
10

(
T
2
MSE
)
wherein MSE represents said mean squared error and T represents a saturation value.

### Claim 50

An encoder for determining an evaluation measure for an image, the encoder comprising processing circuitry configured to:
create, for each tone mapper setting value among a set of multiple tone mapper setting values, each tone mapper setting value represents a respective shutter speed, a tone mapped version of said image and a tone mapped version of a version of said image obtained following compression of said image;
calculating, for each tone mapper setting value among said set of multiple tone mapper setting values, an error over a non-excluded subset of pixels in said tone mapped version of said image and said tone mapped version of said version of said image obtained following compression of said image, said non-excluded subset of pixels does not meet at least one exclusion criterion;
calculate a mean squared error as an average error for said respective non-excluded subset of pixels in said multiple tone mapped versions of said image and said tone mapped versions of said version of said image obtained following compression of said image; and
calculate said evaluation measure based on said mean squared error.

### Claim 51

An image encoding device comprising processing circuitry configured to:
compress at least a portion of an image according to multiple compression modes to obtain multiple compressed versions of said at least a portion of said image;
determine, for each compression mode of said multiple compression modes, a respective evaluation measure for said image; and
select a compressed version of said at least a portion of said image as an encoded representation of said at least a portion of said image based on said multiple evaluation measures;
wherein the processing circuitry is configured determine the respective evaluation for said image, for each compression mode of said multiple compression modes, by:
performing, for each tone mapper setting value among a set of multiple tone mapper setting values, each tone mapper setting value representing a respective shutter speed, the steps of:
creating a tone mapped version of said image;
creating a tone mapped version of a version of said image obtained following compression of said image; and
calculating an error over a non-excluded subset of pixels in said tone mapped version of said image and in said tone mapped version of said version of said image obtained following compression of said image, said non-excluded subset of pixels does not meet at least one exclusion criterion;
calculating a mean squared error as an average error for said respective non-excluded subset of pixels in said multiple tone mapped versions of said image and said multiple tone mapped versions of said version of said image obtained following compression of said image; and
calculating said evaluation measure based on said mean squared error.

### Claim 52 (depends on 51)

The image encoding device of claim 51, wherein said selector is configured to select, as said encoded representation of said at least a portion of said image, a compressed version of said image having a highest evaluation measure among said multiple compressed versions of said at least a portion said image.

### Claim 53

A non-transitory computer-readable medium comprising computer program instructions stored thereon, which computer program instructions are configured so that, when executed by a processor, the computer program instructions cause said processor to:
create, for each tone mapper setting value representing among a set of multiple tone mapper setting values, each tone mapper setting value represents a respective shutter speed, a tone mapped version of said image and a tone mapped version of a version of said image obtained following compression of said image;
calculate, for each tone mapper setting value among said set of multiple tone mapper setting values, an error over a non-excluded subset of pixels in said tone mapped version of said image and said tone mapped version of said version of said image obtained following compression of said image, said non-excluded subset of pixels does not meet at least one exclusion criterion;
calculate a mean squared error as an average error for said respective non-excluded subset of pixels in said multiple tone mapped versions of said image and said multiple tone mapped versions of said version of said image obtained following compression of said image; and
calculate said evaluation measure based on said mean squared error.
