# US 2017/0289539 A1 · Method for Determining a Visual Quality Index of a High Dynamic Range Video Sequence

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0289539 A1
- Document ID: US20170289539A1
- Status: Published application
- Kind: A1
- Application number: 15/087,578
- Filed: 2016-03-31
- Published: 2017-10-05
- Earliest claimed: 2016-03-31
- Assignees: Centre National de la Recherche Scientifique (CNRS); Universite De Nantes
- Applicants: Centre National de la Recherche Scientifique - CNRS; Universite de Nantes
- Inventors: Patrick Le Callet; Matthieu Perreira Da Silva; Manish Narwaria
- Web page: https://patentgrep.com/patent/US20170289539A1

## CPC

- H04N 17/02: for colour television signals
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06V 10/993: {Evaluation of the quality of the acquired pattern}
- G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for determining objectively a visual quality index of at least one high dynamic range video sequence, referred to as an HDR sequence, distorted by image processing operations and issued from a reference high dynamic range video sequence, referred to a reference sequence or a reference HDR sequence. The method is based on signal pre-processing, transformation, and subsequent frequency based decomposition. Video quality is then computed based on a spatio-temporal analysis that relates to human eye fixation behavior during video viewing. One advantage of this method is that it does not involve expensive computations.

## Claims

### Claim 1

A method for determining a visual quality index of at least one high dynamic range video sequence (HDR sequence), distorted by image processing operations and issued from a reference high dynamic range video sequence, each of the HDR sequence and the reference HDR sequence comprising Nframe video frames t, with N≧2 and tε[1 . . . Nframe], each video frame t comprising a plurality of pixels organized into rows and columns and each pixel having at least a luminance value, the method comprising:
applying a transformation to the video frames t of the HDR sequence and the reference HDR sequence in order to obtain video frames t of the HDR sequence and the reference HDR sequence in a perceived luminance domain, the transformed luminance values of the video frames tin the perceived luminance domain being substantially linear to the luminance values perceived by the human visual system for the HDR sequence and the reference HDR sequence,
computing, for each couple of frames t of the HDR sequence and the reference HDR sequence in the perceived luminance domain, Nscale×Norient similarity frames Simt,s,o representative of a perceptual similarity between the frame t of the HDR sequence and the frame t of the reference HDR sequence at different spatial scales s and different spatial orientations o, with sε[1 . . . Nscale] and oε[1 . . . Norient], a similarity value being associated to each pixel of the similarity frame Simt,s,o,
computing, for each couple of frames t of the HDR sequence and the reference HDR sequence, a global similarity frame Simt based on the computed similarity frames Simt,s,o at the different spatial scales s and the different spatial orientations o,
pooling, for each group of q consecutive global similarity frames Simt, with q≧2, and for each one of a plurality of spatio-temporal tubes within said group of q consecutive global similarity frames Simt, the similarity values of the pixels included in said spatio-temporal tubes in order to generate a short term error value for each said spatio-temporal tubes, the short term error values of each spatio-temporal tube being included into an error map,
pooling at least a portion of the short term error values of each error map in order to generate a short term quality score for each group of q consecutive global similarity frames Simt, and
computing the visual quality index of the HDR sequence based on said short term quality scores.

### Claim 2 (depends on 1)

The method of claim 1, wherein the portion of the short term error values of each error map comprises the m lowest short term error values of the error map, with m being an integer value lower than the total number of short term error values in the error map.

### Claim 3 (depends on 2)

The method of claim 2, wherein the number m is a predetermined percentage of the total number of short term error values in the error map.

### Claim 4 (depends on 3)

The method of claim 3, wherein the predetermined percentage is between about 5% and about 50%.

### Claim 5 (depends on 1)

The method of claim 1, wherein the method further comprises a preliminary step, before transforming the HDR sequence and the reference HDR sequence into the perceived luminance domain, the preliminary step including transforming the luminance values of the HDR sequence and the reference HDR sequence into emitted luminance values.

### Claim 6 (depends on 1)

The method of claim 1, wherein the similarity frame Simt,s,o associated to the frames t of the HDR sequence and the reference HDR sequence for a spatial scale s and a spatial orientation o is computed by the steps of:
applying a log-Gabor filter to the frame t of the HDR sequence and the frame t of the reference HDR sequence in the perceived luminance domain at the spatial scale s and the spatial orientation o,
computing an inverse Fourier Transform of the product of the results of the log-Gabor filter for the frame t of the HDR sequence and the frame t of the reference HDR sequence in order to generate a subband frame lt,s,o(HDR) for the frame t of the HDR sequence and a subband frame lt,s,o(REF) or the frame t of the reference HDR sequence, and
computing the similarity frame Simt,s,o based on the subband frames lt,s,o(HDR) and lt,s,o(REF).

### Claim 7 (depends on 1)

The method of claim 1, wherein the global similarity frame Simt is defined by the formula:
Sim
t
=
1
N
scale
×
N
orient

∑
s
=
1
N
scale

∑
o
=
1
N
orient

Sim
t
,
s
,
o

### Claim 8 (depends on 1)

The method of claim 1, wherein the spatio-temporal tubes are non-overlapping spatio-temporal tubes.

### Claim 9 (depends on 1)

The method of claim 1, wherein the video frames t of the HDR sequence and the reference HDR sequence in the perceived luminance domain are generated by applying a perceptually uniform encoding to the video frames t of the HDR sequence and the reference HDR sequence or, when appropriate, to the video frames t of the HDR sequence and the reference sequence issued from said preliminary step.
