# US 10,455,235 B2 · Perceptual three-dimensional (3D) video coding based on depth information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,455,235 B2
- Document ID: US10455235B2
- Status: Granted
- Kind: B2
- Application number: 16/148,764
- Filed: 2018-10-01
- Published: 2019-01-31
- Granted: 2019-10-22
- Earliest claimed: 2011-09-29
- Assignees: Texas Instruments Incorporated
- Applicants: Texas Instruments Incorporated
- Inventors: Do-Kyoung Kwon; Madhukar Budagavi; Ming-Jun Chen
- Web page: https://patentgrep.com/patent/US10455235B2

## CPC

- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- H04N 19/124: Quantisation
- H04N 19/142: Detection of scene cut or scene change
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for encoding a multi-view frame in a video encoder is provided that includes computing a depth quality sensitivity measure for a multi-view coding block in the multi-view frame, computing a depth-based perceptual quantization scale for a 2D coding block of the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the 2D frame including the 2D coding block, and encoding the 2D coding block using the depth-based perceptual quantization scale.

## Claims

### Claim 1

A method comprising:
receiving a depth map for a multi-view frame, the multi-view frame comprises a multi-view coding block and the multi-view coding block comprises a 2D coding block;
computing a depth quality sensitivity measure for the multi-view coding block in the multi-view frame based in part on the received depth map;
computing a depth-based perceptual quantization scale for the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the multi-view frame; and
encoding the 2D coding block using the depth-based perceptual quantization scale.

### Claim 2 (depends on 1)

The method of claim 1, wherein the 2D coding block is a macroblock.

### Claim 3 (depends on 1)

The method of claim 1, wherein the multi-view frame is a stereoscopic frame comprising a left 2D frame and a right 2D frame.

### Claim 4 (depends on 1)

The method of claim 1, wherein computing a depth quality sensitivity measure comprises computing the depth quality sensitivity measure as a function of a depth of the multi-view coding block and a depth with a least quality sensitivity.

### Claim 5 (depends on 4)

The method of claim 4, wherein the function is f(d,d0)=c0·|d−d0|+c1, wherein c0 and c1 are tuning parameters, d is the depth of the multi-view coding block, and d0 is the depth with the least quality sensitivity.

### Claim 6 (depends on 5)

The method of claim 5, wherein the depth with the least quality sensitivity is the farthest depth of a multi-view coding block in the multi-view frame.

### Claim 7 (depends on 5)

The method of claim 5, wherein the depth with the least quality sensitivity is user-specified.

### Claim 8 (depends on 1)

The method of claim 1, wherein computing a depth-based perceptual quantization scale comprises applying a depth-based perceptual scaling factor to the base quantization scale, wherein the depth-based perceptual scaling factor is determined based on the depth quality sensitive measure, an average of depth quality sensitivity measures of all multi-view coding blocks in the multi-view frame, and a number of multi-view coding blocks in the multi-view frame.

### Claim 9 (depends on 8)

The method of claim 8, wherein the depth-based perceptual scaling factor is computed as
N
·
DQS
avg
N
·
f
⁡
(
d
j
,
d
0
)
+
DQS
avg
,
wherein N is the number of multi-view coding blocks in the multi-view frame, DQSavg is the average of depth quality sensitivity measures, and f(dj,d0) is the depth quality sensitivity measure.

### Claim 10

An apparatus comprising:
Means for receiving a depth map for a multi-view frame, the multi-view frame comprises a multi-view coding block and the multi-view coding block comprises a 2D coding block;
means for computing a depth quality sensitivity measure for a multi-view coding block in the multi-view frame based in part on the received depth map;
means for computing a depth-based perceptual quantization scale the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the multi-view frame; and
means for encoding the 2D coding block using the depth-based perceptual quantization scale.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the 2D coding block is a macroblock.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein the multi-view frame is a stereoscopic frame comprising a left 2D frame and a right 2D frame.

### Claim 13 (depends on 10)

The apparatus of claim 10, wherein the means for computing a depth quality sensitivity measure comprises means for computing the depth quality sensitivity measure as a function of a depth of the multi-view coding block and a depth with a least quality sensitivity.

### Claim 14 (depends on 10)

The apparatus of claim 10, wherein the function is f(d,d0)=c0·|d−d0|+c1, wherein c0 and c1 are tuning parameters, d is the depth of the multi-view coding block, and d0 is the depth with the least quality sensitivity.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the depth with the least quality sensitivity is the farthest depth of a multi-view coding block in the multi-view frame.

### Claim 16 (depends on 14)

The apparatus of claim 14, wherein the depth with the least quality sensitivity is user-specified.

### Claim 17 (depends on 10)

The apparatus of claim 10, wherein computing a depth-based perceptual quantization scale comprises applying a depth-based perceptual scaling factor to the base quantization scale, wherein the depth-based perceptual scaling factor is determined based on the depth quality sensitive measure, an average of depth quality sensitivity measures of all multi-view coding blocks in the multi-view frame, and a number of multi-view coding blocks in the multi-view frame.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the depth-based perceptual scaling factor is computed as
N
·
DQS
avg
N
·
f
⁡
(
d
j
,
d
0
)
+
DQS
avg
,
wherein N is the number of multi-view coding blocks in the multi-view frame, DQSavg is the average of depth quality sensitivity measures, and f(dj,d0) is the depth quality sensitivity measure.

### Claim 19

A non-transitory computer-readable medium storing software instructions that, when executed by a processor, perform a method, the method comprising:
receiving a depth map for a multi-view frame, the multi-view frame comprises a multi-view coding block and the multi-view coding block comprises a 2D coding block;
computing a depth quality sensitivity measure for the multi-view coding block in the multi-view frame based in part on the received depth map;
computing a depth-based perceptual quantization scale for the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the multi-view frame; and
encoding the 2D coding block using the depth-based perceptual quantization scale.

### Claim 20 (depends on 19)

The computer readable medium of claim 19, wherein computing a depth quality sensitivity measure comprises computing the depth quality sensitivity measure as a function of a depth of the multi-view coding block and a depth with a least quality sensitivity.