# US 2020/0288136 A1 · Video picture processing method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0288136 A1
- Document ID: US20200288136A1
- Status: Published application
- Kind: A1
- Application number: 16/880,757
- Filed: 2020-05-21
- Published: 2020-09-10
- Earliest claimed: 2018-01-03
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Huameng FANG; Peiyun DI
- Web page: https://patentgrep.com/patent/US20200288136A1

## CPC

- H04N 19/597: specially adapted for multi-view video sequence encoding
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 7/40: Analysis of texture (depth or shape recovery from texture G06T7/529)
- G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00)
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/146: Data rate or code amount at the encoder output
- H04N 19/167: Position within a video image, e.g. region of interest [ROI]
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N 19/86: involving reduction of coding artifacts, e.g. of blockiness
- H04N 19/88: involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A video picture processing method includes: sampling an initial picture, to obtain a target picture; dividing the target picture, to obtain a plurality of sub-pictures; extending picture borders of a first-category sub-picture, to obtain an extended first-category sub-picture; downsampling the extended first-category sub-picture, to obtain a downsampled first-category sub-picture; encoding a second-category sub-picture and the downsampled first-category sub-picture, to obtain bitstreams; and writing location information and extension information of the plurality of sub-pictures into the bitstreams, where extension information of the first-category sub-picture is used to indicate that the picture borders of the first-category sub-picture have been extended and indicate extension widths of the picture borders, and extension information of the second-category sub-picture is used to indicate that picture borders of the second-category sub-picture have not been extended.

## Claims

### Claim 1

A video picture processing method, comprising:
sampling an initial picture, to obtain a target picture;
dividing the target picture, to obtain a plurality of sub-pictures, wherein the plurality of sub-pictures comprise a first-category sub-picture and a second-category sub-picture, the first-category sub-picture is obtained by sampling a picture in a first region of the initial picture, the second-category sub-picture is obtained by sampling a picture in a second region of the initial picture, a sampling rate of the picture in the first region of the initial picture is lower than a sampling rate of the picture in the second region of the initial picture, and resolution of the first-category sub-picture is higher than resolution of the second-category sub-picture;
extending picture borders of the first-category sub-picture, to obtain an extended first-category sub-picture;
downsampling the extended first-category sub-picture, to obtain a downsampled first-category sub-picture, wherein resolution of the downsampled first-category sub-picture is the same as the resolution of the second-category sub-picture;
encoding the second-category sub-picture and the downsampled first-category sub-picture, to obtain bitstreams; and
writing location information and extension information of the plurality of sub-pictures into the bitstreams, wherein location information is used to indicate a location of a sub-picture in the target picture, extension information of the first-category sub-picture is used to indicate that the picture borders of the first-category sub-picture have been extended and indicate extension widths of the picture borders of the first-category sub-picture, and extension information of the second-category sub-picture is used to indicate that picture borders of the second-category sub-picture have not been extended.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the writing location information and extension information of the plurality of sub-pictures into the bitstreams comprises:
encapsulating the location information and the extension information of each of the plurality of sub-pictures into a same box.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the initial picture is a panoramic video picture.

### Claim 4

A video picture processing method, comprising:
obtaining target bitstreams;
parsing the target bitstreams, to obtain a plurality of sub-pictures and location information and extension information of the plurality of sub-pictures, wherein location information is used to indicate a location of a sub-picture in a to-be-displayed picture, extension information is used to indicate whether picture borders of a sub-picture have been extended, and when the extension information indicates that the picture borders of the sub-picture have been extended, the sub-picture belongs to a first-category sub-picture, and the extension information is further used to indicate extension widths of the picture borders of the sub-picture, or when the extension information indicates that the picture borders of the sub-picture have not been extended, the sub-picture belongs to a second-category sub-picture;
determining the first-category sub-picture and the second-category sub-picture from the plurality of sub-pictures based on the extension information of the plurality of sub-pictures;
determining target resolution based on the location information and the extension information of the first-category sub-picture;
upsampling the first-category sub-picture based on the target resolution, to obtain an upsampled first-category sub-picture, wherein resolution of the upsampled first-category sub-picture is the target resolution;
splicing the upsampled first-category sub-picture and the second-category sub-picture based on the location information of the plurality of sub-pictures;
performing weighted processing on values of pixels in an overlapping region between upsampled first-category sub-pictures, to obtain weighted first-category sub-pictures; and
filtering a border region between adjacent second-category sub-pictures and a border region between a weighted first-category sub-picture and a second-category sub-picture, to obtain the to-be-displayed picture, wherein the border region comprises regions of preset sizes on two sides of a boundary between two sub-pictures.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the determining target resolution based on the location information and the extension information of the first-category sub-picture comprises:
determining a width and a height of the first-category sub-picture based on the location information of the first-category sub-picture;
determining, based on the extension information of the first-category sub-picture, extension widths of picture borders of the first-category sub-picture; and
determining the target resolution based on the width and the height of the first-category sub-picture and the extension widths of the picture borders of the first-category sub-picture.

### Claim 6 (depends on 4)

The method according to claim 4, wherein the location information and the extension information of each of the plurality of sub-pictures are encapsulated into a same box.

### Claim 7 (depends on 4)

The method according to claim 4, wherein the location information and the extension information of the plurality of sub-pictures are encapsulated into supplemental enhancement information (SEI) or a media presentation description (MPD).

### Claim 8 (depends on 4)

The method according to claim 4, wherein the overlapping region between the upsampled first-category sub-pictures comprises an overlapping region between a first sub-picture and a second sub-picture, a picture that is in the overlapping region and that is of the first sub-picture is an extended picture, and a picture that is in the overlapping region and that is of the second sub-picture is an overlapping picture; and the performing weighted processing on values of pixels in an overlapping region between upsampled first-category sub-pictures comprises:
performing weighted processing on values of pixels that are in the extended picture and the overlapping picture and that are at a same location in the overlapping region, wherein a pixel that is in the overlapping picture and that is closer to a center location of the first sub-picture has a greater weighting coefficient, and a pixel that is in the extended picture and that is closer to the center location of the first sub-picture has a smaller weighting coefficient.

### Claim 9 (depends on 4)

The method according to claim 4, wherein the filtering the border region between adjacent second-category sub-pictures and the border region between the weighted first-category sub-picture and the second-category sub-picture, to obtain the to-be-displayed picture comprises:
determining pixel value gradients of pictures in the border region; and
when a pixel value gradient of a picture in the border region is less than a preset gradient, performing filtering processing on the picture in the border region; or
when a pixel value gradient of a picture in the border region is greater than or equal to the preset gradient, omitting filtering processing on the picture in the border region.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the border region comprises a first region and a second region, the first region and the second region are separately located on two sides of a boundary, and the performing filtering processing on the picture in the border region comprises:
determining a texture strength of the border region;
performing filtering processing on the border region by using a first filtering strength, when the texture strength of the border region is greater than or equal to a first preset strength; or determining a texture strength of a target region when the texture strength of the border region is less than the first preset strength, wherein the target region is the first region or the second region; and
filtering a picture in the target region by using the first filtering strength, when the texture strength of the target region is greater than or equal to a second preset strength; or
filtering the picture in the target region by using a second filtering strength, when the texture strength of the target region is less than the second preset strength.

### Claim 11 (depends on 10)

The method according to claim 10, wherein an effect range of the first filtering strength is less than an effect range of the second filtering strength.

### Claim 12

A video picture processing apparatus, comprising:
a non-transitory memory having processor-executable instructions stored thereon; and
a processor, coupled to the memory, configured to execute the processor-executable instructions to:
sample an initial picture, to obtain a target picture;
divide the target picture, to obtain a plurality of sub-pictures, wherein the plurality of sub-pictures comprise a first-category sub-picture and a second-category sub-picture, the first-category sub-picture is obtained by sampling a picture in a first region of the initial picture, the second-category sub-picture is obtained by sampling a picture in a second region of the initial picture, a sampling rate of the picture in the first region of the initial picture is lower than a sampling rate of the picture in the second region of the initial picture, and resolution of the first-category sub-picture is higher than resolution of the second-category sub-picture;
extend picture borders of the first-category sub-picture, to obtain an extended first-category sub-picture;
downsample the extended first-category sub-picture, to obtain a downsampled first-category sub-picture, wherein resolution of the downsampled first-category sub-picture is the same as the resolution of the second-category sub-picture;
encode the second-category sub-picture and the downsampled first-category sub-picture, to obtain bitstreams; and
write location information and extension information of the plurality of sub-pictures into the bitstreams, wherein location information is used to indicate a location of a sub-picture in the target picture, extension information of the first-category sub-picture is used to indicate that the picture borders of the first-category sub-picture have been extended and indicate extension widths of the picture borders of the first-category sub-picture, and extension information of the second-category sub-picture is used to indicate that picture borders of the second-category sub-picture have not been extended.

### Claim 13 (depends on 12)

The apparatus according to claim 12, wherein the processor is configured to execute the processor-executable instructions to encapsulate the location information and the extension information of each of the plurality of sub-pictures into a same box.

### Claim 14 (depends on 12)

The apparatus according to claim 12, wherein the initial picture is a planar picture obtained by mapping a panoramic video picture.