# US 11,909,957 B2 · Encoder and decoder, encoding method and decoding method for reference picture resampling extensions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,909,957 B2
- Document ID: US11909957B2
- Status: Granted
- Kind: B2
- Application number: 17/763,508
- Filed: 2020-09-24
- Published: 2022-11-24
- Granted: 2024-02-20
- Earliest claimed: 2019-09-24
- Assignees: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
- Applicants: Fraunhofer-Gesellschaft Zur FöRderung Der Angewandten Forschung E.V.
- Inventors: Yago Sánchez De La Fuente; Karsten Sühring; Cornelius Hellge; Thomas Schierl; Robert Skupin; Thomas Wiegand
- Web page: https://patentgrep.com/patent/US11909957B2

## CPC

- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/167: Position within a video image, e.g. region of interest [ROI]
- H04N 19/172: the region being a picture, frame or field
- H04N 19/29: involving scalability at the object level, e.g. video object layer [VOL]
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A video decoder (151) for decoding an encoded video signal comprising encoded picture data to reconstruct a plurality of pictures of a video sequence of a video. The video decoder (151) comprises an input interface (160) configured for receiving the encoded video signal comprising the encoded picture data. Moreover, the video decoder (151) comprises a data decoder (170) configured for reconstructing the plurality of pictures of the video sequence depending on the encoded picture data. Moreover, further video decoders, video encoders, systems, methods for encoding and decoding, computer programs and encoded video signals according to embodiments are provided.

## Claims

### Claim 1

A decoding apparatus comprising a microprocessor and memory, the memory comprising a computer program comprising instructions, which, when executed by the microprocessor, cause the decoding apparatus to:
decode, from a picture parameter set (PPS) of a video data stream, conformance cropping window parameters comprising a pps_conf_win_left_offset, a pps_conf_win_top_offset, a pps_conf_win_right_offset, and a pps_conf_win_bottom_offset;
decode, from the picture parameter set (PPS), a
pps_scaling_window_explicit_signalling_flag;
determine that the pps_scaling_window_explicit_signalling_flag has a value of false; and
based on the determination that the pps_scaling_window_explicit_signalling_flag has a value of false, perform upscaling of a region within a reference picture by using the pps_conf_win_left_offset, the pps_conf_win_top_offset, the pps_conf_win_right_offset, and the pps_conf_win_bottom_offset.

### Claim 2 (depends on 1)

The decoding apparatus of claim 1, wherein the computer program comprises instructions, which, when executed by the microprocessor, further cause the decoding apparatus to:
decode, from a second picture parameter set (PPS) of the video data stream, second conformance cropping window parameters comprising a second pps_conf_win_left_offset, a second pps_conf_win_top_offset, a second pps_conf_win_right_offset, and a second pps_conf_win_bottom_offset;
decode, from the second picture parameter set (PPS), a second pps_scaling_window_explicit_signalling_flag;
determine that the second pps_scaling_window_explicit_signalling_flag has a value of true; and
based on the determination that the second
pps_scaling_window_explicit_signalling_flag has a value of true, perform upscaling of a region within a second reference picture without using the second pps_conf_win_left_offset, the second pps_conf_win_top_offset, the second pps_conf_win_right_offset, and the second pps_conf_win_bottom_offset.

### Claim 3 (depends on 2)

The decoding apparatus of claim 2, wherein the computer program comprises instructions, which, when executed by the microprocessor, further cause the decoding apparatus to:
decode, from the second PPS, ROI offset parameters and additional ROI offset parameters,
wherein the upscaling of the region within the second reference picture is performed by using the ROI offset parameters and additional ROI offset parameters to determine resampled reference picture.

### Claim 4 (depends on 3)

The decoding apparatus of claim 3, wherein the computer program comprises instructions, which, when executed by the microprocessor, further cause the decoding apparatus to:
identify a reference sample by finding a collocated sample using the ROI offset parameters and a motion vector;
determine that the reference sample is outside an enlarged ROI indicated by the additional offset parameters; and
based on the determination that the reference sample is outside the enlarged ROI, clip the motion vector.

### Claim 5

An encoding apparatus comprising a microprocessor and memory, the memory comprising a computer program comprising instructions, which, when executed by the microprocessor, cause the encoding apparatus to:
encode, into a picture parameter set (PPS) of a video data stream, conformance cropping window parameters comprising a pps_conf_win_left_offset, a pps_conf_win_top_offset, a pps_conf_win_right_offset, and a pps_conf_win_bottom_offset;
determine to use the pps_conf_win_left_offset, the pps_conf_win_top_offset, the pps_conf_win_right_offset, and the pps_conf_win_bottom_offset to perform upscaling of a region within a reference picture;
based on the determination to use the pps_conf_win_left_offset, the pps_conf_win_top_offset, the pps_conf_win_right_offset, and the pps_conf_win_bottom_offset to perform upscaling of the region within the reference picture,
encode, into the picture parameter set (PPS), a pps_scaling_window_explicit_signalling_flag with a value of false.

### Claim 6 (depends on 5)

The encoding apparatus of claim 5, wherein the computer program comprises instructions, which, when executed by the microprocessor, further cause the encoding apparatus to:
encode, into a second picture parameter set (PPS) of a video data stream, second conformance cropping window parameters comprising a second
pps_conf_win_left_offset, a second pps_conf_win_top_offset, a second pps_conf_win_right_offset, and a second pps_conf_win_bottom_offset;
determine to not use the second pps_conf_win_left_offset, the second pps_conf_win_top_offset, the second pps_conf_win_right_offset, and the second pps_conf_win_bottom_offset for upscaling of a region within a second reference picture;
based on the determination not to use the second pps_conf_win_left_offset, the second pps_conf_win_top_offset, the second pps_conf_win_right_offset, and the second pps_conf_win_bottom_offset to perform upscaling of the region within the second reference picture,
encode, into the second picture parameter set (PPS), a second pps_scaling_window_explicit_signalling_flag with a value of true.

### Claim 7 (depends on 6)

The encoding apparatus of claim 6, wherein the computer program comprises instructions, which, when executed by the microprocessor, further cause the encoding apparatus to:
encode, into the second PPS, ROI offset parameters and additional ROI offset parameters,
wherein the upscaling of the region within the second reference picture is performed by using the ROI offset parameters and additional ROI offset parameters to determine resampled reference picture.

### Claim 8 (depends on 7)

The encoding apparatus of claim 7, wherein the computer program comprises instructions, which, when executed by the microprocessor, further cause the encoding apparatus to:
identify a reference sample by finding a collocated sample using the ROI offset parameters and a motion vector;
determine that the reference sample is outside an enlarged ROI indicated by the additional offset parameters; and
based on the determination that the reference sample is outside the enlarged ROI, clip the motion vector.

### Claim 9

A method for decoding video, the method comprising:
decoding, from a picture parameter set (PPS) of a video data stream, conformance cropping window parameters comprising a pps_conf_win_left_offset, a pps_conf_win_top_offset, a pps_conf_win_right_offset, and a pps_conf_win_bottom_offset;
decoding, from the picture parameter set (PPS), a pps_scaling_window_explicit_signalling_flag;
determining that the pps_scaling_window_explicit_signalling_flag has a value of false; and
based on the determination that the pps_scaling_window_explicit_signalling_flag has a value of false, performing upscaling of a region within a reference picture by using the pps_conf_win_left_offset, the pps_conf_win_top_offset, the pps_conf_win_right_offset, and the pps_conf_win_bottom_offset.

### Claim 10 (depends on 9)

The method of claim 9, the method further comprising:
decoding, from a second picture parameter set (PPS) of the video data stream, second conformance cropping window parameters comprising a second pps_conf_win_left_offset, a second pps_conf_win_top_offset, a second pps_conf_win_right_offset, and a second pps_conf_win_bottom_offset;
decoding, from the second picture parameter set (PPS), a second pps_scaling_window_explicit_signalling_flag;
determining that the second pps_scaling_window_explicit_signalling_flag has a value of true; and
based on the determination that the second
pps_scaling_window_explicit_signalling_flag has a value of true, performing upscaling of a region within a second reference picture without using the second pps_conf_win_left_offset, the second pps_conf_win_top_offset, the second pps_conf_win_right_offset, and the second pps_conf_win_bottom_offset.

### Claim 11 (depends on 10)

The method of claim 10, the method further comprising:
decoding, from the second PPS, ROI offset parameters and additional ROI offset parameters,
wherein the upscaling of the region within the second reference picture is performed by using the ROI offset parameters and additional ROI offset parameters to determine resampled reference picture.

### Claim 12 (depends on 11)

The method of claim 11, the method further comprising:
identifying a reference sample by finding a collocated sample using the ROI offset parameters and a motion vector;
determining that the reference sample is outside an enlarged ROI indicated by the additional offset parameters; and
based on the determination that the reference sample is outside the enlarged ROI, clipping the motion vector.

### Claim 13

A method for encoding video, the method comprising:
encoding, into a picture parameter set (PPS) of a video data stream, conformance cropping window parameters comprising a pps_conf_win_left_offset, a pps_conf_win_top_offset, a pps_conf_win_right_offset, and a pps_conf_win_bottom_offset;
determining to use the pps_conf_win_left_offset, the pps_conf_win_top_offset, the pps_conf_win_right_offset, and the pps_conf_win_bottom_offset to perform upscaling of a region within a reference picture;
based on the determination to use the pps_conf_win_left_offset, the pps_conf_win_top_offset, the pps_conf_win_right_offset, and the pps_conf_win_bottom_offset to perform upscaling of the region within the reference picture,
encoding, into the picture parameter set (PPS), a pps_scaling_window_explicit_signalling_flag with a value of false.

### Claim 14 (depends on 13)

The method of claim 13, the method further comprising:
encoding, into a second picture parameter set (PPS) of a video data stream, second conformance cropping window parameters comprising a second pps_conf_win_left_offset, a second pps_conf_win_top_offset, a second pps_conf_win_right_offset, and a second pps_conf_win_bottom_offset;
determining to not use the second pps_conf_win_left_offset, the second pps_conf_win_top_offset, the second pps_conf_win_right_offset, and the second pps_conf_win_bottom_offset for upscaling of a region within a second reference picture;
based on the determination not to use the second pps_conf_win_left_offset, the second pps_conf_win_top_offset, the second pps_conf_win_right_offset, and the second pps_conf_win_bottom_offset to perform upscaling of the region within the second reference picture,
encoding, into the second picture parameter set (PPS), a second pps_scaling_window_explicit_signalling_flag with a value of true.

### Claim 15 (depends on 14)

The method of claim 14, the method further comprising:
encoding, into the second PPS, ROI offset parameters and additional ROI offset parameters,
wherein the upscaling of the region within the second reference picture is performed by using the ROI offset parameters and additional ROI offset parameters to determine resampled reference picture.

### Claim 16 (depends on 15)

The method of claim 15, the method further comprising:
identifying a reference sample by finding a collocated sample using the ROI offset parameters and a motion vector;
determining that the reference sample is outside an enlarged ROI indicated by the additional offset parameters; and
based on the determination that the reference sample is outside the enlarged ROI, clipping the motion vector.