# US 12,113,962 B1 · Methods and systems for an efficient inter-prediction structure and signaling for low-delay video streaming

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,113,962 B1
- Document ID: US12113962B1
- Status: Granted
- Kind: B1
- Application number: 17/886,111
- Filed: 2022-08-11
- Granted: 2024-10-08
- Earliest claimed: 2022-08-11
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Byeongdoo Choi; Christopher Andrew Segall; Kiran Mukesh Misra
- Web page: https://patentgrep.com/patent/US12113962B1

## CPC

- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/172: the region being a picture, frame or field
- H04N 19/577: Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
- H04N 19/58: Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one (H04N19/23 takes precedence)
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Techniques for an efficient inter-prediction structure and signaling for low-delay streaming of live video are described. According to some examples, a computer-implemented method includes receiving a live video at a content delivery service, determining a subset of candidate reference frames from a plurality of frames received of the live video, generating an identification code, for the subset of candidate reference frames, having a multiple-bit format that includes a first bit value to indicate a corresponding candidate reference frame is a reference frame for an input frame from the live video and a second bit value to indicate the corresponding candidate reference frame is not the reference frame for the input frame from the live video, and, when a bit of the identification code for a first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is one of a forward reference frame and a backward reference frame for the input frame from the live video, an immediately following bit of the identification code being set to the first bit value indicates the first candidate reference frame is also another of the one of the forward reference frame and the backward reference frame for the input frame from the live video, performing a real time encode of the input frame of the live video based at least in part on the identification code to generate an encoded frame by the content delivery service, and transmitting the encoded frame from the content delivery service to a viewer device.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving a live video at a content delivery service;
determining a subset of candidate reference frames from a plurality of frames received of the live video;
generating an identification code, for the subset of candidate reference frames, having a multiple-bit format that includes a first bit value to indicate a corresponding candidate reference frame is a reference frame for an input frame from the live video and a second bit value to indicate the corresponding candidate reference frame is not the reference frame for the input frame from the live video, and, when a bit of the identification code for a first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is one of a forward reference frame and a backward reference frame for the input frame from the live video, an immediately following bit of the identification code being set to the first bit value indicates the first candidate reference frame is also another of the one of the forward reference frame and the backward reference frame for the input frame from the live video;
performing a real time encode of the input frame of the live video based at least in part on the identification code to generate an encoded frame by the content delivery service; and
transmitting the encoded frame from the content delivery service to a viewer device.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein, when the bit of the identification code of the multiple-bit format for the first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is the one of the forward reference frame and the backward reference frame for the input frame from the live video, the immediately following bit of the identification code being set to the second bit value indicates the first candidate reference frame is not the another of the one of the forward reference frame and the backward reference frame for the input frame from the live video.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein, when the bit of the identification code of the multiple-bit format for the first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is the one of the forward reference frame and the backward reference frame for the input frame from the live video, and the immediately following bit of the identification code is set to the second bit value to indicate the first candidate reference frame is not the another of the one of the forward reference frame and the backward reference frame for the input frame from the live video, a next bit of the identification code being set to the first bit value indicates a second candidate reference frame is the another of the one of the forward reference frame and the backward reference frame for the input frame from the live video.

### Claim 4

A computer-implemented method comprising:
receiving a video at a content delivery service;
determining a subset of candidate reference frames from a plurality of frames received of the video;
generating an identification code, for the subset of candidate reference frames, having a multiple-bit format that includes a first bit value to indicate a corresponding candidate reference frame is a reference frame for an input frame from the video and a second bit value to indicate the corresponding candidate reference frame is not the reference frame for the input frame from the video, and, when a bit of the identification code for a first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is a first reference frame for the input frame from the video, an immediately following bit of the identification code being set to the first bit value indicates the first candidate reference frame is also a second reference frame for the input frame from the video;
performing an encode of the input frame of the video based at least in part on the identification code to generate an encoded frame by the content delivery service; and
transmitting the encoded frame from the content delivery service to a viewer device.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein, when the bit of the identification code of the multiple-bit format for the first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is the first reference frame for the input frame from the video, the immediately following bit of the identification code being set to the second bit value indicates the first candidate reference frame is not another reference frame for the input frame from the video.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, wherein, when the bit of the identification code of the multiple-bit format for the first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is the first reference frame for the input frame from the video, and the immediately following bit of the identification code is set to the second bit value to indicate the first candidate reference frame is not another reference frame for the input frame from the video, a next bit of the identification code being set to the first bit value indicates a second candidate reference frame is the second reference frame for the input frame from the video.

### Claim 7 (depends on 5)

The computer-implemented method of claim 5, wherein, when the bit of the identification code of the multiple-bit format for the first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is the first reference frame for the input frame from the video, and the immediately following bit of the identification code is set to the second bit value to indicate the first candidate reference frame is not another reference frame for the input frame from the video, a next immediately following bit of the identification code being set to the first bit value indicates a second immediately following candidate reference frame is the second reference frame for the input frame from the video.

### Claim 8 (depends on 4)

The computer-implemented method of claim 4, wherein a coding standard for the encode only allows two reference frames for the input frame.

### Claim 9 (depends on 4)

The computer-implemented method of claim 4, wherein the performing the encode comprises a compound prediction for the input frame using the first candidate reference frame as both the first reference frame and the second reference frame.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein the compound prediction comprises performing a first prediction on a first block of the first candidate reference frame and a second prediction on a second different block of the first candidate reference frame.

### Claim 11 (depends on 9)

The computer-implemented method of claim 9, further comprising reading a header of the video to determine a number of candidate reference frames which are allowed to be used as a same reference frame for the compound prediction.

### Claim 12 (depends on 4)

The computer-implemented method of claim 4, wherein the subset of candidate reference frames are decoded frames from the encode.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, wherein the decoded frames are from a decoded frame buffer of an encoder performing the encode.

### Claim 14 (depends on 12)

The computer-implemented method of claim 12, further comprising replacing at least one of the subset of candidate reference frames with a decoded version of the encoded frame.

### Claim 15 (depends on 4)

The computer-implemented method of claim 4, wherein the identification code is according to the multiple-bit format when in a first mode, and the identification code is according to a second multiple-bit format when in a second mode, and the second multiple-bit format includes, when a bit of the identification code for the first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is the first reference frame for the input frame from the video, an immediately following bit of the identification code being set to the first bit value indicates a second candidate reference frame is the second reference frame for the input frame from the video.

### Claim 16

A computer-implemented method comprising:
receiving a bitstream of a video by a media player;
decoding a plurality of frames received in the bitstream by a decoder of the media player;
determining a subset of candidate reference frames from the plurality of decoded frames received in the bitstream by the media player;
receiving an identification code in the bitstream by the media player, for the subset of candidate reference frames, having a multiple-bit format that includes a first bit value to indicate a corresponding candidate reference frame is a reference frame for an input frame from the video and a second bit value to indicate the corresponding candidate reference frame is not the reference frame for the input frame from the video, and, when a bit of the identification code for a first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is a first reference frame for the input frame from the video, an immediately following bit of the identification code being set to the first bit value indicates the first candidate reference frame is also a second reference frame for the input frame from the video;
decoding the bitstream based at least in part on the identification code by the decoder of the media player to generate a decoded frame; and
displaying the decoded frame on a display coupled to the media player.

### Claim 17 (depends on 16)

The computer-implemented method of claim 16, wherein, when the bit of the identification code of the multiple-bit format for the first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is the first reference frame for the decoded frame from the video, the immediately following bit of the identification code being set to the second bit value indicates the first candidate reference frame is not another reference frame for the decoded frame.

### Claim 18 (depends on 17)

The computer-implemented method of claim 17, wherein, when the bit of the identification code of the multiple-bit format for the first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is the first reference frame for the decoded frame from the video, and the immediately following bit of the identification code is set to the second bit value to indicate the first candidate reference frame is not another reference frame for the decoded frame, a next bit of the identification code being set to the first bit value indicates a second candidate reference frame is the second reference frame for the decoded frame.

### Claim 19 (depends on 16)

The computer-implemented method of claim 16, wherein the decoding the bitstream to generate the decoded frame comprises a compound prediction using the first candidate reference frame as both the first reference frame and the second reference frame.

### Claim 20 (depends on 16)

The computer-implemented method of claim 16, wherein the first candidate reference frame is from a decoded frame buffer of the decoder of the media player performing the decode.

### Claim 21 (depends on 16)

The computer-implemented method of claim 16, wherein the identification code is according to the multiple-bit format when in a first mode, and the identification code is according to a second multiple-bit format when in a second mode, and the second multiple-bit format includes, when a bit of the identification code for the first candidate reference frame is set to the first bit value to indicate the first candidate reference frame is the first reference frame for the decoded frame, an immediately following bit of the identification code being set to the first bit value indicates a second candidate reference frame is the second reference frame for the decoded frame.
