# US 10,313,698 B2 · Syntax and semantics for buffering information to simplify video splicing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,313,698 B2
- Document ID: US10313698B2
- Status: Granted
- Kind: B2
- Application number: 15/601,615
- Filed: 2017-05-22
- Published: 2017-09-07
- Granted: 2019-06-04
- Earliest claimed: 2013-01-07
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Gary J. Sullivan; Lihua Zhu
- Web page: https://patentgrep.com/patent/US10313698B2

## CPC

- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement}
- H04N 21/44016: {involving splicing one content stream with another content stream, e.g. for substituting a video clip}
- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- H04N 7/15: Conference systems
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N 19/573: Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
- H04N 19/58: Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one (H04N19/23 takes precedence)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Innovations in syntax and semantics of coded picture buffer removal delay (“CPBRD”) values potentially simplify splicing operations. For example, a video encoder sets a CPBRD value for a current picture that indicates an increment value relative to a nominal coded picture buffer removal time of a preceding picture in decoding order, regardless of whether the preceding picture has a buffering period SEI message. The encoder can signal the CPBRD value according to a single-value approach in which a flag indicates how to interpret the CPBRD value, according to a two-value approach in which another CPBRD value (having a different interpretation) is also signaled, or according to a two-value approach that uses a flag and a delta value. A corresponding video decoder receives and parses the CPBRD value for the current picture. A splicing tool can perform simple concatenation operations to splice bitstreams using the CPBRD value for the current picture.

## Claims

### Claim 1

In a computing device that implements a video encoder, a method comprising:
encoding video for a video sequence, including:
setting a coded picture buffer removal delay (“CPBRD”) delta value for a given access unit for a current picture of the video sequence, the current picture having a buffering period SEI message associated with the current picture;
setting a value of a flag for the given access unit, wherein:
if the value of the flag is a first value, a CPBRD value in a picture timing SEI message for the given access unit indicates an increment value specifying a nominal coded picture buffer (“CPB”) removal time of the current picture relative to a nominal CPB removal time of a first preceding picture in decoding order, the first preceding picture having a buffering period SEI message associated with the first preceding picture; and
if the value of the flag is a second value, the nominal CPB removal time of the current picture is indicated, by the CPBRD delta value, as an increment value relative to a nominal CPB removal time of a second preceding picture in decoding order; and
signaling the CPBRD delta value and the value of the flag for the given access unit in the buffering period SEI message associated with the current picture; and
outputting the encoded video.

### Claim 2 (depends on 1)

The method of claim 1, wherein the CPBRD delta value is signaled in the buffering period SEI message associated with the current picture as a syntax element representing the CPBRD delta value minus 1.

### Claim 3 (depends on 1)

The method of claim 1, wherein the second preceding picture immediately precedes the current picture in decoding order.

### Claim 4 (depends on 1)

The method of claim 1, wherein the encoding further comprises:
setting the CPBRD value for the given access unit; and
signaling the CPBRD value for the given access unit in the picture timing SEI message as a syntax element representing the CPBRD value minus 1.

### Claim 5 (depends on 1)

The method of claim 1, wherein, if the value of the flag is the first value:
if the second preceding picture lacks a buffering period SEI message associated with the second preceding picture, the CPBRD delta value indicates a difference between the CPBRD value for the given access unit and a CPBRD value for the second preceding picture; and
if the second preceding picture has a buffering period SEI message associated with the second preceding picture, the CPBRD delta value equals the CPBRD value for the given access unit.

### Claim 6 (depends on 1)

The method of claim 1, wherein:
the second preceding picture lacks a buffering period SEI message associated with the second preceding picture, and the first preceding picture and the second preceding picture are different pictures; or
the second preceding picture has a buffering period SEI message associated with the second preceding picture, and the first preceding picture is the second preceding picture.

### Claim 7

One or more computer-readable memory or storage devices having stored thereon computer-executable instructions for causing a computing device, when programmed thereby, to perform operations comprising:
encoding video for a video sequence, including:
setting a coded picture buffer removal delay (“CPBRD”) delta value for a given access unit for a current picture of the video sequence, the current picture having a buffering period SEI message associated with the current picture;
setting a value of a flag for the given access unit, wherein:
if the value of the flag is a first value, a CPBRD value in a picture timing SEI message for the given access unit indicates an increment value specifying a nominal coded picture buffer (“CPB”) removal time of the current picture relative to a nominal CPB removal time of a first preceding picture in decoding order, the first preceding picture having a buffering period SEI message associated with the first preceding picture; and
if the value of the flag is a second value, the nominal CPB removal time of the current picture is indicated, by the CPBRD delta value, as an increment value relative to a nominal CPB removal time of a second preceding picture in decoding order; and
signaling the CPBRD delta value and the value of the flag for the given access unit in the buffering period SEI message associated with the current picture; and
outputting the encoded video.

### Claim 8 (depends on 7)

The one or more computer-readable memory or storage devices of claim 7, wherein the CPBRD delta value is signaled in the buffering period SEI message associated with the current picture as a syntax element representing the CPBRD delta value minus 1.

### Claim 9 (depends on 7)

The one or more computer-readable memory or storage devices of claim 7, wherein the second preceding picture immediately precedes the current picture in decoding order.

### Claim 10 (depends on 7)

The one or more computer-readable memory or storage devices of claim 7, wherein the encoding further comprises:
setting the CPBRD value for the given access unit; and
signaling the CPBRD value for the given access unit in the picture timing SEI message as a syntax element representing the CPBRD value minus 1.

### Claim 11 (depends on 7)

The one or more computer-readable memory or storage devices of claim 7, wherein, if the value of the flag is the first value:
if the second preceding picture lacks a buffering period SEI message associated with the second preceding picture, the CPBRD delta value indicates a difference between the CPBRD value for the given access unit and a CPBRD value for the second preceding picture; and
if the second preceding picture has a buffering period SEI message associated with the second preceding picture, the CPBRD delta value equals the CPBRD value for the given access unit.

### Claim 12 (depends on 7)

The one or more computer-readable memory or storage devices of claim 7, wherein:
the second preceding picture lacks a buffering period SEI message associated with the second preceding picture, and the first preceding picture and the second preceding picture are different pictures; or
the second preceding picture has a buffering period SEI message associated with the second preceding picture, and the first preceding picture is the second preceding picture.

### Claim 13

A computing device comprising:
a buffer configured to store encoded video for a video sequence; and
a video decoder configured to perform operations including:
receiving a coded picture buffer removal delay (“CPBRD”) delta value for a given access unit for a current picture of the video sequence, the current picture having a buffering period SEI message associated with the current picture; and
receiving a value of a flag for the given access unit, wherein:
if the value of the flag is a first value, a CPBRD value in a picture timing SEI message for the given access unit indicates an increment value specifying a nominal coded picture buffer (“CPB”) removal time of the current picture relative to a nominal CPB removal time of a first preceding picture in decoding order, the first preceding picture having a buffering period SEI message associated with the first preceding picture; and
if the value of the flag is a second value, the nominal CPB removal time of the current picture is indicated, by the CPBRD delta value, as an increment value relative to a nominal CPB removal time of a second preceding picture in decoding order; and
decoding the encoded video, including parsing the CPBRD delta value and the value of the flag for the given access unit from the buffering period SEI message associated with the current picture.

### Claim 14 (depends on 13)

The computing device of claim 13, wherein the CPBRD delta value is signaled in the buffering period SEI message associated with the current picture as a syntax element representing the CPBRD delta value minus 1.

### Claim 15 (depends on 13)

The computing device of claim 13, wherein the operations further include:
receiving the CPBRD value for the given access unit, signaled in the picture timing SEI message for the given access unit as a syntax element representing the CPBRD value minus 1.

### Claim 16 (depends on 15)

The computing device of claim 15, wherein the operations further include:
detecting loss of data by comparing the CPBRD value for the given access unit to a sum based on a CPBRD value for the second preceding picture and the CPBRD delta value for the given access unit.

### Claim 17 (depends on 13)

The computing device of claim 13, wherein, for a spliced bitstream, the nominal CPB removal time of the current picture can be computed as a sum of (a) the nominal CPB removal time of the second preceding picture and (b) a product of a clock tick value and the CPBRD delta value for the given access unit, constrained by a constraint that ensures that delay is at least as long as it would be if decoding started at the given access unit.

### Claim 18 (depends on 13)

The computing device of claim 13, wherein the operations further include:
based at least in part on the CPBRD delta value for the given access unit, computing the nominal CPB removal time of the current picture.

### Claim 19 (depends on 13)

The computing device of claim 13, wherein, if the value of the flag is the first value:
if the second preceding picture lacks a buffering period SEI message associated with the second preceding picture, the CPBRD delta value indicates a difference between the CPBRD value for the given access unit and a CPBRD value for the second preceding picture; and
if the second preceding picture has a buffering period SEI message associated with the second preceding picture, the CPBRD delta value equals the CPBRD value for the given access unit.

### Claim 20 (depends on 13)

The computing device of claim 13, wherein:
the second preceding picture lacks a buffering period SEI message associated with the second preceding picture, and the first preceding picture and the second preceding picture are different pictures; or
the second preceding picture has a buffering period SEI message associated with the second preceding picture, and the first preceding picture is the second preceding picture.

### Claim 21 (depends on 13)

The computing device of claim 13, wherein the current picture is a random access picture.

### Claim 22 (depends on 13)

The computing device of claim 13, wherein:
the current picture is part of a temporal layer, and the second preceding picture immediately precedes the current picture in decoding order in the temporal layer of the current picture or a lower temporal layer; or
the second preceding picture, among pictures having a temporal identifier of zero, immediately precedes the current picture in decoding order.

### Claim 23 (depends on 1)

The method of claim 1, wherein:
the current picture is part of a temporal layer, and the second preceding picture immediately precedes the current picture in decoding order in the temporal layer of the current picture or a lower temporal layer; or
the second preceding picture, among pictures having a temporal identifier of zero, immediately precedes the current picture in decoding order.

### Claim 24 (depends on 7)

The one or more computer-readable memory or storage device of claim 7, wherein:
the current picture is part of a temporal layer, and the second preceding picture immediately precedes the current picture in decoding order in the temporal layer of the current picture or a lower temporal layer; or
the second preceding picture, among pictures having a temporal identifier of zero, immediately precedes the current picture in decoding order.