# US 2018/0027266 A1 · Video and data multiplexing in an adaptive bitrate server

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0027266 A1
- Document ID: US20180027266A1
- Status: Published application
- Kind: A1
- Application number: 15/655,016
- Filed: 2017-07-20
- Published: 2018-01-25
- Earliest claimed: 2016-07-20
- Assignees: ARRIS Enterprises LLC
- Applicants: ARRIS Enterprises LLC
- Inventors: Mark S. Schmidt; Praveen N. Moorthy; Ajay Luthra; Baozhou Li
- Web page: https://patentgrep.com/patent/US20180027266A1

## CPC

- H04N 21/236: Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream {(multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00)}
- H04N 19/134: characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/23614: {Multiplexing of additional data and video streams (arrangements for simultaneous broadcast of plural pieces of information H04H20/28)}
- H04N 21/238: Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams {(hybrid fiber coaxial [HFC] networks for downstream channel allocation for video distribution H04L12/2801; flow control in packet networks H04L47/10; real-time communication protocols in data switching networks H04L65/00; scheduling or organising the servicing of application requests H04L67/60)}
- H04N 21/2402: {Monitoring of the downstream path of the transmission network, e.g. bandwidth available (traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876)}
- H04N 21/2662: Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
- H04N 21/6118: {involving cable transmission, e.g. using a cable modem}
- H04N 19/14: Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence)
- H04N 19/156: Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method is provided for multiplexing video services with data services in a constrained delivery pipeline. A soft upper bound for the video data bandwidth for a current schedule window BWvideosoft(i) is generated, wherein the soft upper bound for the video data bandwidth such that an average video data bandwidth over a time period T is no greater than a hard upper bound for the video data bandwidth BVvideo. The video data is then multiplexed with the other data according to the soft upper bound for the video data bandwidth BWvideosoft(i).

## Claims

### Claim 1

A method of multiplexing video data having a video data bandwidth and other data, comprising:
generating a soft upper bound for the video data bandwidth for a current schedule window BWvideosoft(i), the soft upper bound for the video data bandwidth such that an average video data bandwidth over a time period T is no greater than a hard upper bound for the video data bandwidth BWvideo; and
multiplexing the video data and the other data according to the soft upper bound for the video data bandwidth BWvideosoft(i).

### Claim 2 (depends on 1)

The method of claim 1, wherein generating a soft upper bound for the video data bandwidth BWvideosoft(i) comprises:
generating a required video data bandwidth BWneed(i) for the current schedule window;
generating an incremental change in video data bandwidth for the current schedule window ΔBW(i); and
generating the soft upper bound for the video data bandwidth BWvideosoft(i) from the required video data bandwidth for the current schedule window BWneed(i) and the incremental change in the video data bandwidth for the current schedule window ΔBW(i).

### Claim 3 (depends on 2)

The method of claim 2, wherein generating the incremental change in the video data bandwidth for the current schedule window ΔBW(i) comprises:
generating a difference between a target video data bandwidth BWvideo and a soft upper bound for the video data bandwidth for a previous schedule window BWvideosoft(i−1); and
scaling the difference between the target video data bandwidth BWvideo and the soft upper bound for the video data bandwidth for the previous schedule window BWvideosoft(i−1); and
generating the incremental change in the video data bandwidth ΔBW(i) from an incremental change in the video data bandwidth for a previous schedule window ΔBW(i−1) and the scaled difference between the target video data bandwidth BWvideo and the soft upper bound for the video data bandwidth for the video data bandwidth for the previous schedule window BWvideosoft(i−1).

### Claim 4 (depends on 3)

The method of claim 3, wherein generating the required video data bandwidth BWneed(i) comprises:
generating a need parameter value for the current schedule window NPV(i);
generating the required video data bandwidth BWneed(i) from the need parameter value for the current schedule window NPV(i) and a quality factor α.

### Claim 5 (depends on 4)

The method of claim 4, wherein generating the required video data bandwidth BWneed(i) from the need parameter value for the current schedule window NPV(i) and a quality factor ∝ comprises:
computing a normalizing function of the need parameter value for the current schedule window ƒ(NPV(i)); and
generating the required video data bandwidth BWneed(i) from the computed normalizing function of the need parameter value for the current schedule window ƒ(NPV(i)) and the quality factor α.

### Claim 6 (depends on 5)

The method of claim 5, wherein the need parameter value for the current schedule window NPV(i) is computed at least in part according to at least one of:
video complexity;
a target device profile; and
service priority level.

### Claim 7 (depends on 6)

The method of claim 6, wherein scaling the difference between the target video data bandwidth BWvideo and the soft upper bound for the video data bandwidth for the previous schedule window BWvideosoft(i−1) comprises:
loop filtering the difference between the target video data bandwidth BWvideo and the soft upper bound for the video data bandwidth for the previous schedule window BWvideosoft(i−1); and
modulating the loop filtered difference between the target video data bandwidth BWvideo and the soft upper bound for the video data bandwidth for the previous schedule window BWvideosoft(i−1).

### Claim 8 (depends on 4)

The method of claim 4, further comprising computing a schedule window quality factor ∝i from the generated soft upper bound for the video data bandwidth for the current schedule window BWvideosoft(i) and the need parameter value for the current schedule window NPV(i).

### Claim 9 (depends on 8)

The method of claim 8, wherein the a schedule window quality factor ∝i is computed as
BW
video
soft

(
i
)
NPV

(
i
)
.

### Claim 10

An apparatus for multiplexing video data having a video data bandwidth and other data, comprising:
a processor;
a memory, communicatively coupled to the processor, the memory storing processor instructions comprising instructions for:
generating a soft upper bound for the video data bandwidth for a current schedule window BWvideosoft(i), the soft upper bound for the video data bandwidth such that an average video data bandwidth over a time period T is no greater than a hard upper bound for the video data bandwidth BVvideo; and
multiplexing the video data and the other data according to the soft upper bound for the video data bandwidth BWvideosoft(i).

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the instructions for generating a soft upper bound for the video data bandwidth BWvideosoft(i) comprise instructions for:
generating a required video data bandwidth BWneed(i) for the current schedule window;
generating an incremental change in video data bandwidth for the current schedule window ΔBW(i);
generating the soft upper bound for the video data bandwidth BWvideosoft(i) from the required video data bandwidth for the current schedule window BWneed(i) and the incremental change in the video data bandwidth for the current schedule window ΔBW(i).

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the instructions for generating the incremental change in the video data bandwidth for the current schedule window ΔBW(i) comprise instructions for:
generating a difference between a target video data bandwidth BWvideo and a soft upper bound for the video data bandwidth for a previous schedule window BWvideosoft(i−1); and
scaling the difference between the target video data bandwidth BWvideo and the soft upper bound for the video data bandwidth for the previous schedule window BWvideosoft(i−1); and
generating the incremental change in the video data bandwidth ΔBW(i) from an incremental change in the video data bandwidth for a previous schedule window ΔBW(i−1) and the scaled difference between the target video data bandwidth BWvideo and the soft upper bound for the video data bandwidth for the video data bandwidth for the previous schedule window BWvideosoft(i−1).

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the instructions for generating the required video data bandwidth BWneed(i) comprise instructions for:
generating a need parameter value for the current schedule window NPV(i);
generating the required video data bandwidth BWneed(i) from the need parameter value for the current schedule window NPV(i) and a quality factor α.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the instructions for generating the required video data bandwidth BWneed(i) from the need parameter value for the current schedule window NPV(i) and a quality factor α comprise instructions for:
computing a normalizing function of the need parameter value for the current schedule window ƒ(NPV(i)); and
generating the required video data bandwidth BWneed(i) from the computed normalizing function of the need parameter value for the current schedule window ƒ(NPV(i)) and the quality factor α.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the need parameter value for the current schedule window NPV(i) is computed at least in part according to at least one of:
video complexity;
a target device profile; and
service priority level.

### Claim 16 (depends on 6)

The apparatus of claim 6, wherein the instructions for scaling the difference between the target video data bandwidth BWvideo and the soft upper bound for the video data bandwidth for the previous schedule window BWvideosoft(i−1) comprise instructions for:
loop filtering the difference between the target video data bandwidth BWvideo and the soft upper bound for the video data bandwidth for the previous schedule window BWvideosoft(i−1); and
modulating the loop filtered difference between the target video data bandwidth BWvideo and the soft upper bound for the video data bandwidth for the previous schedule window BWvideosoft(i−1).

### Claim 17 (depends on 13)

The apparatus of claim 13, wherein the instructions further comprise instructions for computing a schedule window quality factor ∝i from the generated soft upper bound for the video data bandwidth for the current schedule window BWvideosoft(i) and the need parameter value for the current schedule window NPV(i).

### Claim 18 (depends on 8)

The apparatus of claim 8, wherein the a schedule window quality factor αi is computed as
BW
video
soft

(
i
)
NPV

(
i
)
.

### Claim 19

An apparatus for method of multiplexing video data having a video data bandwidth and other data, comprising:
a processor, the processor for generating a soft upper bound for the video data bandwidth for a current schedule window BWvideosoft(i), the soft upper bound for the video data bandwidth such that an average video data bandwidth over a time period T is no greater than a hard upper bound for the video data bandwidth BVvideo; and
a multiplexor for multiplexing the video data and the other data according to the soft upper bound for the video data bandwidth BWvideosoft(i).

### Claim 20 (depends on 1)

The method of claim 1, wherein the processor generates a soft upper bound for the video data bandwidth BWvideosoft(i) by:
generating a required video data bandwidth BWneed(i) for the current schedule window;
generating an incremental change in video data bandwidth for the current schedule window ΔBW(i); and
generating the soft upper bound for the video data bandwidth BWvideosoft(i) from the required video data bandwidth for the current schedule window BWneed(i) and the incremental change in the video data bandwidth for the current schedule window ΔBW(i).
