# US 8,279,260 B2 · System and method for a conference server architecture for low delay and distributed conferencing applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,279,260 B2
- Document ID: US8279260B2
- Status: Granted
- Kind: B2
- Application number: 12/539,501
- Filed: 2009-08-11
- Published: 2009-12-03
- Granted: 2012-10-02
- Earliest claimed: 2005-07-20
- Assignees: Vidyo, Inc.
- Inventors: Reha Civanlar; Alexandros Eleftheriadis; Ofer Shapiro
- Web page: https://patentgrep.com/patent/US8279260B2

## CPC

- H04L 12/4604: {LAN interconnection over a backbone network, e.g. Internet, Frame Relay}
- H04L 47/10: Flow control; Congestion control
- H04L 47/2441: relying on flow classification, e.g. using integrated services [IntServ]
- H04L 47/70: Admission control; Resource allocation
- H04L 47/724: at intermediate nodes, e.g. resource reservation protocol [RSVP]
- H04L 47/801: {Real time traffic}
- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- H04L 65/611: for multicast or broadcast (systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06)
- H04L 65/80: Responding to QoS
- H04L 67/5683: Storage of data provided by user terminals, i.e. reverse caching
- H04N 7/152: {Multipoint control units therefor}
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 21/2662: Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
- H04N 21/6405: Multicasting {(data broadcast and multicast in packet switching networks H04L12/18)}
- H04N 21/6408: Unicasting
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems and methods for conducting a multi-endpoint video signal conference are provided. Conferencing endpoints are linked by pairs of a reliable and a less reliable communication channel. Conference video signals are scaleable coded in base layer and enhancement layers format. Video signal base layers, which correspond to a minimum picture quality, are communicated over reliable channels. The video signal enhancements layers may be communicated over the less reliable channels. A conference server mediates the switching of video layer information from transmitting endpoints to receiving endpoints without any intermediate coding or re-coding operations. The video conference can be integrated with an audio conference using either scalable coded audio signals or non-scaleable coded audio signals.

## Claims

### Claim 1

A multi-endpoint video signal conferencing system for communicating video signals to at least one receiving endpoint over at least one communication channel, wherein the video signals are scalably coded into layers including a base layer and one or more enhancement layers, the conferencing system comprising:
a scalable video coding server (SVCS) adapted to be linked to the at least one receiving endpoint by the at least one communication channel,
wherein the at least one communication channel offers improved quality of service; and
wherein the SVCS is configured to receive and selectively forward a video signal layer to the at least one receiving endpoint over the at least one communication channel.

### Claim 2 (depends on 1)

The conferencing system of claim 1 wherein the SVCS is further configured to provide at least one of continuous presence, personalized layout, rate matching, error localization, and random entry features to the at least one receiving endpoint by selectively multiplexing and forwarding video signal layers thereto.

### Claim 3 (depends on 2)

The conferencing system of claim 2 wherein the SVCS is further configured to selectively multiplex and forward video signal layers having two or more signal characteristics to the at least one receivingendpoint, the signal characteristics including different resolution, bit rate, quality, and frame rate characteristics.

### Claim 4 (depends on 1)

The conferencing system of claim 1, wherein the SVCS is further configured to process audio signals in addition to processing video signals.

### Claim 5 (depends on 1)

The conferencing system of claim 1, wherein the SVCS is further configured to provide at least one of session network border control, media proxy, firewall, and network address translation functions.

### Claim 6 (depends on 1)

The conferencing system of claim 1, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to proactively repeat transmission of at least the base layer packets.

### Claim 7 (depends on 1)

The conferencing system of claim 1, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to transmit at least the base layer packets with forward error correction.

### Claim 8 (depends on 1)

The conferencing system of claim 1, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to cache at least the base layer packets and to retransmit cached packets upon request.

### Claim 9

A multi-endpoint video signal conferencing system for communicating video signals with at least one transmitting endpoint over at least one communication channel, wherein video signals are scalably coded into layers including a base layer and one or more enhancement layers, the conferencing system comprising:
a scalable video coding server (SVCS) adapted to be linked to the at least one transmitting endpoint by the at least one communication channel and to receive one or more video signal layers therefrom,
wherein the at least one communication channel offers improved quality of service; and
wherein the SVCS is configured to selectively forward the one or more video signal layers received from the transmitting endpoint over the at least one communication channel.

### Claim 10 (depends on 9)

The conferencing system of claim 9 wherein the SVCS is further configured to provide at least one of continuous presence, personalized layout, rate matching, error localization, and random entry featuresfor the at least one transmitting endpoint by selectively multiplexing and forwarding video signal layers received therefrom.

### Claim 11 (depends on 9)

The conferencing system of claim 9, wherein the SVCS is further configured to process audio signals in addition to processing video signals.

### Claim 12 (depends on 9)

The conferencing system of claim 9, wherein the SVCS is further configured to provide at least one of session network border control, media proxy, firewall, and network address translation functions.

### Claim 13 (depends on 9)

The conferencing system of claim 9, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to proactively repeat transmission of at least the base layer.

### Claim 14 (depends on 9)

The conferencing system of claim 9, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to receive packets with forward error correction.

### Claim 15 (depends on 9)

The conferencing system of claim 9, wherein the improved quality of service of the communication channel comprises the SVCS being furtherconfigured to receive retransmitted cached packets upon request.

### Claim 16

A multi-endpoint audio signal conferencing system for communicating audio signals to at least one receiving endpoint over at least one communication channel, wherein the audio signals are coded in components such that multiple qualities can be derived from the bitstream in the coded domain, the conferencing system comprising:
a scaleable audio coding server (SACS) adapted to be linked to the at least one receiving endpoint in an audio conference by the at least one communication channel,
wherein the SACS is configured to receive and selectively forward an audio signal component of the audio signals to the at least one receiving endpoint over the at least one communication channel.

### Claim 17 (depends on 16)

The conferencing system of claim 16, wherein the audio signal is scalably coded into layered components comprising a base layer and one or more enhancement layers.

### Claim 18 (depends on 16)

The conferencing system of claim 16, wherein the at least one communication channel offers improved quality of service.

### Claim 19 (depends on 18)

The conferencing system of claim 18, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to proactively repeat transmission of the at least one audio signal component.

### Claim 20 (depends on 18)

The conferencing system of claim 18, wherein the improved quality of service of the communication comprises the SVCS being furtherconfigured to transmit the at least one audio signal component with forward error correction.

### Claim 21 (depends on 18)

The conferencing system of claim 18, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to cache data for the at least one audio signal component and to retransmit cached data upon request.

### Claim 22 (depends on 16)

The conferencing system of claim 16, wherein the audio signals are associated with signal strength indicators.

### Claim 23 (depends on 22)

The conferencing system of claim 22, wherein the SACS is further configured to:
forward all quality components of a strongest received audio signal to all participants except an originating participant
forward less than all quality components of a number of less strong received audio signals, if any, to all participants except an originating participant; and
forward no quality components of any remaining less strong received audio signals.

### Claim 24 (depends on 16)

The conferencing system of claim 16, wherein the audio signals are forwarded to the at least one receiving endpoint by the SACS for mixing.

### Claim 25 (depends on 16)

The conferencing system of claim 16, wherein the SACS is further configured to cache received audio components and to forward the cached components to a first receiver when the receiver needs to commence decoding of the audio signal at a specific quality level if such quality level was not previously received.

### Claim 26 (depends on 16)

The conferencing system of claim 16, wherein the at least one communication channel comprises a packet-based channel, and wherein the SACS is further configured to aggregate a number of audio packets intended for a receiver into a combined packet, and to forward the combined packet to the receiver.

### Claim 27 (depends on 17)

The conferencing system of claim 17, wherein the audio signals are associated with one or more signal strength indicators, and wherein the SACS is further configured to forward only base layer information when one of the one or more signal strength indicators exceeds a first threshold, and to forward base layer and enhancement layer information when one of the one or more signal strength indicators exceeds a second threshold.

### Claim 28 (depends on 16)

The conferencing system of claim 16 wherein the SACS is further configured to selectively multiplex and forward audio signal components having two or more signal characteristics to the at least one receiving endpoint, the characteristics including different sampling rate, bit rate, quality, and number of audio signal channels.

### Claim 29 (depends on 16)

The conferencing system of claim 16 wherein the SACS is further configured to forward enhancement signal layers to receiving endpoints in priority according to a conferencing system priority policy, which assigns priority to receiving endpoints.

### Claim 30 (depends on 16)

The conferencing system of claim 16 wherein the SACS is further configured to process video signals.

### Claim 31 (depends on 16)

The conferencing system of claim 16 wherein the SACS is further configured to provide at least one of a session border control, media proxy, firewall, and network address translation functions.

### Claim 32 (depends on 16)

The conferencing system of claim 16, wherein the multiple quality components of the coded audio signals comprise independently decodable encodings of said audio signals.

### Claim 33 (depends on 16)

The conferencing system of claim 16, wherein the audio signals are associated with one or more signal strength indicators, and wherein the SACS is further configured to forward one quality component when one of the one or more signal strength indicators exceeds a first threshold, and to forward additional quality components when one of the one or more signal strength indicators exceeds a second threshold.

### Claim 34 (depends on 16)

The conferencing system of claim 16, wherein the audio signals are associated with one or more signal strength indicators that are computed at the SACS.

### Claim 35

A multi-endpoint audio signal conferencing system for communicating audio signals with at least one transmitting endpoint over at least one communication channel, wherein the audio signals are coded in components such that multiple qualities can be derived from the bitstream in the coded domain, the conferencing system comprising:
a scaleable audio coding server (SACS) adapted to be linked to at least one transmitting endpoint in an audio conference by the at least one communication channel,
wherein the SACS is configured to receive and selectively forward an audio signal component of the audio signals received from the at least one transmitting endpoint overthe at least one communication channel.

### Claim 36 (depends on 35)

The conferencing system of claim 35, wherein the audio signal is scalably coded into layered components comprising a base layer and one or more enhancement layers.

### Claim 37 (depends on 35)

The conferencing system of claim 35, wherein at the least one communication channel offers improved quality of service.

### Claim 38 (depends on 37)

The conferencing system of claim 37, wherein the improved quality of service of the communication channel comprises the SACS being further configured to receive a proactively repeated transmission of at least one audio signal component from the at least one transmitting endpoint.

### Claim 39 (depends on 37)

The conferencing system of claim 37, wherein the improved quality of service of the communication channel comprises the SACS being further configured to receive transmit audio signal components with forward error correction.

### Claim 40 (depends on 37)

The conferencing system of claim 37, wherein the improved quality of service of the communication comprises the SACS being further configured to receive retransmited cached data upon request.

### Claim 41 (depends on 35)

The conferencing system of claim 35, wherein the audio signals are associated with one or more signal strength indicators.

### Claim 42 (depends on 41)

The conferencing system of claim 41, wherein the SACS is further configured to:
forward all quality components of a strongest received audio signal to all participants except an originating participant; forward less than all quality components of a number of less strong received audio signals, if any, to all participants except the originating participant; and
forward no quality components of any remaining less strong received audio signals.

### Claim 43 (depends on 35)

The conferencing system of claim 35, wherein the at least one communication channel comprises a packet-based channel and wherein the SACS is further configured to aggregate a number of audio packets into one combined packet, and to forward the combined packet.

### Claim 44 (depends on 36)

The conferencing system of claim 36, wherein the received audio signals are associated with one or more signal strength indicators, and wherein the SACS is further configured to forward only base layer information when one of the one or more signal strength indicators exceeds a first threshold, and to forward base layer and enhancement layer information when one of the one or more signal strength indicators exceeds a second threshold.

### Claim 45 (depends on 36)

The conferencing system of claim 36, wherein the audio signals are associated with one or more signal strength indicators, and wherein the one or more transmitting endpoints are configured to transmit only base layer information when one of the one or more signal strength indicators exceeds a first threshold, and to transmit base layer and enhancement layer information only when one of the one or more signal strength indicators exceeds a second threshold.

### Claim 46 (depends on 35)

The conferencing system of claim 35 wherein the SACS is configured to selectively multiplex and forward audio signal components having two or more signal characteristics, the characteristics including at least one of different sampling rate, bit rate, quality, and number of audio signal channels.

### Claim 47 (depends on 35)

The conferencing system of claim 35 wherein the SACS is further configured to process video signal in addition to processing audio signals.

### Claim 48 (depends on 35)

The conferencing system of claim 35 wherein the SACS is further configured to provide at least one of a session border control, media proxy, firewall, and network address translation functions.

### Claim 49 (depends on 35)

The conferencing system of claim 35, wherein the multiple quality components of the coded audio signals each comprise independently decodable encodings of the said audio signals.

### Claim 50 (depends on 35)

The conferencing system of claim 35, wherein the audio signals received from transmitting endpoints are associated with signal strength indicators, and wherein the SACS is further configured to forward only base layer information when one of the one or more signal strength indicators exceeds a first threshold, and to forward base layer and enhancement layer information when one of the one or more signal strength indicators exceeds a second threshold.

### Claim 51 (depends on 35)

The conferencing system of claim 35, wherein the audio signals are associated with signal one or more strength indicators, and wherein the at least one transmitting endpoint is configured to transmit only base layer information when one of the one or more signal strength indicators exceeds a first threshold, and to transmit base layer and enhancement layer information only when one of the one or more signal strength indicators exceeds a second threshold.

### Claim 52 (depends on 35)

The conferencing system of claim 35, wherein the audio signals received from the at least one transmitting endpoint is associated with one or more signal strength indicators that are computed at the SACS.

### Claim 53 (depends on 35)

The conferencing system of claim 35, wherein the audio signals received from the at least one transmitting endpoint is are associated with signal strength indicators that are computed at the transmitting endpoints.

### Claim 54

An apparatus for multi-endpoint video signal conferencing over an electronic communications network, the network having communication channels linking the conferencing endpoints with at least one communication channel linking an endpoint having a superior quality of service compared to other channels, said apparatus comprising:
one or more computer-readable storage media; and
software embodied in the one or more computer-readable storage media that is operable when executed to:
obtain a scalable video coded video signal, wherein the video signal is coded in a layered format including a base layer and one or more enhancement layers
select at least one layer of the coded video signal; and
forward information in the selected layer to the endpoint over the communication channel having superior quality of service.

### Claim 55

An apparatus for multi-endpoint audio signal conferencing over an electronic communications network, the network having communication channels linking the conferencing endpoints (i.e., the transmitting and receiving endpoints), the apparatus comprising:
one or more computer-readable storage media; and
software embodied in the one or more computer-readable storage media that is operable when executed to:
obtain audio signals that are coded in component bitstreams such that multiple qualities can be derived from a bitstream in the coded domain; and
selectively forward audio signal components received from transmitting endpoints over their respective linking communication channels to receiving endpoints over their respective linking communication channels.

### Claim 56

A apparatus for multi-endpoint video signal conferencing over an electronic communications network, the network having communication channels linking the conferencing endpoints, the apparatus comprising:
one or more computer-readable storage media; and
software embodied in the one or more computer-readable storage media that is operable when executed to:
obtain a scalable video coded video signal, wherein the video signal is coded in a layered format including at least a base layer and one or more enhancement layers;
select at least one layer of the coded video signal; and
selectively multiplex and forward video signal layers to the conferencing endpoints over the linking communication channels, thereby providing at least one of continuous presence, personalized layout, rate matching, error localization, and random entry features to the conferencing endpoints.
