# US 2015/0312521 A1 · Split Frame Multistream Encode

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0312521 A1
- Document ID: US20150312521A1
- Status: Published application
- Kind: A1
- Application number: 14/793,936
- Filed: 2015-07-08
- Published: 2015-10-29
- Earliest claimed: 2013-03-15
- Assignees: Cisco Technology, Inc.
- Applicants: Cisco Technology, Inc.
- Inventors: Paul G. Bright-Thomas
- Web page: https://patentgrep.com/patent/US20150312521A1

## CPC

- H04N 7/152: {Multipoint control units therefor}
- H04N 19/00: Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N 19/40: using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
- H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
- H04N 19/88: involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Techniques for video conferencing including receiving bandwidth and/or codec characteristics of a plurality of video conference participants, determining whether or not any of the bandwidth and/or codec characteristics are sufficiently different from others of the bandwidth and/or codec characteristics to warrant different treatment, when one or more of the bandwidth and/or codec characteristics are sufficiently different, grouping video conference participants into at least a first group and a second group according to video conference participants having same or similar bandwidth and/or codec characteristics, and establishing a video conference with at least first and second subconferences to service the first and second groups, respectively, wherein each of the video conference participants receives frames of video in which a first portion of the frames is encoded by a shared encoder, and wherein a second portion of the frames is encoded by different encoders respectively designated for each of the video conference participants.

## Claims

### Claim 1

A method comprising:
receiving bandwidth and/or codec characteristics of a plurality of video conference participants;
determining whether or not any of the bandwidth and/or codec characteristics are sufficiently different from others of the bandwidth and/or codec characteristics to warrant different treatment;
when one or more of the bandwidth and/or codec characteristics are sufficiently different, grouping video conference participants into at least a first group and a second group according to video conference participants having same or similar bandwidth and/or codec characteristics; and
establishing a video conference with at least first and second subconferences to service the first and second groups, respectively,
wherein each of the video conference participants receives frames of video in which a first portion of the frames is encoded by a shared encoder, and wherein a second portion of the frames is encoded by different encoders respectively designated for each of the video conference participants.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining whether or not any of the bandwidth and/or codec characteristics are sufficiently different from others of the bandwidth and/or codec characteristics comprises determining whether the bandwidth and/or codec characteristics are within 10% of each other.

### Claim 3 (depends on 1)

The method of claim 1, further comprising designating a video stream from one of the video conference participants as a primary video stream output by a primary recipient and designating other video streams from other video conference participants as secondary video streams output by secondary recipients of the plurality of recipients; and
decoding each of the video streams into frames of video.

### Claim 4 (depends on 3)

The method of claim 3, further comprising scaling resulting decoded video streams to fit within predetermined areas of a frame of video and/or to change a resolution of the video streams to accommodate a target bit rate.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
composing a frame for one of the recipients other than a primary recipient to include video of the primary recipient in a primary sub-picture portion of the frame and to include video of one or more secondary recipients in a secondary sub-picture portion of the frame,
wherein the frame composed for the one of the recipients does not include video of that recipient.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
composing a respective frame for other recipients not including the one recipient and the primary recipient to include video of the primary recipient in a primary sub-picture portion of the respective frame and to include video of one or more secondary recipients in a secondary sub-picture portion of the respective frame,
wherein the respective frame composed for the other recipients does not include video of that recipient.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
determining a number of secondary recipients to be displayed with a primary recipient within respective frames provided to a plurality of displays of the plurality of the video conference participants; and
based on the determining, setting a boundary within the respective frame for each of the recipients, the boundary separating a primary sub-picture portion of the respective frame that includes video of the primary recipient and a secondary sub-picture portion of the respective frame that includes video of one or more of the secondary recipients.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
including at least one slice representing the first subconference and at least one slice representing the second subconference in one or more packets to respective recipients; and
transmitting the one or more packets to the respective recipients.

### Claim 9 (depends on 8)

The method of claim 8, wherein the at least one slice representing the first subconference and the at least one slice representing the second subconference are included in a payload portion of the one or more packets

### Claim 10

An apparatus comprising:
a network interface unit configured to enable communications via a network;
a memory configured to store logic instructions; and
at least one processor, when executing the logic instructions, is configured to:
receive bandwidth and/or codec characteristics of a plurality of video conference participants;
determine whether or not any of the bandwidth and/or codec characteristics are sufficiently different from others of the bandwidth and/or codec characteristics to warrant different treatment;
when one or more of the bandwidth and/or codec characteristics are sufficiently different, group video conference participants into at least a first group and a second group according to video conference participants having same or similar bandwidth and/or codec characteristics; and
establish a video conference with at least first and second subconferences to service the first and second groups, respectively,
wherein each of the video conference participants receives frames of video in which a first portion of the frames is encoded by a shared encoder, and wherein a second portion of the frames is encoded by different encoders respectively designated for each of the video conference participants.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the processor is configured to determine whether or not any of the bandwidth and/or codec characteristics are sufficiently different from others of the bandwidth and/or codec characteristics by determining whether the bandwidth and/or codec characteristics are within 10% of each other.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein the processor is further configured to designate a video stream from one of the video conference participants as a primary video stream output by a primary recipient and designate other video streams from other video conference participants as secondary video streams output by secondary recipients of the plurality of recipients; and
decode each of the video streams into frames of video.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the processor is further configured to scale resulting decoded video streams to fit within predetermined areas of a frame of video and/or to change a resolution of the video streams to accommodate a target bit rate.

### Claim 14 (depends on 10)

The apparatus of claim 10, wherein the processor is further configured to:
compose a frame for one of the recipients other than a primary recipient to include video of the primary recipient in a primary sub-picture portion of the frame and to include video of one or more secondary recipients in a secondary sub-picture portion of the frame,
wherein the frame composed for the one of the recipients does not include video of that recipient.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the processor is further configured to:
compose a respective frame for other recipients not including the one recipient and the primary recipient to include video of the primary recipient in a primary sub-picture portion of the respective frame and to include video of one or more secondary recipients in a secondary sub-picture portion of the respective frame,
wherein the respective frame composed for the other recipients does not include video of that recipient.

### Claim 16 (depends on 10)

The apparatus of claim 10, wherein the processor is further configured to:
determine a number of secondary recipients to be displayed with a primary recipient within respective frames provided to a plurality of displays of the plurality of the video conference participants; and
based on the determining, set a boundary within the respective frame for each of the recipients, the boundary separating a primary sub-picture portion of the respective frame that includes video of the primary recipient and a secondary sub-picture portion of the respective frame that includes video of one or more of the secondary recipients.

### Claim 17 (depends on 10)

The apparatus of claim 10, wherein the processor is further configured to:
include at least one slice representing the first subconference and at least one slice representing the second subconference in one or more packets to respective recipients; and
transmit the one or more packets to the respective recipients.

### Claim 18

A non-transitory tangible computer readable storage media encoded with instructions that, when executed by at least one processor, is configured to cause the processor to:
receive bandwidth and/or codec characteristics of a plurality of video conference participants;
determine whether or not any of the bandwidth and/or codec characteristics are sufficiently different from others of the bandwidth and/or codec characteristics to warrant different treatment;
when one or more of the bandwidth and/or codec characteristics are sufficiently different, group video conference participants into at least a first group and a second group according to video conference participants having same or similar bandwidth and/or codec characteristics; and
establish a video conference with at least first and second subconferences to service the first and second groups, respectively,
wherein each of the video conference participants receives frames of video in which a first portion of the frames is encoded by a shared encoder, and wherein a second portion of the frames is encoded by different encoders respectively designated for each of the video conference participants.

### Claim 19 (depends on 18)

The non-transitory tangible computer readable storage media of claim 18, further encoded with instructions that, when executed by at least one processor, is further configured to cause the processor to:
determine whether or not any of the bandwidth and/or codec characteristics are sufficiently different from others of the bandwidth and/or codec characteristics by determining whether the bandwidth and/or codec characteristics are within 10% of each other.

### Claim 20 (depends on 18)

The non-transitory tangible computer readable storage media of claim 18, further encoded with instructions that, when executed by at least one processor, is configured to cause the processor to:
designate a video stream from one of the video conference participants as a primary video stream output by a primary recipient and designate other video streams from other video conference participants as secondary video streams output by secondary recipients of the plurality of recipients; and
decode each of the video streams into frames of video.
