# US 10,313,685 B2 · Video coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,313,685 B2
- Document ID: US10313685B2
- Status: Granted
- Kind: B2
- Application number: 14/847,318
- Filed: 2015-09-08
- Published: 2017-03-09
- Granted: 2019-06-04
- Earliest claimed: 2015-09-08
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Ming-Chieh Lee; Amy Lu; Pontus Carlsson; Mattias Dan Nilsson; Sergey Sablin; Sergey Silkin; David Yuheng Zhao; Magnus Hemmendorff; Sergei Nikiforov
- Web page: https://patentgrep.com/patent/US10313685B2

## CPC

- H04N 19/176: the region being a block, e.g. a macroblock
- H04L 1/0014: {by adapting the source coding}
- H04L 1/1635: {Cumulative acknowledgement, i.e. the acknowledgement message applying to all previous messages}
- H04L 1/1835: {Buffer management}
- H04L 1/1838: {for semi-reliable protocols, e.g. for less sensitive applications such as streaming video (buffer level management for video bitstream receiver H04N21/44004)}
- H04L 1/1877: {for semi-reliable protocols, e.g. for less sensitive applications like streaming video (buffer level management for video bitstream control arrangements H04N21/44004)}
- H04L 12/1868: {Measures taken after transmission, e.g. acknowledgments}
- H04L 65/70: Media network packetisation
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/122: Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
- H04N 19/164: Feedback from the receiver or from the transmission channel
- H04N 19/172: the region being a picture, frame or field
- H04N 19/174: the region being a slice, e.g. a line of blocks or a group of blocks
- H04N 19/182: the unit being a pixel
- H04N 19/573: Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
- H04N 19/58: Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one (H04N19/23 takes precedence)
- H04N 19/65: using error resilience
- H04N 21/6375: for requesting retransmission {, e.g. of data packets lost or corrupted during transmission from server}(ARQ protocols H04L1/18; implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP] H04L69/16)
- H04N 21/6405: Multicasting {(data broadcast and multicast in packet switching networks H04L12/18)}
- H04L 2001/0093: {Point-to-multipoint}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A transmitting device for generating a plurality of encoded portions of a video to be transmitted to a receiving device over a network configured to: receive an error message over a feedback channel from the receiving device indicating at least one of said plurality of encoded portions that has been lost at the receiving device; encode a recovery portion responsive to said receiving said error message; and transmit said recovery portion to the receiving device over said network; wherein said error message includes information pertaining to a decoded portion successfully decoded at the receiving device and said recovery portion is encoded relative to said decoded portion.

## Claims

### Claim 1

A transmitting device for generating a plurality of encoded frames of a video to be transmitted to a plurality of receiving devices over a network, the transmitting device including:
a processor; and
a memory including instructions configured to be executed by the processor, the instructions configuring the processor to:
receive multiple error messages over respective feedback channels from multiple respective receiving devices of the plurality of receiving devices, each error message indicating a respective current one of said plurality of encoded frames that has not been received or has been rejected as corrupted at the respective receiving device and an indication of a last known good frame decoded by the respective receiving device;
select an earliest last known good frame from the multiple error messages;
select a latest current encoded frame from the multiple error messages;
encode a recovery frame based on the earliest last known good frame and the latest current frame; and
transmit said recovery frame to the plurality of receiving devices over said network.

### Claim 2 (depends on 1)

The transmitting device of claim 1, wherein at least one of the plurality of encoded frames is marked for storage at the plurality of receiving devices and the plurality of receiving devices are configured to decode and store said at least one of the plurality of encoded frames.

### Claim 3 (depends on 2)

The transmitting device of claim 2, wherein the at least one of the plurality of encoded frames is stored in respective buffers at the plurality of receiving devices.

### Claim 4 (depends on 1)

The transmitting device of claim 1, wherein each of said multiple error messages indicates that the current one of said plurality of encoded frames has not been both received and decoded at the receiving device due to packet loss over the network.

### Claim 5 (depends on 1)

The transmitting device of claim 1, wherein said video is a live video call.

### Claim 6 (depends on 1)

The transmitting device of claim 1, wherein said video is a one-way multicast stream.

### Claim 7 (depends on 1)

The transmitting device of claim 1, wherein said network is the internet.

### Claim 8 (depends on 1)

The transmitting device of claim 1, wherein the transmitting device is arranged to transmit the recovery frames periodically.

### Claim 9

A receiving device for receiving a plurality of encoded multicast frames of a video from a transmitting device over a network, the receiving device including:
a processor; and
a memory including instructions configured to be executed by the processor, the instructions configuring the processor to:
generate an error message indicating at least one erroneous multicast frame of said plurality of multicast frames that has not been both received and decoded at the receiving device and indicating a last known good multicast frame that has been both received and decoded at the receiving device;
transmit said error message over a feedback channel to the transmitting device;
receive a recovery multicast frame from the transmitting device over said network;
wherein:
said recovery multicast frame includes information pertaining to a previously decoded multicast frame and information pertaining to a current multicast frame to be decoded based on the previously decoded multicast frame,
the receiving device is one receiving device of a plurality of receiving devices configured to receive the plurality of multicast video frames; and
the previously decoded multicast frame is an earliest last known good frame, and the current multicast frame is a latest erroneous multicast frame derived from respective error messages generated by the plurality of receiving devices.

### Claim 10 (depends on 9)

The receiving device of claim 9, wherein at least one of the plurality of encoded multicast frames is marked for storage at the receiving device and said previously decoded multicast frame is decoded from the at least one of the plurality of encoded multicast frames marked for storage.

### Claim 11 (depends on 10)

The receiving device of claim 10, wherein the at least one of the plurality of encoded multicast frames is stored in a buffer at the receiving device.

### Claim 12 (depends on 9)

The receiving device of claim 9, wherein said error message indicates at least one of said plurality of encoded multicast frames that has not been both received and decoded at the receiving device due to packet loss over the network.

### Claim 13 (depends on 9)

The receiving device of claim 9, wherein said video is a live video call.

### Claim 14 (depends on 9)

The receiving device of claim 9, wherein said network is the internet.

### Claim 15

A method comprising:
receiving a plurality of error messages over a respective plurality of feedback channels from a respective plurality of receiving devices, each error message including an indication of a respective current frame of a multicast video that has not been received or has been rejected as corrupted at the respective receiving device and an indication of a last known good frame of the multicast video that has been decoded by the respective receiving device;
selecting an earliest last known good frame of the multicast video from the plurality of error messages;
selecting a latest current frame of the multicast video from the plurality of error messages;
encoding a recovery frame of the multicast video based on the earliest last known good frame and the latest current frame; and
transmitting said recovery frame of the multicast video to the plurality of receiving devices.