# US 10,313,759 B1 · Enabling playback and request of partial media fragments

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,313,759 B1
- Document ID: US10313759B1
- Status: Granted
- Kind: B1
- Application number: 15/194,341
- Filed: 2016-06-27
- Granted: 2019-06-04
- Earliest claimed: 2016-06-27
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Charles Benjamin Franklin Waggoner; Marc Joliveau; Lauri Juhani Särkkä
- Web page: https://patentgrep.com/patent/US10313759B1

## CPC

- H04N 21/47217: {for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks (specific graphical features in visual interfaces H04N21/4312)}
- H04L 65/612: for unicast
- H04L 65/752: adapting media to network capabilities
- H04L 65/80: Responding to QoS
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04N 21/2387: Stream processing in response to a playback request from an end-user, e.g. for trick-play
- H04N 21/24: Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests {(monitoring of server performance or load G06F11/34; arrangements for observation, testing or troubleshooting for broadcast or for distribution combined with broadcast H04H20/12)}
- H04N 21/4126: {The peripheral being portable, e.g. PDAs or mobile phones}
- H04N 21/4325: {by playing back content from the storage medium (reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79)}
- H04N 21/44004: {involving video buffer management, e.g. video decoder buffer or video display buffer}
- H04N 21/442: Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk {(arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00)}
- H04N 21/47202: {for requesting content on demand, e.g. video on demand}
- H04N 21/845: Structuring of content, e.g. decomposing content into time segments
- H04N 21/8455: {involving pointers to the content, e.g. pointers to the I-frames of the video stream}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Techniques are described that enable initiating playback of a media fragment before the entire fragment has been downloaded. Also described are techniques that enable a more granular approach to the way in which a client device requests and plays back media fragments.

## Claims

### Claim 1

A computer-program product, comprising one or more non-transitory computer-readable media having computer program instructions stored therein, the computer program instructions being configured such that, when executed by one or more computing devices, the computer program instructions cause the one or more computing devices to:
transmit a request for a video presentation, the video presentation including a plurality of video fragments;
receive a manifest configured to enable the one or more computing devices to request video fragments in the plurality of video fragments;
generate a first request for a video fragment of the video presentation using the manifest associated with the video presentation, the video fragment including a plurality of video frames and a fragment index, the fragment index identifying each of the video frames of the video fragment and including a byte offset for each of the video frames;
transmit the first request to a remote platform specified by the manifest, the remote platform hosting the plurality of fragments of the video presentation;
receive a first portion of the video fragment from the remote platform, the first portion of the video fragment including the fragment index and fewer than all of the video frames of the video fragment;
initiate playback of the video frames of the first portion of the video fragment at the one or more computing devices before the video fragment is completely downloaded to the one or more computing devices;
determine that downloading of the video fragment has terminated without the video fragment being completely downloaded;
identify video frames of the video fragment not included in the first portion of the video fragment using the fragment index;
generate a second request for a remainder portion of the video fragment, the remainder portion of the video fragment including at least some of the video frames of the video fragment not included in the first portion of the video fragment, the request specifying one or more byte ranges corresponding to the video frames of the remainder portion of the video fragment, the one or more byte ranges being derived using one or more of the byte offsets in the fragment index;
transmit the second request to the remote platform;
receive at least some of the video frames of the remainder portion of the video fragment while playback of the video frames of the first portion of the video fragment is ongoing; and
initiate playback of the video frames of the remainder portion of the video fragment at the one or more computing devices after playback of the video frames of the first portion of the video fragment terminates and without rebuffering all of the video fragment.

### Claim 2 (depends on 1)

The computer program product of claim 1, the computer program instructions being further configured to cause the one or more computing devices to:
track a buffer duration of the video presentation that has been downloaded, the buffer duration representing at least some of the first portion of the video fragment; and
initiate playback of the video frames of the first portion of the video fragment only where the buffer duration exceeds a threshold.

### Claim 3 (depends on 1)

The computer program product of claim 1, wherein each of the video frames of the video fragment corresponds to one of a plurality of frame types, each of the frame types being characterized by a set of frame dependencies, the plurality of frame types being characterized by a hierarchy of the frame types, and wherein the video frames of the video fragment are grouped within the video fragment according to the frame types and ordered within the video fragment according to the hierarchy of the frame types, and wherein the video frames of the remainder portion of the video fragment do not include one or more of the frame types.

### Claim 4

A computer-implemented method, comprising:
transmitting, by a client device, a request for media content, the media content including a plurality of media fragments;
receiving, by the client device, manifest data configured to enable the client device to request the media fragments;
transmitting, by the client device, a request for a media fragment, the media fragment including a plurality of frames;
receiving, by the client device, a portion of the media fragment, the portion of the media fragment including fewer than all of the frames of the media fragment;
tracking a received playback duration of the media content, the received playback duration of the media content including a first playback duration corresponding to the portion of the media fragment, the first playback duration being determined using a fragment index associated with the media fragment and a number of bytes for the portion of the media fragment, wherein the fragment index identifies frames, a reference hierarchy, and frame offsets for the media fragment;
determining that the received playback duration exceeds a threshold;
responsive to determining that the received playback duration exceeds a threshold, initiating, by the client device, playback of the frames of the portion of the media fragment without the media fragment being completely downloaded by the client device;
determining, by the client device, that downloading of the media fragment has terminated without the media fragment being completely downloaded;
identifying, by the client device, frames of the media fragment not included in the portion of the media fragment; and
transmitting, by the client device, a second request specifying a remainder portion of the media fragment that is less than the entire media fragment, the remainder portion of the media fragment including at least some of the frames of the media fragment identified as not being included in the portion of the media fragment.

### Claim 5 (depends on 4)

The method of claim 4 wherein the media fragment is characterized by a fragment duration, and wherein the threshold is below the fragment duration of the media fragment.

### Claim 6 (depends on 4)

The method of claim 4, wherein tracking the received playback duration of the media content includes monitoring one or more of a download buffer associated with a media player, an internal buffer of a demultiplexer associated with the media player, or an internal buffer of a decoder associated with the media player.

### Claim 7 (depends on 4)

The method of claim 4, further comprising:
determining that downloading of the remainder portion of the media fragment has terminated without being completely downloaded;
proceeding with playback of the media fragment using only the frames of the portion of the media fragment; and
proceeding with playback of one or more subsequent media fragments after playback of the media fragment and without rebuffering the media fragment.

### Claim 8 (depends on 7)

The method of claim 7, wherein each of the frames of the media fragment corresponds to one of a plurality of frame types, each of the frame types being characterized by a set of frame dependencies, the plurality of frame types being characterized by a hierarchy of the frame types, and wherein the frames of the media fragment are grouped within the media fragment according to the frame types and ordered within the media fragment according to the hierarchy of the frame types, and wherein the frames of the portion of the media fragment do not include one or more of the frame types.

### Claim 9 (depends on 7)

The method of claim 7, further comprising modifying time stamps associated with at least some of the frames of the portion of the media fragment to smooth out a temporal distribution of the frames of the portion of the media fragment during playback.

### Claim 10

A client device, comprising:
memory; and
one or more processors configured, in conjunction with the memory, to:
transmit, by the client device, a request for media content, the media content including a plurality of media fragments;
receive, by the client device, manifest data configured to enable the client device to request the media fragments;
transmit, by the client device, a request for a media fragment, the media fragment including a plurality of frames;
receive, by the client device, a portion of the media fragment, the portion of the media fragment including fewer than all of the frames of the media fragment;
track a received playback duration of the media content, the received playback duration of the media content including a first playback duration corresponding to the portion of the media fragment, the first playback duration being determined using a fragment index associated with the media fragment and a number of bytes for the portion of the media fragment, wherein the fragment index identifies frames, a reference hierarchy, and frame offsets for the media fragment;
determine that the received playback duration exceeds a threshold;
responsive to determining that the received playback duration exceeds a threshold, initiate, by the client device, playback of the frames of the portion of the media fragment without the media fragment being completely downloaded;
determine, by the client device, that downloading of the media fragment has terminated without the media fragment being completely downloaded;
identify, by the client device, frames of the media fragment not included in the portion of the media fragment; and
transmit, by the client device, a second request specifying a remainder portion of the media fragment that is less than the entire media fragment, the remainder portion of the media fragment including at least some of the frames of the media fragment identified as not being included in the portion of the media fragment.

### Claim 11 (depends on 10)

The client device of claim 10 wherein the media fragment is characterized by a fragment duration, and wherein the threshold is below the fragment duration of the media fragment.

### Claim 12 (depends on 10)

The client device of claim 10, wherein tracking the received playback duration of the media content includes monitoring one or more of a download buffer associated with a media player, an internal buffer of a demultiplexer associated with the media player, or an internal buffer of a decoder associated with the media player.

### Claim 13 (depends on 10)

The client device of claim 10, wherein the one or more processors are further configured to:
determine that downloading of the remainder portion of the media fragment has terminated without being completely downloaded;
proceed with playback of the media fragment using only the frames of the portion of the media fragment; and
proceed with playback of one or more subsequent media fragments after playback of the media fragment and without rebuffering the media fragment.

### Claim 14 (depends on 13)

The client device of claim 13, wherein each of the frames of the media fragment corresponds to one of a plurality of frame types, each of the frame types being characterized by a set of frame dependencies, the plurality of frame types being characterized by a hierarchy of the frame types, and wherein the frames of the media fragment are grouped within the media fragment according to the frame types and ordered within the media fragment according to the hierarchy of the frame types, and wherein the frames of the portion of the media fragment do not include one or more of the frame types.

### Claim 15 (depends on 13)

The client device of claim 13, wherein the one or more processors are further configured to modify time stamps associated with at least some of the frames of the portion of the media fragment to smooth out a temporal distribution of the frames of the portion of the media fragment during playback.