# US 2023/0058707 A1 · Methods, systems, and media for selecting video formats for adaptive video streaming

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0058707 A1
- Document ID: US20230058707A1
- Status: Published application
- Kind: A1
- Application number: 17/980,031
- Filed: 2022-11-03
- Published: 2023-02-23
- Earliest claimed: 2020-03-31
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Yeping Su; Balu Adsumilli
- Web page: https://patentgrep.com/patent/US20230058707A1

## CPC

- H04L 65/80: Responding to QoS
- H04L 65/612: for unicast
- H04L 65/752: adapting media to network capabilities
- H04L 65/762: {at the source (reformatting of additional data in video distribution servers H04N21/2355)}
- H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04L 67/565: Conversion or adaptation of application format or content (adding application control or application functional data H04L67/561)
- H04N 7/01: Conversion of standards {, e.g. involving analogue television standards or digital television standards processed at pixel level}
- H04N 21/26258: {for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40)}
- H04N 21/442: Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk {(arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00)}
- H04N 21/44209: {Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network (arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896)}
- H04N 21/4516: {involving client characteristics, e.g. Set-Top-Box type, software version or amount of memory available}
- H04N 21/4662: {characterized by learning algorithms}
- H04N 21/6581: {Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application}
- H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments}
- H04L 67/61: taking into account QoS or priority requirements
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Methods, systems, and media for selecting video formats for adaptive video streaming are provided. In some embodiments, the method comprises: receiving an indication of a video to be presented on a user device; identifying a group of quality metrics for each of a plurality of segments of the video, wherein each quality metric includes values for a particular segment and for a particular format of a group of available formats for the video; selecting a first format for a first segment of the video; causing the first segment of the video to be presented on the user device; identifying a quality of a network connection between the user device and a server that hosts the video; identifying a second format for a second segment of the video based on the quality of the network connection; determining whether a format of the video is to be changed from the first format to the second format based at least on the group of quality metrics for the second segment of the video; and, in response to determining that the format of the video is to be changed from the first format to the second format, causing the second segment having the second format to be presented by the user device.

## Claims

### Claim 1

A method for selecting video formats for adaptive video streaming, the method comprising:
receiving, at a server including a hardware processor, an indication of a first video to be presented on a user device;
generating, at the server, by inputting video features of the first video into a trained model, a group of predicted viewership scores for each format from a group of available formats for the first video;
selecting, at the server, a first format for a first video based on the group of predicted viewership scores for the first video;
causing, at the server, the first video to be presented on the user device;
identifying, at the server, a quality of a network connection between the user device and the server that hosts the first video during the presentation of the first video on the user device;
identifying, at the server, a second format for a second video based on the quality of the network connection;
determining, at the server, whether a format of the second video is to be changed from the first format to the second format based on the group of predicted viewership scores for the second video; and
in response to determining that the format of the second video is to be changed from the first format to the second format, causing, at the server, the second video to be presented by the user device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising generating, by inputting the video features of the first video into the trained model, a group of quality metrics for each segment from a plurality of segments and for each format from the group of available formats for the video.

### Claim 3 (depends on 2)

The method of claim 2, wherein each quality metric for a subsequent segment of the first video corresponds to a difference between the quality metric for the subsequent segment and the quality metric for a preceding segment.

### Claim 4 (depends on 2)

The method of claim 2, wherein the group of quality metrics includes data from a sensor used to capture the first video, and wherein determining whether the format of the second video is to be changed from the first format to the second format is based on the data from the sensor of the camera.

### Claim 5 (depends on 4)

The method of claim 4, wherein the data from the sensor includes motion information that indicates positioning of the camera during capture of the first video.

### Claim 6 (depends on 2)

The method of claim 2, wherein the group of quality metrics includes analysis data from processing of the video after capture of the first video.

### Claim 7 (depends on 6)

The method of claim 6, wherein the analysis data includes activity information associated with content included in the first video.

### Claim 8 (depends on 2)

The method of claim 2, wherein at least one quality metric in the group of quality metrics includes Just Noticeable Difference (JND) scores that indicate a minimum difference in a value of the quality metric from the first video to the second video that is perceptible to a viewer of the video.

### Claim 9 (depends on 8)

The method of claim 8, wherein determining that the format of the second video is to be changed from the first format to the second format is based on JND scores for the first format and the second format.

### Claim 10 (depends on 2)

The method of claim 2, wherein determining that the format of the second video is to be changed from the first format to the second format is based on the first trained model that takes the group of quality metrics and the quality of the network connection as inputs and generates an output indicating that the format of the second video is to be changed.

### Claim 11

A system for selecting video formats for adaptive video streaming, the system comprising:
a server including a hardware processor that is configured to:
receive, at the server, an indication of a first video to be presented on a user device;
generate, at the server, by inputting video features of the first video into a trained model, a group of predicted viewership scores for each format from a group of available formats for the first video;
select, at the server, a first format for a first video based on the group of predicted viewership scores for the first video;
cause, at the server, the first video to be presented on the user device;
identify, at the server, a quality of a network connection between the user device and the server that hosts the first video during the presentation of the first video on the user device;
identify, at the server, a second format for a second video based on the quality of the network connection;
determine, at the server, whether a format of the second video is to be changed from the first format to the second format based on the group of predicted viewership scores for the second video; and
in response to determining that the format of the second video is to be changed from the first format to the second format, cause, at the server, the second video to be presented by the user device.

### Claim 12 (depends on 11)

The system of claim 11, wherein the hardware processor is further configured to generate, by inputting the video features of the first video into the trained model, a group of quality metrics for each segment from a plurality of segments and for each format from the group of available formats for the video.

### Claim 13 (depends on 12)

The system of claim 12, wherein each quality metric for a subsequent segment of the first video corresponds to a difference between the quality metric for the subsequent segment and the quality metric for a preceding segment.

### Claim 14 (depends on 12)

The system of claim 12, wherein the group of quality metrics includes data from a sensor used to capture the first video, and wherein determining whether the format of the second video is to be changed from the first format to the second format is based on the data from the sensor of the camera.

### Claim 15 (depends on 14)

The system of claim 14, wherein the data from the sensor includes motion information that indicates positioning of the camera during capture of the first video.

### Claim 16 (depends on 12)

The system of claim 12, wherein the group of quality metrics includes analysis data from processing of the video after capture of the first video.

### Claim 17 (depends on 16)

The system of claim 16, wherein the analysis data includes activity information associated with content included in the first video.

### Claim 18 (depends on 12)

The system of claim 12, wherein at least one quality metric in the group of quality metrics includes Just Noticeable Difference (JND) scores that indicate a minimum difference in a value of the quality metric from the first video to the second video that is perceptible to a viewer of the video.

### Claim 19 (depends on 18)

The system of claim 18, wherein determining that the format of the second video is to be changed from the first format to the second format is based on JND scores for the first format and the second format.

### Claim 20 (depends on 12)

The system of claim 12, wherein determining that the format of the second video is to be changed from the first format to the second format is based on the first trained model that takes the group of quality metrics and the quality of the network connection as inputs and generates an output indicating that the format of the second video is to be changed.

### Claim 21

A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for selecting video formats for adaptive video streaming, the method comprising:
receiving, at a server including a hardware processor, an indication of a first video to be presented on a user device;
generating, at the server, by inputting video features of the first video into a trained model, a group of predicted viewership scores for each format from a group of available formats for the first video;
selecting, at the server, a first format for a first video based on the group of predicted viewership scores for the first video;
causing, at the server, the first video to be presented on the user device;
identifying, at the server, a quality of a network connection between the user device and the server that hosts the first video during the presentation of the first video on the user device;
identifying, at the server, a second format for a second video based on the quality of the network connection;
determining, at the server, whether a format of the second video is to be changed from the first format to the second format based on the group of predicted viewership scores for the second video; and
in response to determining that the format of the second video is to be changed from the first format to the second format, causing, at the server, the second video to be presented by the user device.
