# US 12,603,931 B2 · Methods and systems for encoder parameter setting optimization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,603,931 B2
- Document ID: US12603931B2
- Status: Granted
- Kind: B2
- Application number: 18/403,659
- Filed: 2024-01-03
- Published: 2024-04-25
- Granted: 2026-04-14
- Earliest claimed: 2021-08-31
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Ching Yin Derek Pang; Kyrah Felder; Akshay Gadde; Paul Wilkins; Cheng Chen; Yao-Chung Lin
- Web page: https://patentgrep.com/patent/US12603931B2

## CPC

- H04L 65/70: Media network packetisation
- H04N 19/196: being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters (processing of motion vectors H04N19/513)
- G06N 20/00: Machine learning
- H04L 65/61: for supporting one-way streaming services, e.g. Internet radio
- H04L 65/80: Responding to QoS
- H04N 19/103: Selection of coding mode or of prediction mode
- H04N 19/115: Selection of the code volume for a coding unit prior to coding
- H04N 19/146: Data rate or code amount at the encoder output
- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- H04N 19/172: the region being a picture, frame or field
- H04N 21/251: {Learning process for intelligent management, e.g. learning user preferences for recommending movies (details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08)}
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06N 20/20: Ensemble learning
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A media item to be provided to users of a platform is identified. The media item is associated with a media class of one or more media classes. An indication of the media item is provided as input to a machine learning model trained based on historical encoding data to predict, for a given media item, a set of encoder parameter settings that satisfy a performance criterion in view of a respective media class of the given media item. The historical encoding data includes a prior set of encoder parameter settings that satisfied the performance criterion with respect to a prior media item associated with the respective class. Encoder parameter settings that satisfy the performance criterion in view of the media class is determined based on an output of the model. The media item is caused to be encoded using the determined encoder parameter settings.

## Claims

### Claim 1

A method comprising:
identifying a media item to be provided to one or more users of a platform, wherein the identified media item is associated with a media class of a plurality of media classes;
providing an indication of the identified media item as input to a machine learning model trained based on historical encoding data to predict, for a given media item, a set of encoder parameter settings that satisfy a performance criterion in view of a respective media class associated with the given media item, the historical encoding data comprising a prior set of encoder parameter settings that satisfied the performance criterion with respect to a prior media item associated with the respective media class;
obtaining one or more outputs of the machine learning model, wherein the one or more outputs identify:
one or more sets of encoder parameter settings, and
for each of the one or more sets of the encoder parameter settings, a level of confidence that a respective set of encoder parameter settings satisfies the performance criterion in view of the media class associated with the identified media item;
identifying, based on the one or more outputs of the machine learning model, at least one respective set of encoder parameter settings having a level of confidence that satisfies a confidence criterion, wherein the level of confidence indicates whether the set of encoder parameter settings satisfies the performance criterion in view of the media class associated with the identified media item; and
causing the identified media item to be encoded using the identified respective set of encoder parameter settings.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
obtaining encoding statistics associated with the identified media item based on an initial encoding of the identified media item by a media item encoder; and
providing the obtained encoding statistics as an additional input to the machine learning model.

### Claim 3 (depends on 2)

The method of claim 2, wherein obtaining the encoding statistics associated with the identified media item comprises:
providing the identified media item and a default set of encoder parameter settings as input to the media item encoder;
obtaining an encoded bit stream based on one or more outputs of the media item encoder; and
determining a value associated with the encoding statistics associated with the encoded bit stream.

### Claim 4 (depends on 1)

The method of claim 1, wherein each of the plurality of media classes corresponds to at least one of a distinct image characteristic or a distinct content category.

### Claim 5 (depends on 4)

The method of claim 4, wherein the image characteristic associated with the media item corresponds to at least one of a spatial resolution associated with the identified media item, a frame rate associated with the identified media item, a motion activity associated with the identified media item, a type of device that generated the identified media item, an amount of image noise associated with the identified media item, an image texture complexity associated with the identified media item, or a spatial complexity associated with the identified media item.

### Claim 6 (depends on 1)

The method of claim 1,
wherein identifying the at least one respective set of encoder parameter settings having the level of confidence that satisfies the confidence criterion comprises at least one of:
determining that the at least one respective set of encoder parameter settings, of the one or more sets of encoder parameter settings, is associated with the level of confidence that satisfies the confidence criterion, or
determining that another set of encoder parameter settings, of the one or more sets of encoder parameter settings, is associated with another level of confidence that does not satisfy the confidence criterion.

### Claim 7 (depends on 1)

The method of claim 1, wherein the historical encoding data comprises (i) a training input comprising the respective media class of the prior media item and (ii) a target output for the training input, the target output comprising an indication of the prior set of encoder parameter settings that satisfied the performance criterion with respect to the prior media item.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
determining the media class associated with the identified media item; and
providing the determined media class as an additional input to the machine learning model.

### Claim 9 (depends on 8)

The method of claim 8, wherein determining the media class associated with the identified media item comprises:
obtaining one or more characteristics associated with the identified media item, wherein the one or more characteristics correspond to at least one of an image characteristic associated with the identified media item or a content category associated with the identified media item; and
providing the one or more characteristics as input to an additional machine learning model, wherein the additional machine learning model is trained to predict, based on given input characteristics associated with a respective media item, a particular media class, of the plurality of media classes, that corresponds to the respective media item in view of the given input characteristics,
wherein the media class associated with the identified media item is determined based on one or more outputs of the additional machine learning model.

### Claim 10 (depends on 1)

The method of claim 1, wherein the performance criterion corresponds to at least one of a bitrate savings performance criterion, a peak signal-to-noise ratio (PSNR) criterion, a structural similarity index (SSIM) criterion, a perceptual media quality criterion, or a psychovisual similarity criterion.

### Claim 11

A system comprising:
a memory device; and
a set of one or more processors coupled to the memory device, the set of one or more processors to perform operations comprising:
identifying a media item to be provided to one or more users of a platform, wherein the identified media item is associated with a media class of a plurality of media classes;
providing an indication of the identified media item as input to a machine learning model trained based on historical encoding data to predict, for a given media item, a set of encoder parameter settings that satisfy a performance criterion in view of a respective media class associated with the given media item, the historical encoding data comprising a prior set of encoder parameter settings that satisfied the performance criterion with respect to a prior media item associated with the respective media class;
obtaining one or more outputs of the machine learning model, wherein the one or more outputs identify:
one or more sets of encoder parameter settings and, for each of the one or more sets of encoder parameter settings, and
for each of the one or more sets of the encoder parameter settings, a level of confidence that a respective set of encoder parameter settings satisfies the performance criterion in view of the media class associated with the media item;
identifying, based on one or more outputs of the machine learning model, at least one respective set of encoder parameter settings having a level of confidence that satisfies a confidence criterion, wherein the level of confidence indicates whether the set of encoder parameter settings satisfies the performance criterion in view of the media class associated with the identified media item; and
causing the identified media item to be encoded using the identified at least one respective set of encoder parameter settings.

### Claim 12 (depends on 11)

The system of claim 11, wherein the operations further comprise:
obtaining encoding statistics associated with the identified media item based on an initial encoding of the identified media item by a media item encoder; and
providing the obtained encoding statistics as an additional input to the machine learning model.

### Claim 13 (depends on 12)

The system of claim 12, wherein obtaining the encoding statistics associated with the identified media item comprises:
providing the identified media item and a default set of encoder parameter settings as input to the media item encoder;
obtaining an encoded bit stream based on one or more outputs of the media item encoder; and
determining a value associated with the encoding statistics associated with the encoded bit stream.

### Claim 14 (depends on 11)

The system of claim 11, wherein each of the plurality of media classes corresponds to at least one of a distinct image characteristic or a distinct content category.

### Claim 15 (depends on 14)

The system of claim 14, wherein the image characteristic associated with the media item corresponds to at least one of a spatial resolution associated with the identified media item, a frame rate associated with the identified media item, a motion activity associated with the identified media item, a type of device that generated the identified media item, an amount of image noise associated with the identified media item, an image texture complexity associated with the identified media item, or a spatial complexity associated with the identified media item.

### Claim 16 (depends on 11)

The system of claim 11,
wherein identifying the at least one respective set of encoder parameter settings having the level of confidence that satisfies the confidence criterion comprises at least one of:
determining that the at least one respective set of encoder parameter settings, of the one or more sets of encoder parameter settings, is associated with the level of confidence that satisfies the confidence criterion, or
determining that another set of encoder parameter settings, of the one or more sets of encoder parameter settings, is associated with another level of confidence that does not satisfy the confidence criterion.

### Claim 17 (depends on 11)

The system of claim 11, wherein the historical encoding data comprises (i) a training input comprising the respective media class of the prior media item and (ii) a target output for the training input, the target output comprising an indication of the prior set of encoder parameter settings that satisfied the performance criterion with respect to the prior media item.

### Claim 18

A non-transitory computer readable storage medium comprising instructions for a server that, when executed by a set of one or more processors, cause the set of one or more processors to perform operations comprising:
identifying a media item to be provided to one or more users of a platform, wherein the identified media item is associated with a media class of a plurality of media classes;
providing an indication of the identified media item as input to a machine learning model trained based on historical encoding data to predict, for a given media item, a set of encoder parameter settings that satisfy a performance criterion in view of a respective media class associated with the given media item, the historical encoding data comprising a prior set of encoder parameter settings that satisfied the performance criterion with respect to a prior media item associated with the respective media class;
obtaining one or more outputs of the machine learning model, wherein the one or more outputs identify:
one or more sets of encoder parameter settings, and
for each of the one or more sets of the encoder parameter settings, a level of confidence that a respective set of encoder parameter settings satisfies the performance criterion in view of the media class associated with the identified media item;
identifying, based on one or more outputs of the machine learning model, at least one respective set of encoder parameter settings having a level of confidence that satisfies a confidence criterion, wherein the level of confidence indicates whether the set of encoder parameter settings satisfies the performance criterion in view of the media class associated with the identified media item; and
causing the identified media item to be encoded using the identified respective set of encoder parameter settings.

### Claim 19 (depends on 18)

The non-transitory computer readable storage medium of claim 18, wherein the operations further comprise:
obtaining encoding statistics associated with the identified media item based on an initial encoding of the identified media item by a media item encoder; and
providing the obtained encoding statistics as an additional input to the machine learning model.

### Claim 20 (depends on 19)

The non-transitory computer readable storage medium of claim 19, wherein obtaining the encoding statistics associated with the identified media item comprises:
providing the identified media item and a default set of encoder parameter settings as input to the media item encoder;
obtaining an encoded bit stream based on one or more outputs of the media item encoder; and
determining a value associated with the encoding statistics associated with the encoded bit stream.