# US 12,225,252 B2 · Dynamic selection of candidate bitrates for video encoding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,225,252 B2 - Document ID: US12225252B2 - Status: Granted - Kind: B2 - Application number: 18/179,281 - Filed: 2023-03-06 - Published: 2024-09-12 - Granted: 2025-02-11 - Earliest claimed: 2023-03-06 - Assignees: Disney Enterprises, Inc.; Beijing YoJaJa Software Technology Development Co., Ltd. - Applicants: Beijing YoJaJa Software Technology Development Co., Ltd.; Disney Enterprises, Inc. - Inventors: Chen Liu; Wenhao Zhang; Scott Labrozzi; Yuanyi Xue; Xuchang Huangfu; Xiaobo Liu - Web page: https://patentgrep.com/patent/US12225252B2 ## CPC - H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567) - H04N 21/238: Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams {(hybrid fiber coaxial [HFC] networks for downstream channel allocation for video distribution H04L12/2801; flow control in packet networks H04L47/10; real-time communication protocols in data switching networks H04L65/00; scheduling or organising the servicing of application requests H04L67/60)} - H04N 19/115: Selection of the code volume for a coding unit prior to coding - H04N 19/146: Data rate or code amount at the encoder output - H04N 19/179: the unit being a scene or a shot - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/23439: {for generating different versions} - H04N 21/2662: Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract In some embodiments, a method generates a first representation of a first relationship between bitrate and quality based on first features of a first portion of a video. Also, the method generates a second representation of a second relationship between bitrate and quality based on second features of a second portion of a video. The first representation is analyzed to determine a first list of bitrates for the first portion of video and the second representation is analyzed to determine a second list of bitrates for the second portion of video. The first list of bitrates is different from the second list of bitrates. The method outputs the first list of bitrates for use encoding the first portion of video and the second list of bitrates for use encoding the second portion of video. ## Claims ### Claim 1 A method comprising: generating, by a computing device, a first representation of a first relationship between bitrate and quality based on first features of a first portion of a video; generating, by the computing device, a second representation of a second relationship between bitrate and quality based on second features of a second portion of the video; analyzing, by the computing device, the first representation to determine a first list of bitrates for the first portion of video and analyzing the second representation to determine a second list of bitrates for the second portion of video, wherein the first list of bitrates is different from the second list of bitrates, wherein analyzing the first representation or analyzing the second representation comprises: generating a list of potential bitrates based on the first representation or the second representation, and refining the list of potential bitrates based to a quality associated with the potential bitrates to determine the first list of bitrates or the second list of bitrates, wherein refining the list of potential bitrates comprises: adding a first potential bitrate to the list of potential bitrates; and outputting, by the computing device, the first list of bitrates for encoding the first portion of video and the second list of bitrates for encoding the second portion of video. ### Claim 2 (depends on 1) The method of claim 1, wherein: generating the first representation comprises generating a first prediction of the first relationship for bitrate and quality, and generating the second representation comprises generating a second prediction of the second relationship for bitrate and quality. ### Claim 3 (depends on 1) The method of claim 1, wherein: generating the first representation comprises: inputting the first features into a prediction network; and generating a first prediction of the first representation based on the first features, and generating the second representation comprises: inputting the second features into the prediction network; and generating a second prediction of the second representation based on the second features. ### Claim 4 (depends on 1) The method of claim 1, wherein refining the list of potential bitrates comprises: removing a first potential bitrate from the list of potential bitrates. ### Claim 5 (depends on 4) The method of claim 4, wherein removing the potential bitrate comprises: determining a second potential bitrate; comparing a first quality of the first potential bitrate to a second quality of the second potential bitrate; and determining whether to remove the first potential bitrate based on the comparing. ### Claim 6 (depends on 5) The method of claim 5, wherein the first potential bitrate is removed from the list of potential bitrates when a difference between the first quality and the second quality meets a threshold. ### Claim 7 (depends on 1) The method of claim 1, wherein adding the potential bitrate comprises: determining a second potential bitrate and a third potential bitrate that are in the list of potential bitrates; comparing a first quality of the second potential bitrate to a second quality of the third potential bitrate; and determining whether to add the first potential bitrate based on the comparing. ### Claim 8 (depends on 7) The method of claim 7, wherein the first potential bitrate is added when a difference between the first quality and the second quality meets a threshold. ### Claim 9 (depends on 1) The method of claim 1, wherein analyzing the first representation or analyzing the second representation comprises: analyzing a plurality of first representations for a plurality of segments in the first portion of video or the second portion of video; determining a first minimum bitrate and a first maximum bitrate for each of the plurality of first representations; and determining a second minimum bitrate and a second maximum bitrate for the first portion of video or the second portion of video based on the first minimum bitrate and the first maximum bitrate for each of the plurality of first representations. ### Claim 10 (depends on 9) The method of claim 9, wherein analyzing the first representation or analyzing the second representation comprises: generating the list of potential bitrates based on the second minimum bitrate and the second maximum bitrate, wherein potential bitrates in the list of potential bitrates are analyzed to determine the first list of bitrates or the second list of bitrates. ### Claim 11 (depends on 10) The method of claim 10, wherein analyzing the first representation or analyzing the second representation comprises: determining potential removal candidate bitrates for each of the plurality of first representations, wherein a potential removal candidate bitrate is a potential removal from the list of potential bitrates; and determining whether to remove a potential removal candidate bitrate based on the potential removal candidate bitrate being included as a potential removal candidate in one or more of each of the plurality of first representations. ### Claim 12 (depends on 10) The method of claim 10, wherein analyzing the first representation or analyzing the second representation comprises: determining potential added candidate bitrates for each of the plurality of first representations, wherein a potential added candidate is a potential addition to the list of potential bitrates; and determining whether to add a potential bitrate based on the potential added candidate bitrate being included as a potential added candidate in one or more of each of the plurality of first representations. ### Claim 13 (depends on 1) The method of claim 1, wherein: the first list of bitrates includes a bitrate that is not included in the second list of bitrates. ### Claim 14 (depends on 1) The method of claim 1, further comprising: encoding the first portion of video using the first list of bitrates to generate a plurality of first encoded portions for the first portion of video; and encoding the second portion of video using the second list of bitrates to generate a plurality of second encoded portions for the second portion of video. ### Claim 15 (depends on 14) The method of claim 14, further comprising: selecting an encoded segment from the plurality of first encoded portions for a first profile in a profile ladder; and selecting an encoded segment from the plurality of second encoded portions for the first profile in the profile ladder. ### Claim 16 A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for: generating a first representation of a first relationship between bitrate and quality based on first features of a first portion of a video; generating a second representation of a second relationship between bitrate and quality based on second features of a second portion of the video; analyzing the first representation to determine a first list of bitrates for the first portion of video and analyzing the second representation to determine a second list of bitrates for the second portion of video, wherein the first list of bitrates is different from the second list of bitrates, wherein analyzing the first representation or analyzing the second representation comprises: generating a list of potential bitrates based on the first representation or the second representation, and refining the list of potential bitrates based on a quality associated with the potential bitrates to determine the first list of bitrates or the second list of bitrates, wherein refining the list of potential bitrates comprises: adding a first potential bitrate to the list of potential bitrates; and outputting the first list of bitrates for use encoding the first portion of video and the second list of bitrates for use encoding the second portion of video. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein: generating the first representation comprises generating a first prediction of the first relationship for bitrate and quality, and generating the second representation comprises generating a second prediction of the second relationship for bitrate and quality. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein refining the list of potential bitrates comprises: removing a first potential bitrate from the list of potential bitrates. ### Claim 19 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein adding the potential bitrate comprises: determining a second potential bitrate and a third potential bitrate that are in the list of potential bitrates; comparing a first quality of the second potential bitrate to a second quality of the third potential bitrate; and determining whether to add the first potential bitrate based on the comparing. ### Claim 20 An apparatus comprising: one or more computer processors; and a computer-readable storage medium comprising instructions for controlling the one or more computer processors to be operable for: generating a first representation of a first relationship between bitrate and quality based on first features of a first portion of a video; generating a second representation of a second relationship between bitrate and quality based on second features of a second portion of the video; analyzing the first representation to determine a first list of bitrates for the first portion of video and analyzing the second representation to determine a second list of bitrates for the second portion of video, wherein the first list of bitrates is different from the second list of bitrates, wherein analyzing the first representation or analyzing the second representation comprises: generating a list of potential bitrates based on the first representation or the second representation; and refining the list of potential bitrates based on a quality associated with the potential bitrates to determine the first list of bitrates or the second list of bitrates, wherein refining the list of potential bitrates comprises: adding a first potential bitrate to the list of potential bitrates; and outputting the first list of bitrates for use encoding the first portion of video and the second list of bitrates for use encoding the second portion of video.