# US 11,310,516 B2 · Adaptive bitrate algorithm with cross-user based viewport prediction for 360-degree video streaming > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,310,516 B2 - Document ID: US11310516B2 - Status: Granted - Kind: B2 - Application number: 16/717,308 - Filed: 2019-12-17 - Published: 2020-06-25 - Granted: 2022-04-19 - Earliest claimed: 2018-12-21 - Assignees: Hulu, LLC - Applicants: Hulu, LLC - Inventors: Lan Xie; Shenglan Huang; Wenhao Zhang - Web page: https://patentgrep.com/patent/US11310516B2 ## CPC - H04N 19/196: being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters (processing of motion vectors H04N19/513) - H04N 19/115: Selection of the code volume for a coding unit prior to coding - H04N 19/146: Data rate or code amount at the encoder output - H04N 19/162: User input - H04N 19/164: Feedback from the receiver or from the transmission channel - H04N 19/174: the region being a slice, e.g. a line of blocks or a group of blocks - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 19/597: specially adapted for multi-view video sequence encoding - H04N 21/23439: {for generating different versions} - H04N 21/2353: {specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata} - H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream - H04N 21/44209: {Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network (arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896)} - H04N 21/4621: {Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen (arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract In some embodiments, a method receives a probability distribution of a likelihood that a user might view one or more tiles in one or more segments of a video. The video is encoded in different profiles that are associated with different bitrates. An available bandwidth is determined. Then, the method selects a profile for each tile in a segment of the video based on the available bandwidth and the probability distribution for the tiles and sends a request for a respective profile for each tile for playback of the segment of the video. ## Claims ### Claim 1 A method comprising: receiving, by a computing device, a probability distribution of a likelihood that a viewport of a user might view one or more tiles in one or more segments of a video, wherein the video is encoded in a plurality of profiles that are associated with a plurality of bitrates, and wherein the probability distribution is based on viewing information from viewports of a plurality of users other than the viewport of the user associated with a playback of the video; determining, by the computing device, an available bandwidth; during playback of the video, selecting, by the computing device, a profile for playback of a segment of the video from the plurality of profiles for each tile in the segment of the video based on the available bandwidth and the probability distribution for the tiles, wherein the selecting using the probability distribution does not use a current viewport fixation of the viewport that is viewing the video; sending, by the computing device, a request for a respective profile for each tile for playback of the segment of the video to a server; and receiving, by the computing device, the segment of video from the server for playback. ### Claim 2 (depends on 1) The method of claim 1, wherein the video is a 360-degree video in which a first portion of the tiles are viewed in the viewport and a second portion are not viewed in the viewport. ### Claim 3 (depends on 1) The method of claim 1, wherein the probability distribution is based a viewport focus of the viewport from the plurality of users. ### Claim 4 (depends on 1) The method of claim 1, wherein the probability distribution includes probabilities that the one or more tiles might be viewed for the one or more segments. ### Claim 5 (depends on 1) The method of claim 1, wherein selecting the profile for each tile in the segment of the video based on the available bandwidth and the probability distribution for the tiles comprises: selecting profiles for tiles to maximize bitrates selected within the available bandwidth based on the probability distribution for the tiles. ### Claim 6 (depends on 1) The method of claim 1, wherein selecting the profile for each tile in the segment of the video based on the available bandwidth and the probability distribution for the tiles comprises: selecting a first profile with a higher bitrate compared to a second profile for a first tile when the first tile is associated with a higher probability in the probability distribution. ### Claim 7 (depends on 1) The method of claim 1, wherein selecting the profile for each tile in the segment of the video based on the available bandwidth and the probability distribution for the tiles comprises: selecting a first set of profiles for a first set of tiles and a second set of profiles for a second set of tiles, wherein the first set of profiles are associated with a first bitrate and the second set of tiles are associated with a second bitrate. ### Claim 8 (depends on 1) The method of claim 1, wherein sending the request for the respective profile for each tile for playback of the segment of the video comprises: requesting multiple profiles associated with different bitrates for different tiles in the segment. ### Claim 9 (depends on 1) The method of claim 1, further comprising: requesting a lowest bitrate profile of the video and the respective profile for each tile for playback of the segment of the video. ### Claim 10 A method comprising: receiving, by a computing device, viewport positioning information in a video for a plurality of viewports of a plurality of users other than a user that is viewing the video; aggregating, by the computing device, the viewport positioning information for each tile in each of one or more segments of the video; generating, by the computing device, a probability distribution of a probability that a viewport of the user might view the one or more tiles in the one or more segments; and sending, by the computing device, the probability distribution of the probability for a segment to a client for use in selecting one of a plurality of profiles that are available for playback for tiles in the segment to request for playback of the segment of the video, wherein the selecting that uses the probability distribution does not use a current viewport fixation of the viewport that is viewing the video. ### Claim 11 (depends on 10) The method of claim 10, wherein the video is a 360-degree video in which a first portion of the tiles are viewed in the viewport and a second portion are not viewed in the viewport. ### Claim 12 (depends on 10) The method of claim 10, wherein aggregating the viewport positioning information for each tile comprises: aggregating views of tiles for the plurality of viewport, wherein the aggregated views are used to generate the probability distribution. ### Claim 13 (depends on 10) The method of claim 10, further comprising: encoding the one or more segments in the plurality of profiles, wherein the plurality of profiles are associated with a plurality of bitrates. ### Claim 14 (depends on 10) The method of claim 10, wherein a media player selects a profile from the plurality of profiles for each tile in a segment of the video based on an available bandwidth and the probability distribution for the tiles. ### Claim 15 (depends on 10) The method of claim 10, further comprising: receiving a selection for a profile for each tile in a segment of the video, wherein the selection is calculated based on an available bandwidth and the probability distribution for the tiles. ### Claim 16 A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for: receiving a probability distribution of a likelihood that a viewport of a user might view one or more tiles in one or more segments of a video, wherein the video is encoded in a plurality of profiles that are associated with a plurality of bitrates, and wherein the probability distribution is based on viewing information from viewports of a plurality of users other than the viewport of the user associated with a playback of the video; determining an available bandwidth; during playback of the video, selecting a profile for playback of a segment of the video from the plurality of profiles for each tile in the segment of the video based on the available bandwidth and the probability distribution for the tiles, wherein the selecting using the probability distribution does not use a current viewport fixation of the viewport that is viewing the video; sending a request for a respective profile for each tile for playback of the segment of the video to a server; and receiving the segment of video from the server for playback. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the probability distribution is based a viewport focus of the viewport from the plurality of users. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the probability distribution includes probabilities that the one or more tiles might be viewed for the one or more segments. ### Claim 19 (depends on 1) The method of claim 1, wherein the probability distribution is received during playback of the video. ### Claim 20 (depends on 1) The method of claim 1, wherein the probability distribution is received for a segment after a current segment being viewed in the video. ### Claim 21 (depends on 1) The method of claim 1, wherein the probability distribution that is received is dynamically calculated during playback of the video.