# US 12,015,805 B2 · Method, an apparatus and a computer program product for video streaming > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,015,805 B2 - Document ID: US12015805B2 - Status: Granted - Kind: B2 - Application number: 17/795,631 - Filed: 2021-01-21 - Published: 2023-03-09 - Granted: 2024-06-18 - Earliest claimed: 2020-01-29 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Ari Hourunranta; Miska Matias Hannuksela; Emre Baris Aksu; Saba Ahsan - Web page: https://patentgrep.com/patent/US12015805B2 ## CPC - H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras} - H04N 21/23439: {for generating different versions} - H04N 21/26258: {for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40)} - H04N 21/472: End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content {(end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612)} - H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region - H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection - H04N 21/816: {involving special video data, e.g 3D video} - H04N 21/845: Structuring of content, e.g. decomposing content into time segments - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The embodiments relate to a method including determining a foreground area covering a viewport of 360-degree video and one or more other areas of 360-degree video, not containing the foreground area in its entirety; concluding a first set of tile streams among available tile streams of the 360-degree video to cover the foreground area; concluding a second set of tile streams among the available tile streams of the 360-degree video, to cover the one or more other areas; and requesting transmission of a first set of portions of the first set of tile streams and a second set of portions of the second set of tile streams, wherein the portions in the first set of portions have a shorter duration that portions in the second set of portions. ## Claims ### Claim 1 A method, comprising: determining a foreground area covering a viewport of 360-degree video, and one or more other areas of the 360-degree video not containing the foreground area in its entirety; selecting a first set of tile streams among available tile streams of the 360-degree video, wherein the first set of tile streams covers the foreground area; selecting a second set of tile streams among the available tile streams of the 360-degree video, wherein the second set of tile streams covers the one or more other areas; and requesting transmission of a first set of portions of the first set of tile streams and a second set of portions of the second set of tile streams, wherein at least one of the first set of portions have a shorter duration than at least one portion of the second set of portions. ### Claim 2 (depends on 1) A method according to claim 1, wherein the one or more other areas comprises a margin area and a background area, and the method further comprises: determining the margin area adjacent to the foreground area, and the background area covering 360-degree video not contained in the foreground or margin areas; selecting a first subset of the second set of tile streams, wherein the first subset of tile streams covers the foreground area; selecting a second subset of the second set of tile streams, wherein the second subset of tile streams comprises tile streams that are among the second set of tile streams and not among the first subset of tile streams; and requesting transmission of a first subset of portions of the first subset of tile streams and a second subset of portions of the second subset of tile streams, wherein at least one of the first subset of portions of the first subset of tile streams have a shorter duration than at least one portion of the second subset of portions of the second subset of tile streams. ### Claim 3 (depends on 1) A method according to claim 1, further comprising requesting transmission of an extractor track in portions that have a longer duration than that for the at least one of the first set of portions. ### Claim 4 (depends on 1) A method according to claim 1, further comprising: obtaining, from a media presentation description and/or a stream initialization data and/or a stream index data, metadata on areas covered with the available tile streams; and selecting the first set of tile streams and the second set of tile streams based on the metadata. ### Claim 5 An apparatus comprising: at least one processor; and at least one non-transitory memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least: determine a foreground area covering a viewport of 360-degree video, and one or more other areas of the 360-degree video not containing the foreground area in its entirety; select a first set of tile streams among available tile streams of the 360-degree video, wherein the first set of tile streams covers the foreground area; select a second set of tile streams among the available tile streams of the 360-degree video, wherein the second set of tile streams covers the one or more other areas; and request transmission of a first set of portions of the first set of tile streams and a second set of portions of the second set of tile streams, wherein at least one of the first set of portions have a shorter duration than at least one portion of the second set of portions. ### Claim 6 (depends on 5) An apparatus according to claim 5, wherein the one or more other areas comprises a margin area and a background area, and the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to: determine the margin area adjacent to the foreground area, and a background area covering 360-degree video not contained in the foreground or margin areas; select a first subset of the second set of tile streams, wherein the first subset of tile streams covers the foreground area; select a second subset of the second set of tile streams, wherein the second subset of tile streams comprises tile streams that are among the second set of tile streams and not among the first subset of tile streams; and request transmission of a first subset of portions of the first subset of tile streams and a second subset of portions of the second subset of tile streams, wherein at least one of the first subset of portions of the first subset of tile streams have a shorter duration than at least one portion of the second subset of portions of the second subset of tile streams. ### Claim 7 (depends on 5) An apparatus according to claim 5, where the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to request transmission of an extractor track in portions that have a longer duration than that for the at least one of the first set of portions. ### Claim 8 (depends on 5) An apparatus according to claim 5, where the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to: obtain, from a media presentation description and/or stream initialization data and/or stream index data, metadata on areas covered with the available tile streams; and select the first set of tile streams and the second set of tile streams based on the metadata. ### Claim 9 A computer program product comprising a non-transitory computer readable medium encoded with computer program code configured to, when executed on at least one processor, cause an apparatus or a system to: determine a foreground area covering a viewport of 360-degree video, and one or more other areas of the 360-degree video not containing the foreground area in its entirety; select a first set of tile streams among available tile streams of the 360-degree video, wherein the first set of tile streams covers the foreground area; select a second set of tile streams among the available tile streams of the 360-degree video, wherein the second set of tile streams covers the one or more other areas; and request transmission of a first set of portions of the first set of tile streams and a second set of portions of the second set of tile streams, wherein at least one of the first set of portions have a shorter duration than at least one portion of the second set of portions. ### Claim 10 (depends on 9) A computer program product according to claim 9, wherein the one or more other areas comprises a margin area and a background area, and the computer program product is configured to cause the apparatus or system to: determine the margin area adjacent to the foreground area, and a background area covering 360-degree video not contained in the foreground or margin areas; select a first subset of the second set of tile streams, wherein the first subset of tile streams covers the foreground area; select a second subset of the second set of tile streams, wherein the second subset of tile streams comprises tile streams that are among the second set of tile streams and not among the first subset of tile streams; and request transmission of a first subset of portions of the first subset of tile streams and a second subset of portions of the second subset of tile streams, wherein at least one of the first subset of portions of the first subset of tile streams have a shorter duration than at least one portion of the second subset of portions of the second subset of tile streams. ### Claim 11 (depends on 9) A computer program product according to claim 9, where the computer program code is configured to cause the apparatus to request transmission of an extractor track in portions that have a longer duration than that for the at least one of the first set of portions. ### Claim 12 (depends on 9) A computer program product according to claim 9, where the computer program code is configured to cause the apparatus to: obtain, from a media presentation description and/or stream initialization data and/or stream index data, metadata on areas covered with the available tile streams; and select the first set of tile streams and the second set of tile streams based on the metadata. ### Claim 13 (depends on 9) A computer program product according to claim 9, where the computer program code is configured to cause the apparatus to: receive the first set of portions and the second set of portions; and perform presentation of at least one of the first set of portions or the second set of portions. ### Claim 14 (depends on 5) An apparatus according to claim 5, wherein the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to: receive the first set of portions and the second set of portions; and perform presentation of at least one of the first set of portions or the second set of portions. ### Claim 15 (depends on 5) An apparatus according to claim 5, wherein a DASH client comprises the apparatus. ### Claim 16 (depends on 5) An apparatus according to claim 5, wherein transmission of the first subset of portions and the second subset of portions is requested from a server. ### Claim 17 (depends on 5) An apparatus according to claim 5, wherein the first set of portions of the first set of tile streams comprises at least one of: one or more first segments of the 360-degree video, or one or more first subsegments of the 360-degree video, wherein the second set of portions of the second set of tile streams comprises at least one of: one or more second segments of the 360-degree video, or one or more second subsegments of the 360-degree video. ### Claim 18 (depends on 1) A method according to claim 1, further comprising: receiving the first set of portions and the second set of portions; and performing presentation of at least one of the first set of portions or the second set of portions. ### Claim 19 (depends on 1) A method according to claim 1, wherein transmission of the first subset of portions and the second subset of portions is requested from a server. ### Claim 20 (depends on 1) A method according to claim 1, wherein the first set of portions of the first set of tile streams comprises at least one of: one or more first segments of the 360-degree video, or one or more first subsegments of the 360-degree video, wherein the second set of portions of the second set of tile streams comprises at least one of: one or more second segments of the 360-degree video, or one or more second subsegments of the 360-degree video.