# US 11,451,838 B2 · Method for adaptive streaming of media > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,451,838 B2 - Document ID: US11451838B2 - Status: Granted - Kind: B2 - Application number: 16/769,726 - Filed: 2018-12-07 - Published: 2021-12-09 - Granted: 2022-09-20 - Earliest claimed: 2017-12-07 - Assignees: Koninklijke KPN N.V.; Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek TNG - Applicants: Koninklijke Kpn N.V.; Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno - Inventors: Alexandre Da Silva Pratas Gabriel; Emmanuel Thomas - Web page: https://patentgrep.com/patent/US11451838B2 ## CPC - H04N 21/234345: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment} - H04L 65/65: Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP] - H04N 21/23439: {for generating different versions} - H04N 21/26258: {for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40)} - H04N 21/2662: Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities - H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region - H04N 21/6581: {Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application} - H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection - H04N 21/816: {involving special video data, e.g 3D video} - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A method for adaptive streaming of video is described, wherein the method comprises: transmitting first metadata to a client apparatus, the first metadata defining a first adaptation set, an adaptation set defining one or more sets of segment identifiers, each set of segment identifiers defining a plurality of linearly playable media segments of a predetermined resolution, each media segment in a set of media segments having an identical playback duration, the first metadata enabling a client apparatus to request media segments and sequentially playout the media segments; receiving navigation information from the client apparatus, the navigation information defining at least a first region, in the video frames of the first adaptation set; selecting second metadata defining a second adaptation set on the basis of spatial playback information and a spatial distance or a spatial overlap between the first region and a second region in video frames of the first adaptation set, wherein the spatial playback information signals the server apparatus that the second region is linked to media segments of the second adaptation set; and, transmitting the second metadata or information for retrieving the second metadata to the client apparatus, the second metadata enabling the client apparatus to playback media segments of the second adaptation set after the playback of media segments of the first adaptation set. ## Claims ### Claim 1 A method for adaptive streaming of video data, the method comprising: a server apparatus transmitting first metadata to a client apparatus, the first metadata defining a first adaptation set, an adaptation set defining one or more sets of segment identifiers, each set of segment identifiers defining a plurality of linearly playable media segments of a predetermined resolution, each media segment in a set of media segments having an identical playback duration, the first metadata enabling a client apparatus to request media segments and sequentially playout the media segments; the server apparatus receiving navigation information from the client apparatus, the navigation information defining at least a first region in video frames of the first adaptation set; the server apparatus selecting second metadata defining a second adaptation set on the basis of spatial playback information and a spatial distance or a spatial overlap between the first region and a second region in the video frames of the first adaptation set, wherein the spatial playback information signals the server apparatus that the second region in the video frames of the first adaptation set is linked to media segments of the second adaptation set; and the server apparatus transmitting the second metadata or information for retrieving the second metadata to the client apparatus, the second metadata enabling the client apparatus to playback media segments of the second adaptation set after the playback of media segments of the first adaptation set. ### Claim 2 (depends on 1) The method according to claim 1 further comprising: the server apparatus receiving one or more request messages from the client apparatus or receiving information associated with the one or more request messages, the one or more request messages signaling the server apparatus to transmit segments of the first adaptation set to the client apparatus. ### Claim 3 (depends on 2) The method according to claim 2 wherein the navigation information and a client identifier are transmitted in one or more request messages to the server apparatus. ### Claim 4 (depends on 1) The method according to claim 1, wherein the navigation information is transmitted over a separate bi-directional communication channel to the server apparatus. ### Claim 5 (depends on 1) The method according to claim 1, wherein the first metadata defines a set of adaptation sets, each adaptation set defining media segments of a tile stream, wherein video frames of a tile stream define a similar spatial subpart, tile, of a video picture. ### Claim 6 (depends on 5) The method according to claim 5, wherein the server apparatus determines the navigation information on the basis of the tile streams requested by the client apparatus. ### Claim 7 (depends on 1) The method according to claim 1, wherein the segment identifiers defined by the first metadata are associated with a client identifier or information for determining a client identifier. ### Claim 8 (depends on 1) The method according to claim 1, wherein the spatial playback information further includes location information of the first region, the location information including coordinates associated with a location of the first region in one or more video frames of the first adaptation set or information to determine coordinates of the first region in one or more video frames of the first adaptation set. ### Claim 9 (depends on 8) The method according to claim 8 wherein the spatial playback information further includes a dynamic region parameter, the dynamic region parameter signaling the server apparatus that the location of the first region changes as a function of time. ### Claim 10 (depends on 1) The method according to claim 1, wherein the spatial playback information defines a plurality of regions in video frames of the at least one first adaptation set and a plurality of destination identifiers associated with the plurality of regions, each destination identifier signaling the server apparatus that a region associated with a destination identifier is linked to a further adaptation set, the server apparatus selecting the second adaptation set from a plurality of further adaptation sets on the basis of the locations of the regions and the location of the at least a second region. ### Claim 11 (depends on 1) The method according to claim 1, wherein the adaptive streaming is based on an HTTP adaptive streaming protocol, the protocol being one of: MPEG Dynamic Adaptive Streaming over HTTP (MPEG DASH), or 3GPP DASH; and wherein a playback period is defined as a Period as defined in the MPEG DASH standard or a MPEG Common Media Application Format (CMAF) Presentation as defined in the CMAF standard, a Period comprising one or more Adaptation Sets, an Adaptation Set including one or more Representations, a Representation including segments comprising video data of a predetermined video quality and CMAF Presentation comprising one or more CMAF Switching Sets, a CMAF Switching Set including one or more CMAF Tracks, a Track including CMAF fragments comprising video data of a predetermined video quality. ### Claim 12 (depends on 1) The method according to claim 1, wherein the video data include spherical video data, the spherical video data defining pixels on a curved surface, wherein a region of interest is associated with a viewing direction of a user viewing the video playback of the spherical video data, the region of interest and the first region being defined on the basis of a spherical coordinate system. ### Claim 13 (depends on 1) The method according to claim 1, wherein a playback period is defined as a Period as defined in the MPEG DASH standard or a MPEG Common Media Application Format (CMAF) Presentation as defined in the CMAF standard, a Period comprising one or more Adaptation Sets, an Adaptation Set including one or more Representations, a Representation including segments comprising video data of a predetermined video quality and CMAF Presentation comprising one or more CMAF Switching Sets, a CMAF Switching Set including one or more CMAF Tracks, a Track including CMAF fragments comprising video data of a predetermined video quality. ### Claim 14 A client apparatus comprising: a computer readable storage medium having computer readable program code embodied therewith, and a processor coupled to the computer readable storage medium, wherein responsive to executing the computer readable program code, the processor is configured to perform executable operations comprising: receiving metadata from a server apparatus, the metadata defining a first adaptation set, an adaptation set defining one or more sets of segment identifiers, each set of segment identifiers defining a plurality of linearly playable media segments of a predetermined resolution, each media segment in a set of media segments preferably having an identical playback duration, the first metadata enabling the client apparatus to request media segments and to sequentially playout the media segments of the set; selecting one or more segment identifiers on the basis of the metadata and transmitting one or more requests comprising the one or more segment identifiers and a client identifier to a server apparatus; transmitting navigation information to the server apparatus, the navigation information defining at least a region in the video frames of the first adaptation set; and receiving second metadata or information for retrieving the second metadata, the second metadata enabling the client apparatus to playback media segments of the second adaptation set after the playback of the first adaptation set. ### Claim 15 A server apparatus configured to generate metadata for a client apparatus, the metadata being configured to enable streaming of media to the client apparatus, the server apparatus comprising: a computer readable storage medium having computer readable program code embodied therewith, and a processor coupled to the computer readable storage medium, wherein responsive to executing the computer readable program code, the processor is configured to perform executable operations comprising: transmitting first metadata to a client apparatus, the first metadata defining a first adaptation set, an adaptation set defining one or more sets of segment identifiers, each set of segment identifiers defining a plurality of linearly playable media segments of a predetermined resolution, each media segment in a set of media, the first metadata enabling a client apparatus to request media segments and sequentially playout the media segments; receiving navigation information from the client apparatus, the navigation information defining at least a first region in video frames of the first adaptation set; selecting second metadata defining a second adaptation set on the basis of spatial playback information and a spatial distance or a spatial overlap between the first region and a second region in the video frames of the first adaptation set, wherein the spatial playback information signals the server apparatus that the second region in the video frames of the first adaptation set is linked to media segments of the second adaptation set; and transmitting the second metadata or information for retrieving the second metadata to the client apparatus, the second metadata enabling the client apparatus to playback media segments of the second adaptation set after the playback of media segments of the first adaptation set. ### Claim 16 A non-transitory computer-readable medium having instructions stored thereon adaptive streaming of video data that, when executed by one or more processors of a server apparatus, cause the server apparatus to carry out operations including: transmitting first metadata to a client apparatus, the first metadata defining a first adaptation set, an adaptation set defining one or more sets of segment identifiers, each set of segment identifiers defining a plurality of linearly playable media segments of a predetermined resolution, each media segment in a set of media segments having an identical playback duration, the first metadata enabling a client apparatus to request media segments and sequentially playout the media segments; receiving navigation information from the client apparatus, the navigation information defining at least a first region in video frames of the first adaptation set; selecting second metadata defining a second adaptation set on the basis of spatial playback information and a spatial distance or a spatial overlap between the first region and a second region in the video frames of the first adaptation set, wherein the spatial playback information signals the server apparatus that the second region in the video frames of the first adaptation set is linked to media segments of the second adaptation set; and transmitting the second metadata or information for retrieving the second metadata to the client apparatus, the second metadata enabling the client apparatus to playback media segments of the second adaptation set after the playback of media segments of the first adaptation set.