# US 12,041,284 B2 · Methods, devices, and systems for publishing key pictures > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,041,284 B2 - Document ID: US12041284B2 - Status: Granted - Kind: B2 - Application number: 17/552,916 - Filed: 2021-12-16 - Published: 2023-05-11 - Granted: 2024-07-16 - Earliest claimed: 2021-11-05 - Assignees: Synamedia Limited - Applicants: Synamedia Limited - Inventors: Rahul Banerjee - Web page: https://patentgrep.com/patent/US12041284B2 ## CPC - H04N 21/26258: {for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40)} - H04N 21/26266: {for determining content or additional data repetition rate, e.g. of a file in a DVB carousel according to its importance (arrangements for broadcast or for distribution of identical information repeatedly in broadcast distribution systems H04H20/16)} - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - H04N 21/2402: {Monitoring of the downstream path of the transmission network, e.g. bandwidth available (traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876)} - H04N 21/2662: Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities - H04N 21/8455: {involving pointers to the content, e.g. pointers to the I-frames of the video stream} - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Techniques for publishing key pictures at a server and using the key picture information for operations on a client device are described herein. In accordance with various embodiments, the server creates an intermediate unit including key picture(s) from media content. The server then generates map(s) specifying for each key picture, a type, properties, and an association to the intermediate unit. The server also packages the intermediate unit into key picture unit(s) according to the map(s). The client device obtains a manifest for the media content referencing the key picture(s) and the type of each key picture. Upon receiving a request to perform operation(s) on a respective key picture, the client device obtains a corresponding key picture unit and parses the manifest to determine a respective type. Based on the respective type matches a criterion, the client device performs the operation(s) on the key picture unit. ## Claims ### Claim 1 A method comprising: at a server including one or more processors and a non-transitory memory: creating an intermediate unit from media content and identifying multiple key pictures within the intermediate unit that are of different types, including a first key picture that is of a first type and a second key picture that is of a second type, different from the first type; recording one or more maps while creating the intermediate unit and upon identifying the multiple key pictures, wherein the one or more maps specify a type of each of the multiple key pictures, properties of the multiple key pictures within the intermediate unit, and an association of each of the multiple key pictures to the intermediate unit, including indicating whether the type is the first type or the second type; and packaging the intermediate unit into multiple key picture units for streaming according to the one or more maps, including specifying in a manifest facilitating the streaming a respective type of a respective key picture within each of the multiple key picture units. ### Claim 2 (depends on 1) The method of claim 1, wherein the first type indicates an I-picture and the second type indicates an instantaneous decoding refresh (IDR) picture. ### Claim 3 (depends on 1) The method of claim 1, wherein the properties of the multiple key pictures include one or more of a start offset of a respective key picture, a size of the respective key picture, and a timestamp associated with the respective key picture. ### Claim 4 (depends on 1) The method of claim 1, wherein the media content is linear content, and the multiple key pictures specified in the one or more maps are within a time shift buffer depth for the linear content. ### Claim 5 (depends on 1) The method of claim 1, wherein the media content is a video-on-demand (VOD) media content item including a plurality of segments, and the one or more maps further include mappings of a plurality of key pictures to a plurality of intermediate units corresponding to the plurality of segments. ### Claim 6 (depends on 1) The method of claim 1, wherein generating the one or more maps includes: generating a mapping file for a bandwidth, wherein the properties in the mapping file specify a start offset for a respective key picture of the multiple key pictures packaged according to the bandwidth. ### Claim 7 (depends on 6) The method of claim 6, further comprising: receiving a key picture request for a respective key picture of the multiple key pictures, the key picture request includes bandwidth parameters; and locating the mapping file corresponding to the bandwidth parameters. ### Claim 8 (depends on 1) The method of claim 1, wherein packaging the intermediate unit into the multiple key picture units includes: receiving a request for a respective key picture of the multiple key pictures; fetching a portion including the respective key picture from the intermediate unit based on the one or more maps in response to the request; and constructing a key picture unit based on the portion and providing the key picture unit. ### Claim 9 (depends on 8) The method of claim 8, wherein the request referencing the respective key picture is originated from a client device and includes one or more of a trick mode play, a thumbnail scrubbing, seeking a safe seek point, seeking to a latest unit of live content, and inserting targeted content. ### Claim 10 (depends on 1) The method of claim 1, further comprising publishing the type of the key picture for each of the multiple key pictures in the manifest according to the one or more maps, including: inserting in the manifest a reference to each of the multiple key pictures and the type of each of the multiple key pictures. ### Claim 11 (depends on 10) The method of claim 10, wherein the manifest is an HTTP Live Streaming (HLS) playlist or a Dynamic Adaptive Streaming over HTTP (DASH) Media Presentation Description (MPD) file. ### Claim 12 A method comprising: at a client device including a processor and a non-transitory memory: obtaining a manifest for media content, wherein the manifest references multiple key pictures that are of different types in the media content and specifies a type of each of the multiple key pictures, wherein the type is indicated upon identifying the multiple key pictures when creating an intermediate unit from the media content for packaging into key picture units; receiving a request to perform at least one operation on a respective key picture of the multiple key pictures; obtaining a key picture unit of the multiple key picture units corresponding to the respective key picture and parsing the manifest to determine a respective type of the respective key picture in response to the request; and performing the at least one operation on the key picture unit in accordance with a determination that the respective type of the respective key picture matches a criterion. ### Claim 13 (depends on 12) The method of claim 12, wherein: the request to perform the at least one operation includes a trick mode play or a thumbnail scrubbing to access the respective key picture; and the criterion specifies the respective type of the respective key picture is an I-picture or an IDR picture. ### Claim 14 (depends on 12) The method of claim 12, wherein: the request to perform the at least one operation includes seeking a safe seek point; and the criterion specifies the respective type of the respective key picture is an IDR picture. ### Claim 15 (depends on 12) The method of claim 12, wherein: the media content is live content; the request to perform the at least one operation includes seeking to a latest unit of the live content; the criterion specifies the respective type of the respective key picture is an IDR picture; and performing the at least one operation includes: identifying the respective key picture being a last key picture within the latest unit; and seeking to the last key picture within the latest unit in accordance with the determination that the respective type of the respective key picture is the IDR picture. ### Claim 16 (depends on 12) The method of claim 12, wherein: the request to perform the at least one operation includes inserting targeted content starting at the respective key picture; the criterion specifies the respective type of the respective key picture is an IDR picture; and performing the at least one operation includes: identifying a first timestamp of the respective key picture and a second timestamp of an IDR picture within a unit according to the manifest; and inserting the targeted content within the unit between the first timestamp and the second timestamp. ### Claim 17 (depends on 12) The method of claim 12, further comprising: receiving the key picture unit within the media content according to the manifest, wherein the key picture unit includes the respective key picture and is generated by a packager at a server fetching a portion from the intermediate unit including the respective key picture according to a map and packaging the portion into the key picture unit. ### Claim 18 A system comprising: a server including an intermediate packager and a just-in-time (JIT) packager, wherein: the intermediate packager is operable to create an intermediate unit from media content and identify multiple key pictures within the intermediate unit that are of different types, including a first key picture that is of a first type and a second key picture that is of a second type, different from the first type, and record one or more maps while creating the intermediate unit and upon identifying the multiple key pictures, wherein the one or more maps specify a type of each of the multiple key pictures, properties of the multiple key pictures within the intermediate unit, and an association of each of the multiple key pictures to the intermediate unit, including indicating whether the type is the first type or the second type; and the JIT packager is operable to package the intermediate unit into multiple key picture units for streaming according to the one or more maps, including specifying in a manifest facilitating the streaming a respective type of a respective key picture within each of the multiple key picture units; and a client device including a processor and a non-transitory memory and operable to: obtain a manifest for the media content, wherein the manifest references the multiple key pictures in the media content and specifies the type of each of the multiple key pictures; receive a request to perform at least one operation on a respective key picture of the multiple key pictures; obtain a key picture unit of the multiple key picture units corresponding to the respective key picture and parse the manifest to determine a respective type of the respective key picture in response to the request; and perform the at least one operation on the key picture unit in accordance with a determination that the respective type of the respective key picture matches a criterion. ### Claim 19 (depends on 18) The system of claim 18, wherein the type of each of the multiple key pictures indicates an I-picture or an instantaneous decoding refresh (IDR) picture. ### Claim 20 (depends on 18) The system of claim 18, wherein the properties of the multiple key pictures include one or more of a start offset of a respective key picture, a size of the respective key picture, and a timestamp associated with the respective key picture.