# US 11,310,567 B2 · Apparatus and methods for thumbnail generation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,310,567 B2 - Document ID: US11310567B2 - Status: Granted - Kind: B2 - Application number: 16/532,329 - Filed: 2019-08-05 - Published: 2020-01-30 - Granted: 2022-04-19 - Earliest claimed: 2015-04-14 - Assignees: Time Warner Cable Enterprises LLC - Applicants: Time Warner Cable Enterprises LLC - Inventors: Glen Hardin; Eric Manchester - Web page: https://patentgrep.com/patent/US11310567B2 ## CPC - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04L 65/611: for multicast or broadcast (systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06) - H04L 65/612: for unicast - H04L 65/70: Media network packetisation - H04L 65/762: {at the source (reformatting of additional data in video distribution servers H04N21/2355)} - H04N 21/440218: {by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 (conversion of standards in analogue television systems H04N7/01)} - H04N 21/8549: Creating video summaries, e.g. movie trailer {(retrieval in video databases by using presentations in form of a video summary G06F16/739)} - G11B 27/005: {Reproducing at a different information rate from the information rate of recording (for television signals H04N5/783)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) ## Abstract Apparatus and methods for thumbnail generation. In one embodiment, a thumbnail stream is generated where one or more bits are assigned to key frames and minimizes the number of bits assigned to delta frames. Each key frame may be used to extract a still image for use in thumbnails during trick mode operation (e.g., fast forward, rewind, pause, or random seek operation). When the encoded video file is given to a Just in Time Packager (JITP), the JITP extracts the image files (via the key frames). Information regarding how to reference these files is stored in generated manifest file and may be reformatted by the JITP to fit the needs of the current product. In a variant, the JITP reformats the image files extracted from the encoded video for operation on a number of devices. ## Claims ### Claim 1 Computerized encoder apparatus, the computerized encoder apparatus comprising: first data interface apparatus configured to communicate with a network; storage apparatus; and processor apparatus in data communication with the first data interface apparatus and the storage apparatus and configured to execute at least one computer program, the at least one computer program comprising a plurality of instructions which are configured to, when executed on the processor apparatus, cause the computerized encoder apparatus to: generate a plurality of video data from a content source for use with adaptive bitrate streaming, the plurality of video data comprising: a thumbnail stream comprising a first plurality of key frames separated by one or more null frames, the thumbnail stream configured to have the first plurality of key frames extracted and be transmitted to a computerized user device during a trick mode operation of the computerized user device; and at least one other encoded stream, the at least one other encoded stream comprising a second plurality of key frames separated by delta frames; wherein the generation of the plurality of video data comprises contemporaneous generation of the thumbnail stream and the at least one other encoded stream. ### Claim 2 (depends on 1) The computerized encoder apparatus of claim 1, wherein the first plurality of key frames of the thumbnail stream and the second plurality of key frames of the at least one other encoded stream are synchronized. ### Claim 3 (depends on 1) The computerized encoder apparatus of claim 1, wherein the at least one other encoded stream comprises a plurality of streams each associated with a different bit rate configured for use with the adaptive bit rate streaming. ### Claim 4 (depends on 1) The computerized encoder apparatus of claim 1, wherein the thumbnail stream and each stream of the at least one other encoded stream are packaged in separate transport stream files. ### Claim 5 (depends on 1) The computerized encoder apparatus of claim 1, wherein the at least one other encoded stream comprises an audio track. ### Claim 6 (depends on 1) The computerized encoder apparatus of claim 1, wherein the plurality of instructions which are further configured to, when executed on the processor apparatus, cause the computerized encoder apparatus to encode the thumbnail stream so as to leave only the first plurality of key frames, the extraction of the first plurality of key frames based on the encoded thumbnail stream. ### Claim 7 (depends on 1) The computerized encoder apparatus of claim 1, wherein the one or more null frames separating the first plurality of key frames comprise metadata descriptive of the first plurality of key frames. ### Claim 8 A non-transitory computer-readable apparatus comprising a storage medium, the storage medium comprising a plurality of instructions configured to, when executed on a processing apparatus, cause a computerized apparatus to: obtain data configured to enable adaptive bitrate streaming of video, the obtained data comprising: first data comprising a thumbnail data stream, the thumbnail data stream comprising a plurality of key frames at least partially interleaved with a plurality of null frames; second data comprising an encoded data stream, the encoded data stream comprising a second plurality of key frames, and a plurality of transitional frames; and third data comprising data indicative of locations of the plurality of key frames; based at least on an initiation of a trick mode operation, extract at least a portion of the plurality of key frames from the thumbnail data stream based at least on the third data; and enable display of a plurality of thumbnails, the plurality of thumbnails generated based at least on the extracted plurality of key frames; wherein the extraction of the at least portion of the plurality of key frames from the thumbnail data stream based at least on the third data comprises an automatic skipping of each of the plurality of null frames. ### Claim 9 (depends on 8) The non-transitory computer-readable apparatus of claim 8, wherein each of the plurality of null frames comprise non-content data, and each of the plurality of key frames is correlated to one or more of the plurality of null frames at least via the non-content data, the non-content data comprising metadata that relates the plurality of key frames to the plurality of null frames. ### Claim 10 (depends on 8) The non-transitory computer-readable apparatus of claim 8, wherein the plurality of key frames are separated by a prescribed temporal distance. ### Claim 11 (depends on 8) The non-transitory computer-readable apparatus of claim 8, wherein: at least some of the plurality of key frames are separated by one of (i) a first temporal distance or (ii) a second temporal distance different from the first temporal distance; and at least some of the second plurality of key frames are separated by another one of the first or second temporal distances. ### Claim 12 (depends on 8) The non-transitory computer-readable apparatus of claim 8, wherein the plurality of instructions are further configured to, when executed on the processing apparatus, cause the computerized apparatus to: enable selection among a plurality of bitrates, and apply the selected bitrate to the encoded data stream and the thumbnail data stream. ### Claim 13 (depends on 8) The non-transitory computer-readable apparatus of claim 8, wherein the second plurality of key frames are separated by the transitional frames by a prescribed temporal distance, the transitional frames comprising delta frames. ### Claim 14 A computerized method of processing a digital content stream, the computerized method comprising: obtaining a content stream and a thumbnail stream, the thumbnail stream comprising a plurality of key frames each separated by one or more null frames, the one or more null frames comprising metadata correlated to a corresponding one of the plurality of key frames, the thumbnail stream configured to enable extraction of at least a portion of the plurality of key frames by a computerized user device based at least on the metadata; and during operation associated with a trick mode, generating one or more thumbnails from the extracted at least portion of the plurality of key frames; wherein the extracted at least portion of the plurality of key frames are grouped into one or more chunks of digital data of dynamically adjustable length, the dynamically adjustable length based at least on one or more network conditions. ### Claim 15 (depends on 14) The computerized method of claim 14, wherein the obtaining of the content stream comprises obtaining a second plurality of key frames each separated by one or more transitional frames. ### Claim 16 (depends on 15) The computerized method of claim 15, further comprising: enabling selection from a plurality of adaptive bitrate levels; and applying the selected adaptive bitrate level to the thumbnail stream and the content stream. ### Claim 17 (depends on 15) The computerized method of claim 15, further comprising separating the plurality of key frames and the second plurality of key frames each by a selected temporal distance. ### Claim 18 (depends on 15) The computerized method of claim 15, further comprising separating the plurality of key frames are by a first temporal distance, and separating the second plurality of key frames are by a second temporal distance different from the first temporal distance. ### Claim 19 (depends on 14) The computerized method of claim 14, further comprising encoding the thumbnail stream, the encoding of the thumbnail stream comprising removing the one or more null frames so as to leave only the plurality of key frames and enable said extraction of the at least portion of the plurality of key frames. ### Claim 20 (depends on 14) The computerized method of claim 14, further comprising encoding the content stream, the encoding of the content stream comprising creating a plurality of versions of the content stream each having a different bitrate associated therewith. ### Claim 21 Computerized encoder apparatus, the computerized encoder apparatus comprising: first data interface apparatus configured to communicate with a network; storage apparatus; and processor apparatus in data communication with the first data interface apparatus and the storage apparatus and configured to execute at least one computer program, the at least one computer program comprising a plurality of instructions which are configured to, when executed on the processor apparatus, cause the computerized encoder apparatus to: generate a plurality of video data from a content source for use with adaptive bitrate streaming, the plurality of video data comprising: a thumbnail stream comprising a first plurality of key frames separated by one or more null frames, the thumbnail stream configured to have the first plurality of key frames extracted and be transmitted to a computerized user device; and at least one other encoded stream, the at least one other encoded stream comprising a second plurality of key frames separated by delta frames; wherein the generation of the plurality of video data is performed so as provide temporal synchronization between the thumbnail stream and a playback time code associated with the at least one other encoded stream.