# US 12,225,267 B2 · Leveraging insights from real-time media stream in delayed versions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,225,267 B2 - Document ID: US12225267B2 - Status: Granted - Kind: B2 - Application number: 17/818,112 - Filed: 2022-08-08 - Published: 2024-02-08 - Granted: 2025-02-11 - Earliest claimed: 2022-08-08 - Assignees: Phenix Real Time Solutions, Inc. - Applicants: Phenix Real Time Solutions, Inc. - Inventors: Fabián E. Bustamante; Stefan Birrer - Web page: https://patentgrep.com/patent/US12225267B2 ## CPC - H04N 21/4667: {Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections} - H04N 21/23439: {for generating different versions} - H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)} - H04N 21/83: Generation or processing of protective or descriptive data associated with content; Content structuring - H04N 21/8547: involving timestamps for synchronizing content - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A set of techniques leverages a real-time version of a media stream to gather information on its content which information is used to improve or augment the service delivery of other versions (e.g., live or video-on-demand) of the same stream. These techniques take advantage of the fact that the real-time version of a stream is sufficiently ahead, in time, from any other version to enable the gathering and use of information on the stream content. ## Claims ### Claim 1 A method comprising: providing a real-time multimedia stream at multiple encoding rates including a higher-quality encoding and a lower-quality encoding, wherein each portion of the real-time multimedia stream is equally ranked in encoding delivery; during a broadcast of the real-time multimedia stream, receiving, a plurality of content tags at specified timestamps, wherein the plurality of content tags are indicative of a priority of encoding delivery; and providing a delayed multimedia stream that includes content corresponding to the real-time multimedia stream, wherein delivery of portions of the delayed multimedia stream that correspond to the plurality of content tags is prioritized via inserting additional camera angles of portions marked by content tags in series, wherein inserting the additional camera angles in series further delays the multimedia stream and affects a length of the delayed multimedia stream. ### Claim 2 (depends on 1) The method of claim 1, further comprising: generating, by a machine learning model, the plurality of content tags at the specified timestamps based on supervised training associated with a subject matter of the real-time multimedia stream, wherein the specified timestamps correlate to portions of the real-time multimedia stream of high interest to a viewing audience as determined by the supervised training. ### Claim 3 (depends on 2) The method of claim 2, wherein the supervised training is performed by model observations of an administrative user generating content tags in past real-time multimedia streams. ### Claim 4 (depends on 1) The method of claim 1, further comprising: generating, by a viewing audience of the real-time multimedia stream, the plurality of content tags at the specified timestamps based on aggregate use of social features associated with a user interface of the real-time multimedia stream. ### Claim 5 (depends on 1) The method of claim 1, wherein the plurality of content tags are generated by an administrative user. ### Claim 6 (depends on 1) The method of claim 1, further comprising: generating, by a viewing audience of the real-time multimedia stream, wherein the plurality of content tags at the specified timestamps are based on aggregate measured behavior of the viewing audience including any combination of: window precedence; eye gaze tracking; or operation of volume or muting controls. ### Claim 7 (depends on 1) The method of claim 1, further comprising: during the broadcast of the real-time multimedia stream, receiving, a plurality of downvote tags at a second set of specified timestamps, wherein the plurality of downvote tags are indicative of a decreased priority of encoding delivery, wherein delivery of portions of the delayed multimedia stream that correspond to the plurality of downvote tags is deemphasized via any of: downshifting encoding rates of downvoted content based on latency; or integrating textual information overlays into the delayed multimedia stream. ### Claim 8 (depends on 1) The method of claim 1, wherein the plurality of content tags are upvote tags and the priority of encoding delivery is an increased priority. ### Claim 9 (depends on 1) The method of claim 1, wherein the real-time multimedia stream includes multiple camera angle inputs that are interchanged into a presenting camera of the real-time multimedia stream during the broadcast, the method further comprising: modifying the content of the delayed multimedia stream based on the specified timestamps of the plurality of content tags via any of: inserting additional camera angles of portions marked by the plurality of content tags either in series or in frame share; or interchanging the presenting camera from a first camera of the multiple camera angle inputs to a second camera of the multiple camera angle inputs. ### Claim 10 (depends on 1) The method of claim 1, wherein the delayed multimedia stream is any of: video on demand; or delayed a specified number of seconds after the real-time multimedia stream. ### Claim 11 (depends on 1) The system of claim 1, wherein delivery of portions of the delayed multimedia stream that correspond to the plurality of content tags is prioritized via prebuffering or downshifting encoding rates of surrounding content. ### Claim 12 A method comprising: providing a first multimedia stream at multiple encoding rates including a higher-quality encoding and a lower-quality encoding, wherein the first multimedia stream includes multiple camera angle inputs that are interchanged into a presenting camera of the first multimedia stream during a broadcast, wherein each portion of the first multimedia stream is equally ranked in encoding delivery; during the broadcast of the first multimedia stream, receiving, a plurality of content tags at specified timestamps, the plurality of content tags are indicative of a priority of encoding delivery; and providing a delayed multimedia stream that includes content corresponding to the first multimedia stream, wherein delivery of portions of the delayed multimedia stream that correspond to the plurality of content tags is prioritized via inserting additional camera angles of portions marked by content tags in series, wherein inserting the additional camera angles in series further delays the multimedia stream and affects a length of the delayed multimedia stream. ### Claim 13 (depends on 12) The method of claim 12, further comprising: generating, by a machine learning model, the plurality of content tags at the specified timestamps based on supervised training associated with a subject matter of the first multimedia stream, wherein the specified timestamps correlate to portions of the first multimedia stream of high interest to a viewing audience as indicated by the supervised training. ### Claim 14 (depends on 13) The method of claim 13, wherein the supervised training is performed by model observations of an administrative user generating content tags in past multimedia streams. ### Claim 15 (depends on 12) The method of claim 12, further comprising: generating, by a viewing audience of the first multimedia stream, the plurality of content tags at the specified timestamps based on aggregate use of social features associated with a user interface of the first multimedia stream. ### Claim 16 (depends on 12) The method of claim 12, wherein the plurality of content tags are generated by an administrative user. ### Claim 17 (depends on 12) The method of claim 12, further comprising: generating, by a viewing audience of the first multimedia stream, the plurality of content tags at the specified timestamps based on aggregate measured behavior of the viewing audience including any combination of: window precedence; eye gaze tracking; or operation of volume or muting controls. ### Claim 18 (depends on 12) The method of claim 12, wherein the plurality of content tags are upvote tags and the priority of encoding delivery is an increased priority. ### Claim 19 (depends on 12) The method of claim 12, wherein delivery of portions of the delayed multimedia stream that correspond to the plurality of content tags is prioritized via interchanging the presenting camera from a first camera of the multiple camera angle inputs to a second camera of the multiple camera angle inputs, prebuffering or downshifting encoding rates of surrounding content. ### Claim 20 A method comprising: providing a real-time multimedia stream at multiple encoding rates including a higher-quality encoding and a lower-quality encoding, wherein each portion of the real-time multimedia stream is equally ranked in encoding delivery; during a broadcast of the real-time multimedia stream, receiving, a plurality of downvote tags at specified timestamps, wherein the plurality of downvote tags are indicative of a decreased priority of encoding delivery; and providing a delayed multimedia stream that includes content corresponding to the real-time multimedia stream, wherein delivery of portions of the delayed multimedia stream that correspond to the plurality of downvote tags is deemphasized via removing camera angles of portions marked by downvote tags and inserting additional camera angles marked by positive content tags, wherein inserting additional camera angles in series further delays the delayed multimedia stream. ### Claim 21 (depends on 20) The method of claim 20, further comprising: generating, by a machine learning model, the plurality of downvote tags at the specified timestamps based on supervised training associated with a subject matter of the real-time multimedia stream, wherein the specified timestamps correlate to portions of the real-time multimedia stream of high interest to a viewing audience as determined by the supervised training. ### Claim 22 (depends on 20) The method of claim 20, further comprising: generating, by a viewing audience of the real-time multimedia stream, the plurality of downvote tags at the specified timestamps based on aggregate use of social features associated with a user interface of the real-time multimedia stream. ### Claim 23 (depends on 20) The method of claim 20, wherein the plurality of downvote tags are generated by an administrative user. ### Claim 24 (depends on 20) The method of claim 20, further comprising: generating, by a viewing audience of the real-time multimedia stream, wherein the plurality of downvote tags at the specified timestamps are based on aggregate measured behavior of the viewing audience including any combination of: window precedence; eye gaze tracking; or operation of volume or muting controls. ### Claim 25 (depends on 20) The method of claim 20, wherein delivery of portions of the delayed multimedia stream that correspond to the plurality of downvote tags is deemphasized via downshifting encoding rates of downvoted content based on latency. ### Claim 26 A system comprising: a broadcast server that is configured to transmit a real-time multimedia stream at multiple encoding rates including a higher-quality encoding and a lower-quality encoding, wherein each portion of the real-time multimedia stream is equally ranked in encoding delivery; a streaming platform configured to receive a plurality of content tags at specified timestamps during broadcast of the real-time multimedia stream, wherein, the plurality of content tags are indicative of a priority of encoding delivery; and a memory storing a delayed multimedia stream that includes content corresponding to the real-time multimedia stream, wherein delivery of portions of the delayed multimedia stream that correspond to the plurality of content tags is prioritized via inserting additional camera angles of portions marked by content tags in series, wherein inserting the additional camera angles affects a length of the delayed stream. ### Claim 27 (depends on 26) The system of claim 26, further comprising: a machine learning model stored in the memory configured to generate the plurality of content tags at the specified timestamps based on supervised training associated with a subject matter of the real-time multimedia stream, wherein the specified timestamps correlate to portions of the real-time multimedia stream of high interest to a viewing audience as indicated by the supervised training. ### Claim 28 (depends on 27) The system of claim 27, wherein the supervised training is performed by model observations of an administrative user generating content tags in past real-time multimedia streams. ### Claim 29 (depends on 26) The system of claim 26, further comprising: a user viewing interface configured to generate, by a viewing audience of the real-time multimedia stream, the plurality of content tags at the specified timestamps based on aggregate use of social features of the user viewing interface. ### Claim 30 (depends on 26) The system of claim 26, wherein the plurality of content tags are generated by an administrative user. ### Claim 31 (depends on 26) The system of claim 26, further comprising: a user viewing interface configured to generate, by a viewing audience of the real-time multimedia stream, the plurality of content tags the specified timestamps based on aggregate measured behavior of the viewing audience including any combination of: window precedence; eye gaze tracking; or operation of volume or muting controls. ### Claim 32 (depends on 26) The system of claim 26, wherein the plurality of content tags are upvote tags and the priority of encoding delivery is an increased priority. ### Claim 33 (depends on 26) The system of claim 26, wherein delivery of portions of the delayed multimedia stream that correspond to the plurality of content tags is prioritized via prebuffering or downshifting encoding rates of surrounding content.