# US 11,445,245 B2 · Synchronized combinations of captured real-time media content with played-back digital content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,445,245 B2 - Document ID: US11445245B2 - Status: Granted - Kind: B2 - Application number: 17/336,533 - Filed: 2021-06-02 - Published: 2021-12-09 - Granted: 2022-09-13 - Earliest claimed: 2020-06-03 - Assignees: Caavo Inc - Applicants: Caavo Inc - Inventors: Andrew E. Einaudi; Ashish D. Aggarwal - Web page: https://patentgrep.com/patent/US11445245B2 ## CPC - H04N 21/44016: {involving splicing one content stream with another content stream, e.g. for substituting a video clip} - H04N 21/4126: {The peripheral being portable, e.g. PDAs or mobile phones} - H04N 21/42203: {sound input device, e.g. microphone} - H04N 21/4223: Cameras (H04N23/00 takes precedence) - H04N 21/43072: {of multiple content streams on the same device} - H04N 21/44004: {involving video buffer management, e.g. video decoder buffer or video display buffer} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)} - H04N 21/632: {using a connection between clients on a wide area network, e.g. setting up a peer-to-peer communication via Internet for retrieving video segments from the hard-disk of other client devices (broadcast-related systems characterised by transmission among terminal devices H04H60/80; distributed application using peer-to-peer [P2P] networks H04L67/104)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Methods, systems, and apparatuses are described for sharing content viewed by a user. Multimedia content may be presented by an output device to an audience. A selection of a portion of the multimedia content is selected for sharing by an audience member. Additionally, synchronized combinations of captured real-time media content with played-back digital content are generated by systems and devices. A media content stream from a source device that is to be played back by a sink device is buffered. Messages pertaining to the capture of a first media clip are received from a media capture device, and based on the messages, a portion of the buffered content stream is identified and extracted to generate a second media clip. Combined media content is generated by combining the first and the second media clips, and the combined media content is stored for later playback and sharing. ## Claims ### Claim 1 A method performed by a multimedia device for generating combined media content, comprising: continuously performing: receiving a media content stream from a source device; buffering a most recent portion of the media content stream; and providing the media content stream to a sink device configured to playback the media content stream to a user of the multimedia device; receiving one or more messages from a media capture device associated with a capturing, by the media capture device, of a first media clip; based on the one or more messages, identifying a portion of the most recent portion of the media content stream that is buffered, and extracting the portion therefrom, generating a second media clip; generating the combined media content by combining first media content of the first media clip and second media content of the second media clip; and sharing the combined media content with a device over a network. ### Claim 2 (depends on 1) The method of claim 1, wherein the multimedia device comprises a switching device, and wherein the media capture device is associated with the user and is at least one of a smart phone, a camera that includes a microphone and that includes artificial intelligence programming configured to determine at least one of a reaction or an emotion of one or more persons in its field of view, or a stand-alone camera with a microphone. ### Claim 3 (depends on 1) The method of claim 1, wherein the one or more messages, and the first media clip, are received from the media capture device by the multimedia device via an external network with respect to the multimedia device. ### Claim 4 (depends on 1) The method of claim 1, wherein the one or more messages include temporal information associated with the first media clip; wherein said identifying the portion of the buffered most recent portion of the media content stream is based at least on the temporal information and includes aligning a beginning and a duration of the portion with a corresponding beginning and a duration of the first media clip with respect to absolute time. ### Claim 5 (depends on 1) The method of claim 1, wherein said combining the first media clip and the second media clip includes: overlaying an area of one of the first media clip or the second media clip with the other one of the first media clip or the second media clip; or stitching the first media clip and the second media clip. ### Claim 6 (depends on 1) The method of claim 1, wherein said sharing the combined media content with a device over a network comprises at least one of: sharing with another multimedia device associated with a person that is different than the user; sharing with a user device of the person; sharing with the user device of the person via a user device of the user; sharing with the user device of the user; or sharing with a social media account of the user. ### Claim 7 (depends on 1) The method of claim 1, wherein said capturing by the media capture device of the first media clip is performed proactively via capture buffering of real-time video, and includes providing the first media clip to the multimedia device; or wherein said capturing by the media capture device of the first media clip is performed reactively based on a received command at the media capture device, and includes providing the first media clip to the multimedia device. ### Claim 8 A system comprising: a multimedia device that includes a memory storing program instructions for generating combined media content, and that includes one or more processors configured to execute the program instructions, which when executed, cause the multimedia device to: continuously during operation of a source device: receive a media content stream from the source device; buffer a most recent portion of the media content stream; and provide the media content stream to a sink device configured to playback the media content stream to a user of the multimedia device; receive one or more messages from a media capture device associated with a capture, by the media capture device, of a first media clip; based on the one or more messages, identify a portion of the most recent portion of the media content stream that is buffered, and extract the portion therefrom to generate a second media clip; generate the combined media content by combining first media content of the first media clip and second media content of the second media clip; and share the combined media content with a device over a network. ### Claim 9 (depends on 8) The system of claim 8, wherein the multimedia device comprises a switching device, and wherein the media capture device is associated with the user and is at least one of a smart phone, a camera that includes a microphone and that includes artificial intelligence programming configured to determine at least one of a reaction or an emotion of one or more persons in its field of view, or a stand-alone camera with a microphone. ### Claim 10 (depends on 8) The system of claim 8, wherein the one or more messages, and the first media clip, are received from the media capture device by the multimedia device via an external network with respect to the multimedia device. ### Claim 11 (depends on 8) The system of claim 8, wherein the one or more messages include temporal information associated with the first media clip; and wherein the program instructions, when executed, cause the multimedia device, for said identify the portion of the buffered most recent portion of the media content stream, to: identify based at least on the temporal information; and align a beginning and a duration of the portion with a corresponding beginning and a duration of the first media clip with respect to absolute time. ### Claim 12 (depends on 8) The system of claim 8, wherein the program instructions, when executed, cause the multimedia device, for said combine the first media clip and the second media clip, to: overlay an area of one of the first media clip or the second media clip with the other one of the first media clip or the second media clip; or stitch the first media clip and the second media clip. ### Claim 13 (depends on 8) The system of claim 8, wherein the program instructions, when executed, cause the multimedia device, for said share the combined media content with a device over a network, to perform at least one of: share with another multimedia device associated with a person that is different than the user; share with a user device of the person; share with the user device of the person via a user device of the user; share with the user device of the user; or share with a social media account of the user. ### Claim 14 (depends on 8) The system of claim 8, wherein said capture by the media capture device of the first media clip is performed proactively via capture buffering of real-time video, and includes providing the first media clip to the multimedia device; or wherein said capture by the media capture device of the first media clip is performed reactively based on a received command at the media capture device, and includes providing the first media clip to the multimedia device. ### Claim 15 A computer-readable storage medium having program instructions recorded thereon that, when executed by a processing device, perform a method, for generating combined media content, that comprises: continuously performing by a multimedia device: receiving a media content stream from a source device; buffering a most recent portion of the media content stream; and providing the media content stream to a sink device configured to playback the media content stream to a user of the multimedia device; receiving one or more messages from a media capture device associated with a capturing, by the media capture device, of a first media clip; based on the one or more messages, identifying a portion of the most recent portion of the media content stream that is buffered, and extracting the portion therefrom, generating a second media clip; generating the combined media content by combining first media content of the first media clip and second media content of the second media clip; and sharing the combined media content with a device over a network. ### Claim 16 (depends on 15) The computer-readable storage medium of claim 15, wherein the multimedia device comprises a switching device, and wherein the media capture device is associated with the user and is at least one of a smart phone, a camera that includes a microphone and that includes artificial intelligence programming configured to determine at least one of a reaction or an emotion of one or more persons in its field of view, or a stand-alone camera with a microphone. ### Claim 17 (depends on 15) The computer-readable storage medium of claim 15, wherein for the method, the one or more messages, and the first media clip, are received from the media capture device by the multimedia device via an external network with respect to the multimedia device. ### Claim 18 (depends on 15) The computer-readable storage medium of claim 15, wherein the one or more messages include temporal information associated with the first media clip; and wherein, for the method, said identifying the portion of the buffered most recent portion of the media content stream is based at least on the temporal information and includes aligning a beginning and a duration of the portion with a corresponding beginning and a duration of the first media clip with respect to absolute time. ### Claim 19 (depends on 15) The computer-readable storage medium of claim 15, wherein for the method, said combining the first media clip and the second media clip includes: overlaying an area of one of the first media clip or the second media clip with the other one of the first media clip or the second media clip; or stitching the first media clip and the second media clip. ### Claim 20 (depends on 15) The computer-readable storage medium of claim 15, wherein the said sharing the combined media content with a device over a network comprises at least one of: sharing with another multimedia device associated with a person that is different than the user; sharing with a user device of the person; sharing with the user device of the person via a user device of the user; sharing with the user device of the user; or sharing with a social media account of the user.