# US 12,120,396 B2 · Dynamic integration of customized supplemental media content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,120,396 B2 - Document ID: US12120396B2 - Status: Granted - Kind: B2 - Application number: 16/627,981 - Filed: 2019-05-15 - Published: 2021-10-28 - Granted: 2024-10-15 - Earliest claimed: 2019-05-15 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Henry Scott-Green - Web page: https://patentgrep.com/patent/US12120396B2 ## CPC - H04N 21/2668: Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles {(information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063)} - H04N 21/4756: {for rating content, e.g. scoring a recommended movie} - G06N 20/00: Machine learning - H04N 21/2187: Live feed - H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement} - H04N 21/2393: {involving handling client requests (scheduling or organising the servicing of application requests H04L67/60)} - H04N 21/25891: {being end-user preferences (retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)} - H04N 21/2743: Video hosting of uploaded data from client - H04N 21/4667: {Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections} - H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)} - H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Methods and systems for providing requested media items with supplemental media items customized for requesting users are described herein. A processing device receives a request of a first user for a first media item of a second user. The processing device determines that the first media item of the second user corresponds to a plurality of second media items of the second user. A set of rules is identified on how to select a second media item, from the plurality of second media items, to be integrated with the first media item of the second user, the set of rules being provided by the second user and comprising one or more conditions pertaining to the first user. The second media item is selected from the plurality of second media items based on the identified set of rules. The processing devices transmits the first media item and the selected second media item to a first user device of the first user, where the selected second media item is to be integrated with the first media item for presentation to the first user. ## Claims ### Claim 1 A computer-implemented method comprising: receiving, at a server, a request of a first user for a first media item of a second user, the first media item including at least one of video content or audio content; determining that the first media item of the second user corresponds to a plurality of second media items of the second user, each of the plurality of second media items including at least one of video content or audio content; identifying a set of rules on how to select a second media item, from the plurality of second media items, to be integrated with the first media item of the second user, the set of rules being provided by the second user and comprising one or more conditions pertaining to the first user; selecting, from the plurality of second media items, the second media item based on the set of rules; and transmitting the first media item and the selected second media item to a first user device of the first user, wherein the selected second media item is to be integrated with the first media item for presentation to the first user. ### Claim 2 (depends on 1) The method of claim 1, wherein each of the plurality of second media items comprises distinct video or audio content. ### Claim 3 (depends on 1) The method of claim 1, wherein the set of rules on how to select a second media item to be integrated with the first media item of the second user further comprises a rule to select the second media item based on a set of rankings provided by the second user, where each of the plurality of second media items is associated with a ranking of the set of rankings. ### Claim 4 (depends on 3) The method of claim 3, wherein selecting the second media item based on the set of rules comprises: identifying, from the plurality of second media items, the second media item that is associated with a highest ranking; and determining that the one or more conditions pertaining to the first user are satisfied with respect to the second media item. ### Claim 5 (depends on 3) The method of claim 3, wherein selecting the second media item based on the set of rules comprises: identifying, from the plurality of second media items, a third media item that is associated with a highest ranking; determining that the one or more conditions pertaining to the first user are not satisfied with respect to the third media item; identifying, from the plurality of second media items, the second media item that is associated with a lower ranking than the third media item associated with the highest ranking; and determining that the one or more conditions pertaining to the first user are satisfied with respect to the second media item. ### Claim 6 (depends on 1) The method of claim 1, wherein the first user device is to render the first media item and the selected second media item as a combined media item, wherein the selected second media item is to be integrated with the first media item, and wherein a transition between the first media item and the second media item is not visible to the first user. ### Claim 7 (depends on 1) The method of claim 1, further comprising: determining a time-based index of a timeline of the first media item, wherein the time-based index indicates a location on the timeline of the first media item where the content of the selected second media item is to be presented to the first user; and transmitting the time-based index with the first media item and the selected second media item to the first user device of the first user, wherein the selected second media item is to be integrated into the first media item at the location corresponding with the time-based index on the timeline of the first media item. ### Claim 8 (depends on 7) The method of claim 7, wherein the time-based index of the timeline of the first media item is created based on interaction of the second user with the timeline of the first media item. ### Claim 9 (depends on 7) The method of claim 7, wherein the time-based index of the timeline of the first media item is determined using a trained machine learning model. ### Claim 10 (depends on 1) The method of claim 1, wherein the first media item comprises a live media stream, and wherein transmitting the first media item and the selected second media item to the first user device of the first user comprises: transmitting the live media stream to the first user device of the first user; receiving, from a second user device of the second user, an indication to integrate the selected second media item with the live media stream; and responsive to receiving the indication to integrate the selected second media item with the live media stream, transmitting the selected second media item to the first user device of the first user, wherein the selected second media item is to be integrated with the live media stream. ### Claim 11 A system comprising: a memory; and a processing device coupled to memory, the processing device to: receive a request of a first user for a first media item of a second user; determine that the first media item of the second user corresponds to a plurality of second media items of the second user; identify a set of rules on how to select a second media item, from the plurality of second media items, to be integrated with the first media item of the second user the set of rules being provided by the second user and comprising one or more conditions pertaining to the first user; select, from the plurality of second media items, a second media item based on the set of rules; and transmit the first media item and the selected second media item to a first user device of the first user, wherein the selected second media item is to be integrated with the first media item for presentation to the first user. ### Claim 12 (depends on 11) The system of claim 11, wherein each of the plurality of second media items comprises distinct video or audio content. ### Claim 13 (depends on 11) The system of claim 11, wherein the set of rules on how to select a second media item to be integrated with the first media item of the second user further comprises a rule to select the second media item based on a set of rankings provided by the second user, where each of the plurality of second media items is associated with a ranking of the set of rankings. ### Claim 14 (depends on 13) The system of claim 13, wherein to select the second media item based on the set of rules, the processing device is to: identify, from the plurality of second media items, the second media item that is associated with a highest ranking; and determine whether the one or more conditions pertaining to the first user are satisfied with respect to the second media item. ### Claim 15 (depends on 13) The system of claim 13, wherein to select the second media item based on the set of rules, the processing device is to: identify, from the plurality of second media items, a third media item that is associated with a highest ranking; determine that the one or more conditions pertaining to the first user are not satisfied with respect to the third media item; identify, from the plurality of second media items, the second media item that is associated with a lower ranking than the third media item associated with the highest ranking; and determine that the one or more conditions pertaining to the first user are satisfied with respect to the second media item. ### Claim 16 A non-transitory computer-readable storage medium including instructions that, when executed by a processing device, cause the processing device to perform operations comprising: receiving a request of a first user for a first media item of a second user, the first media item including at least one of video content or audio content; determining that the first media item of the second user corresponds to a plurality of second media items of the second user, each of the plurality of second media items including at least one of video content or audio content; identifying a set of rules on how to select a second media item, from the plurality of second media items, to be integrated with the first media item of the second user, the set of rules being provided by the second user and comprising one or more conditions pertaining to the first user; selecting, from the plurality of second media items, the second media item based on the set of rules; and transmitting the first media item and the selected second media item to a first user device of the first user, wherein the selected second media item is to be integrated with the first media item for presentation to the first user. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein each of the plurality of second media items comprises distinct video or audio content. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the set of rules on how to select a second media item to be integrated with the first media item of the second user further comprises a rule to select the second media item based on a set of rankings provided by the second user, where each of the plurality of second media items is associated with a ranking of the set of rankings. ### Claim 19 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein selecting the second media item based on the set of rules comprises: identifying, from the plurality of second media items, the second media item that is associated with a highest ranking; and determining that the one or more conditions pertaining to the first user are satisfied with respect to the second media item. ### Claim 20 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein selecting the second media item based on the set of rules comprises: identifying, from the plurality of second media items, a third media item that is associated with a highest ranking; determining that the one or more conditions pertaining to the first user are not satisfied with respect to the third media item; identifying, from the plurality of second media items, the second media item that is associated with a lower ranking than the third media item associated with the highest ranking; and determining that the one or more conditions pertaining to the first user are satisfied with respect to the second media item.