# US 12,192,595 B2 · Automatically processing content streams for insertion points > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,595 B2 - Document ID: US12192595B2 - Status: Granted - Kind: B2 - Application number: 18/488,390 - Filed: 2023-10-17 - Published: 2024-02-08 - Granted: 2025-01-07 - Earliest claimed: 2018-09-04 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Varun Ram; Ki Myung Han; Meera Jindal; Viriya Ratanasangpunth; Chris Price - Web page: https://patentgrep.com/patent/US12192595B2 ## CPC - H04N 21/8455: {involving pointers to the content, e.g. pointers to the I-frames of the video stream} - H04N 21/23106: {involving caching operations (prefetching while addressing of a memory level in which the access to the desired data or data block requires associative addressing means within memory systems or architectures G06F12/0862; caching at an intermediate stage in a data network H04L67/568)} - H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement} - H04N 21/23614: {Multiplexing of additional data and video streams (arrangements for simultaneous broadcast of plural pieces of information H04H20/28)} - H04N 21/2393: {involving handling client requests (scheduling or organising the servicing of application requests H04L67/60)} - H04N 21/44226: {on social networks} - H04N 21/8405: represented by keywords - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A video packaging and origination service can process requests for content segments from requesting user devices. The video packaging and origination service can processing video attributes, audio attributes and social media feeds to dynamically determine insertion points for supplemental content. Additionally, the video packaging and origination service can identify supplemental content utilizing the same attribute information. ## Claims ### Claim 1 A system to transmit content comprising: one or more computing devices associated with a video packaging and origination service, wherein the video packaging and origination service is configured to: encode received content into a set of encoded content segments, the received content including markers that correspond to manually determined insertion points for insertion of supplemental content; receive content requests from a user device; determine video and audio attributes of sequential segments of the encoded content segments; characterize one or more segments of the sequential segments as indicative of a transition based on differences between the determined video and audio attributes of the sequential segments, wherein the transitions are indicative of locations in the set of encoded content for insertion of supplemental content; dynamically determine insertion points for insertion of the supplemental content in the set of encoded content segments based on the transitions; form a set of insertion points for the insertion of the supplemental content, wherein the set of insertion points includes at least one dynamically determined insertion point and at least one manually determined insertion point; render the encoded content segments; identify a subset of a plurality of detectable objects based on processing the set of encoded content segments using a machine-learning algorithm to detect specific objects within the set of encoded content segments, the identified objects associated with the supplemental content; and prioritize, in the set of insertion points, the insertions points that can be matched with the identified objects associated with the supplemental content over the insertion points that do not match with the identified objects associated with the supplemental content. ### Claim 2 (depends on 1) The system of claim 1, wherein the video packaging and origination service is further configured to train the machine learning algorithms to identify the plurality of detectable objects. ### Claim 3 (depends on 1) The system of claim 1, wherein the dynamically determined insertion points are different from the manually determined insertion points. ### Claim 4 (depends on 1) The system of claim 1, wherein the video packaging and origination service is further configured to generate content manifest that identifies a first portion of content corresponding to the requested content and a second portion of content corresponding to the supplemental content. ### Claim 5 (depends on 1) The system of claim 1, wherein the video attribute corresponds to a determination of a scene change associated with a sequence of the encoded content segments and the audio attribute corresponds to a determination of an expression of sentiment based on detected keywords. ### Claim 6 A computer-implemented method to manage delivery of encoded content segments comprising: receiving content requests for encoded content from one or more computing devices, the encoded content including manually configured markers for insertion of supplemental content; determining video and audio attributes of sequential segments of the encoded content segments; characterizing one or more segments of the sequential segments as indicative of a transition based on differences between the determined video and audio attributes of the sequential segments, wherein dynamically determined insertion points are based on the transitions, and are indicative of locations in the set of encoded content for insertion of the supplemental content; forming a set of insertion points for the insertion of the supplemental content, wherein the set of insertion points includes at least one dynamically determined insertion point and at least one manually configured marker; rendering the encoded content segments; utilizing machine learning algorithms to identify objects in the rendered encoded content segments, the identified objects associated with the supplemental content; and prioritizing, in the set of insertion points, the insertions points that can be matched with the identified objects associated with the supplemental content over the insertion points that do not match with the identified objects associated with the supplemental content. ### Claim 7 (depends on 6) The computer-implemented method of claim 6 further comprising dynamically modifying a frequency of occurrence of insertion points based at least in part on at least one of content type, user request, and content provider criteria. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, wherein dynamically modifying the frequency of occurrence of insertion points is further based on social media information. ### Claim 9 (depends on 6) The computer-implemented method of claim 6 further comprising generating a content manifest that identifies a first portion of content corresponding to the requested content and a second portion of content corresponding to the supplemental content. ### Claim 10 (depends on 6) The computer-implemented method of claim 6 further comprising generating a content manifest that identifies a listing of available encoding bitrates or bitrate/format combinations for a first encoded segment of the requested content. ### Claim 11 (depends on 6) The computer-implemented method of claim 6 further comprising generating a content manifest for transmission to the one or more computing devices. ### Claim 12 (depends on 11) The computer-implemented method of claim 11, wherein a first portion of the content manifest corresponds to the requested content. ### Claim 13 (depends on 11) The computer-implemented method of claim 11, wherein a second portion of the content manifest corresponds to the supplemental content. ### Claim 14 (depends on 6) The computer-implemented method of claim 6, wherein dynamically determining insertion points includes bypassing a manually inserted insertion point. ### Claim 15 (depends on 6) The computer-implemented method of claim 6, wherein the dynamically determined insertion points are further based on at least one social media input. ### Claim 16 (depends on 6) The computer-implemented method of claim 6, wherein audio attributes include a closed caption feed. ### Claim 17 A computer-implemented method to manage delivery of encoded content segments comprising: receiving content requests for encoded content from one or more computing devices, the encoded content including markers that correspond to manually determined insertion points for insertion of supplemental content; determining video and audio attributes of sequential segments of the encoded content segments; characterizing one or more segments of the sequential segments as indicative of a transition based on differences between the determined video and audio attributes of the sequential segments; dynamically determining insertion points for insertion of the supplemental content in the set of encoded content segments based on the transitions forming a set of insertion points for the insertion of the supplemental content, wherein the set of insertion points includes at least one dynamically determined insertion point and at least one manually determined insertion point; rendering the encoded content segments; utilizing machine learning algorithms to detect objects in the rendered encoded content segments, wherein the machine learning algorithms are trained to identify a subset of the detectable objects, wherein the subset of detectable objects are associated with the supplemental content; and prioritizing, in the set of insertion points, the insertions points that can be matched with the subset of detectable objects associated with the supplemental content over the insertion points that do not match with the subset of detectable objects associated with the supplemental content. ### Claim 18 (depends on 17) The computer-implemented method of claim 17, wherein the learning algorithm include templates. ### Claim 19 (depends on 17) The computer-implemented method of claim 17 further comprising dynamically modifying a frequency of occurrence of insertion points based at least in part on at least one of content type, user request, and content provider criteria. ### Claim 20 (depends on 17) The computer-implemented method of claim 17 further comprising generating a content manifest that identifies a listing of available encoding bitrates or bitrate/format combinations for a first encoded segment of the requested content.