# US 12,225,250 B2 · System and method for controlling data stream modifications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,225,250 B2 - Document ID: US12225250B2 - Status: Granted - Kind: B2 - Application number: 18/376,344 - Filed: 2023-10-03 - Published: 2024-01-25 - Granted: 2025-02-11 - Earliest claimed: 2019-02-01 - Assignees: Dejero Labs Inc. - Applicants: Dejero Labs Inc. - Inventors: David Sze; Jonathon Oberholzer; Todd Schneider; Bogdan Frusina - Web page: https://patentgrep.com/patent/US12225250B2 ## CPC - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - H04N 21/2343: involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements {(video transcoding H04N19/40; media packet handling at the source H04L65/762)} - G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} - H04L 65/765: {intermediate} - H04N 21/234: Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs - H04N 21/2365: Multiplexing of several video streams - H04N 21/242: Synchronisation processes, e.g. processing of PCR [Programme Clock References] {(arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18)} - H04N 21/4402: involving reformatting operations of video signals for household redistribution, storage or real-time display {(details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764)} - H04N 21/6332: directed to client - H04N 21/854: Content authoring - H04N 21/8549: Creating video summaries, e.g. movie trailer {(retrieval in video databases by using presentations in form of a video summary G06F16/739)} - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract In some embodiments, there is provided a system for remotely processing data streams including: at least one transmitter configured for transmission of at least two data streams, the at least two data streams including at least a lower quality preview stream and a higher quality content stream. A plurality of editor computing devices is configured for receiving at least the lower quality preview stream and facilitating processing and editing on the lower quality preview stream, generating a set of machine-readable instructions representative of the processing and editing. A plurality of routing computing devices located from the at least one transmitter is configured for receiving at least the higher quality content stream and the set of machine-readable instructions, processing the higher quality content stream by encoding the higher quality content stream based on the set of machine-readable instructions to generate an output content stream. ## Claims ### Claim 1 A system for controlling data stream modifications, the system comprising: a router configured to receive a first data stream, and to generate, based on the first data stream, one or more sets of processing instructions for modifying a second data stream; a data stream encoder configured to receive the second data stream and to apply one or more data modifications based at least on executing the one or more sets of processing instructions in processing the second data stream, the data stream encoder generating an output processed stream including the data modifications; wherein the first data stream and the second data stream are both derived from a source video data stream; wherein receipt of the corresponding frames of the first data stream and the corresponding frames of the second data stream is temporally shifted relative to one another at least in part due to one or more communications latency differences between the first data stream and the second data stream; and wherein the receipt of corresponding frames of the first data stream by the router occurs ahead of the receipt of corresponding frames of the second data stream by the data stream encoder. ### Claim 2 (depends on 1) The system of claim 1, wherein the communications latency differences are caused due to different geospatial characteristics associated with the first data stream and the second data stream. ### Claim 3 (depends on 2) The system of claim 2, wherein a duration associated with the communications latency differences is controllable through selection of geospatially diverse cloud computing resources for controlling data stream modifications. ### Claim 4 (depends on 3) The system of claim 3, wherein the duration associated with the communications latency differences is controlled based on a selection of geospatially diverse cloud computing resources for usage as the router. ### Claim 5 (depends on 3) The system of claim 3, wherein the duration associated with the communications latency differences is controlled based on a selection of geospatially diverse cloud computing resources for usage as the data stream encoder. ### Claim 6 (depends on 3) The system of claim 3, wherein the duration associated with the communications latency differences is controlled based on a selection of geospatially diverse cloud computing resources for usage as the router and for usage as the data stream encoder. ### Claim 7 (depends on 3) The system of claim 3, wherein the duration associated with the communications latency differences is controlled to satisfy a temporal constraint for executing the one or more sets of processing instructions in processing the second data stream. ### Claim 8 (depends on 7) The system of claim 7, wherein the temporal constraint for executing the one or more sets of processing instructions is scaled based on a processing complexity of the one or more sets of processing instructions. ### Claim 9 (depends on 2) The system of claim 2, wherein the communication latency differences are utilized to scale a processing complexity of the one or more sets of processing instructions generated by the router. ### Claim 10 (depends on 6) The system of claim 6, wherein the geospatially diverse cloud computing resources for usage as the router and usage as the data stream encoder are located in different time zones, and the different time zones are selected based on availability of computing resources during off-peak hours associated with each of the different time zones. ### Claim 11 A method for controlling data stream modifications, the method comprising: receiving, by a router, a first data stream; generating, by the router or a processing device coupled to the router, based on the first data stream, one or more sets of processing instructions for modifying a second data stream, the first data stream and the second data stream both derived from a source video data stream; receiving, by a data stream encoder, the second data stream, wherein receipt of the corresponding frames of the first data stream and the corresponding frames of the second data stream is temporally shifted relative to one another at least in part due to one or more communications latency differences between the first data stream and the second data stream and wherein the receipt of corresponding frames of the first data stream by the router occurs ahead of the receipt of corresponding frames of the second data stream by the data stream encoder; applying, by the data stream encoder, one or more data modifications based at least on executing the one or more sets of processing instructions in processing the second data stream; generating, by the data stream encoder, an output processed stream including the data modifications. ### Claim 12 (depends on 11) The method of claim 11, wherein the communications latency differences are caused due to different geospatial characteristics associated with the first data stream and the second data stream. ### Claim 13 (depends on 12) The method of claim 12, wherein a duration associated with the communications latency differences is controllable through selection of geospatially diverse cloud computing resources for controlling data stream modifications. ### Claim 14 (depends on 13) The method of claim 13, wherein the duration associated with the communications latency differences is controlled based on a selection of geospatially diverse cloud computing resources for usage as the router. ### Claim 15 (depends on 13) The method of claim 13, wherein the duration associated with the communications latency differences is controlled based on a selection of geospatially diverse cloud computing resources for usage as the stream encoder. ### Claim 16 (depends on 13) The method of claim 13, wherein the duration associated with the communications latency differences is controlled based on a selection of geospatially diverse cloud computing resources for usage as the router and for usage as the data stream encoder. ### Claim 17 (depends on 13) The method of claim 13, wherein the duration associated with the communications latency differences is controlled to satisfy a temporal constraint for executing the one or more sets of processing instructions in processing the second data stream. ### Claim 18 (depends on 17) The method of claim 17, wherein the temporal constraint for executing the one or more sets of processing instructions is scaled based on a processing complexity of the one or more sets of processing instructions. ### Claim 19 (depends on 12) The method of claim 12, wherein the communication latency differences are utilized to scale a processing complexity of the one or more sets of processing instructions generated by the router. ### Claim 20 A non-transitory computer readable medium storing instructions, which when executed by a processor, cause the processor to perform a method for controlling data stream modifications, the method comprising: receiving, by a router, a first data stream; generating, by the router or a processing device coupled to the router, based on the first data stream, one or more sets of processing instructions for modifying a second data stream, the first data stream and the second data stream both derived from a source video data stream; receiving, by a data stream encoder, the second data stream, wherein receipt of the corresponding frames of the first data stream and the corresponding frames of the second data stream is temporally shifted relative to one another at least in part due to one or more communications latency differences between the first data stream and the second data stream and wherein the receipt of corresponding frames of the first data stream by the router occurs ahead of the receipt of corresponding frames of the second data stream by the data stream encoder; applying, by the data stream encoder, one or more data modifications based at least on executing the one or more sets of processing instructions in processing the second data stream; generating, by the data stream encoder, an output processed stream including the data modifications.