# US 9,438,936 B1 · Producing video data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,438,936 B1 - Document ID: US9438936B1 - Status: Granted - Kind: B1 - Application number: 14/678,780 - Filed: 2015-04-03 - Granted: 2016-09-06 - Earliest claimed: 2015-04-03 - Assignees: MirriAd Limited - Applicants: MirriAd Advertising Limited - Inventors: Ram Srinivasan; Philip McLauchlan - Web page: https://patentgrep.com/patent/US9438936B1 ## CPC - H04N 21/8583: {by creating hot-spots} - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement} - H04N 21/234345: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment} - H04N 21/23439: {for generating different versions} - H04N 21/2353: {specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata} - H04N 21/2393: {involving handling client requests (scheduling or organising the servicing of application requests H04L67/60)} - H04N 21/26258: {for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40)} - H04N 21/2668: Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles {(information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063)} - H04N 21/6379: directed to encoder {, e.g. for requesting a lower encoding rate} - H04N 21/64769: {for rate control (flow control in packet networks H04L47/10)} - H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)} - H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)} - H04N 21/8358: involving watermark {(protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467)} - H04N 21/845: Structuring of content, e.g. decomposing content into time segments - H04N 21/8455: {involving pointers to the content, e.g. pointers to the I-frames of the video stream} - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04N 21/854: Content authoring - H04N 21/858: Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method of incorporating additional video objects into source video data to produce output video data. The method includes retrieving source video data and data defining a segment size used by a distributor, into which video data is divided when transmitted. The method includes analyzing the source video data to identify selected frames of video material which include insertion zones which correspond to regions which are suitable for receiving an additional video object. The method includes identifying a boundary point of the source video data. The method includes embedding additional video objects into the selected frames, creating output video data which has a boundary which corresponds with the identified boundary point. The method includes generating metadata including information on said boundary point of the source video data to be replaced by the created output video data; and transmitting the output video data and the metadata to the distributor. ## Claims ### Claim 1 A method of incorporating one or more additional video objects into source video data to produce output video data, the method comprising: retrieving source video data, the source video data comprising: frames of video material; and data defining a segment size used by a distributor, into which video data is divided when transmitted by the distributor; analyzing the source video data to identify selected frames of video material which include one or more insertion zones, wherein the insertion zones correspond to one or more regions within the selected frames of video material which are suitable for receiving an additional video object; identifying at least one boundary point of the source video data, based on the data defining the segment size into which video data is divided prior to distribution, and based on the selected frames; embedding the one or more additional video objects into the selected frames; creating output video data which includes the one or more additional video objects and which has a boundary which corresponds with the identified at least one boundary point; generating metadata including information on said at least one boundary point of the source video data to be replaced by the created output video data; transmitting the output video data and the metadata to the distributor; encoding a first version of the output video data into a first bitrate; and encoding more than one additional versions of the output video data into more than one bitrate, different to the first bitrate; wherein the at least one boundary point of the source video data comprises an Instantaneous Decoder Refresh (IDR) frame; wherein the retrieved source video data is divided into segments based on the retrieved data regarding segment size provided by the distributor, the method comprising identifying the segments in which the identified selected frames correspond; and wherein the metadata includes a tracking pixel, which can be configured to transmit a signal in response to an embed segment being played. ### Claim 2 (depends on 1) The method of claim 1, wherein the segment size into which video data is divided when transmitted by the distributor is uniform. ### Claim 3 (depends on 1) The method of claim 1, wherein the segment size into which video data is divided when transmitted by the distributor is defined by a time period. ### Claim 4 (depends on 1) The method of claim 1, wherein the segment size into which video data is divided when transmitted by the distributor is defined by a number of frames of video material. ### Claim 5 (depends on 1) The method of claim 1, wherein the output video data is divided into segments for use in adaptive bitrate streaming. ### Claim 6 (depends on 1) The method of claim 1, wherein the metadata includes information on the length of time of the output video data. ### Claim 7 (depends on 1) The method of claim 1, wherein the metadata is included in a Video Multiple Ad Playlist (VMAP) file. ### Claim 8 (depends on 1) The method of claim 1, comprising creating one or more additional versions of the output video data, the one or more additional versions of the output video data comprising one or more additional video objects which are different to the one or more additional objects which are inserted into other versions of the output video data. ### Claim 9 (depends on 8) The method of claim 8, wherein a version of the output video data is selected for including into the source video data based on one or more characteristics associated with the user. ### Claim 10 A method of incorporating one or more additional video objects into source video data to produce output video data, the method comprising: retrieving source video data, the source video data comprising: frames of video material; and data defining a segment size used by a distributor, into which video data is divided when transmitted by the distributor; analyzing the source video data to identify selected frames of video material which include one or more insertion zones, wherein the insertion zones correspond to one or more regions within the selected frames of video material which are suitable for receiving an additional video object; identifying at least one boundary point of the source video data, based on the data defining the segment size into which video data is divided prior to distribution, and based on the selected frames; embedding the one or more additional video objects into the selected frames; creating output video data comprising: the selected frames into which the one or more additional video objects are embedded; and one or more additional frames of video material, which are located between the selected frames and the at least one boundary point, and wherein the selected frames have a predetermined proximity relationship with the at least one boundary point; generating metadata including information on the source video data to be replaced by the created output video data; transmitting the output video data and the metadata to the distributor encoding a first version of the output video data into a first bitrate; and encoding more than one additional versions of the output video data into more than one bitrate, different to the first bitrate; wherein the at least one boundary point of the source video data comprises an Instantaneous Decoder Refresh (IDR) frame; wherein the retrieved source video data is divided into segments based on the retrieved data regarding segment size provided by the distributor, the method comprising identifying the segments in which the identified selected frames correspond; and wherein the metadata includes a tracking pixel, which can be configured to transmit a signal in response to an embed segment being played. ### Claim 11 A system for incorporating one or more additional video objects into source video data to produce output video data, comprising: a memory configured to store retrieved source video data, the source video data comprising: frames of video material; and data defining a segment size used by a distributor, into which video data is divided when transmitted by the distributor; a processor configured to: analyze the source video data to identify selected frames of video material which include one or more insertion zones, wherein the insertion zones correspond to one or more regions within the selected frames of video material which are suitable for receiving an additional video object; identify at least one boundary point of the source video data, based on the data defining the segment size into which video data is divided prior to distribution, and based on the selected frames; embed the one or more additional video objects into the selected frames; create output video data which includes the one or more additional video objects and which has a boundary which corresponds with the identified at least one boundary point; generate metadata including information on said at least one boundary point of the source video data to be replaced by the created output video data; transmit the output video data and the metadata to the distributor encode a first version of the output video data into a first bitrate; and encode more than one additional versions of the output video data into more than one bitrate, different to the first bitrate; wherein the at least one boundary point of the source video data comprises an Instantaneous Decoder Refresh (IDR) frame; wherein the retrieved source video data is divided into segments based on the retrieved data regarding segment size provided by the distributor, the method comprising identifying the segments in which the identified selected frames correspond; and wherein the metadata includes a tracking pixel, which can be configured to transmit a signal in response to an embed segment being played. ### Claim 12 A non-transitory computer-readable medium having computer executable instructions stored thereon, which when executed by a computing device cause the computing device to perform a method of incorporating one or more one or more additional video objects into source video data to produce output video data, comprising: retrieving source video data, the source video data comprising: frames of video material; and data defining a segment size used by a distributor, into which video data is divided when transmitted by the distributor; analyzing the source video data to identify selected frames of video material which include one or more insertion zones, wherein the insertion zones correspond to one or more regions within the selected frames of video material which are suitable for receiving an additional video object; identifying at least one boundary point of the source video data, based on the data defining the segment size into which video data is divided prior to distribution, and based on the selected frames; embedding the one or more additional video objects into the selected frames; creating output video data which includes the one or more additional video objects and which has a boundary which corresponds with the identified at least one boundary point; generating metadata including information on said at least one boundary point of the source video data to be replaced by the created output video data; transmitting the output video data and the metadata to the distributor; encoding a first version of the output video data into a first bitrate; and encoding more than one additional versions of the output video data into more than one bitrate, different to the first bitrate; wherein the at least one boundary point of the source video data comprises an Instantaneous Decoder Refresh (IDR) frame; wherein the retrieved source video data is divided into segments based on the retrieved data regarding segment size provided by the distributor, the method comprising identifying the segments in which the identified selected frames correspond; and wherein the metadata includes a tracking pixel, which can be configured to transmit a signal in response to an embed segment being played.