# US 10,455,268 B2 · Method and system for targeted commerce in network broadcasting > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,455,268 B2 - Document ID: US10455268B2 - Status: Granted - Kind: B2 - Application number: 15/590,744 - Filed: 2017-05-09 - Published: 2017-10-26 - Granted: 2019-10-22 - Earliest claimed: 2012-12-19 - Assignees: Paypal, Inc. - Applicants: PayPal, Inc. - Inventors: Heather Anne Waibel; Nicholas Albert Rudock - Web page: https://patentgrep.com/patent/US10455268B2 ## CPC - H04N 21/2668: Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles {(information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063)} - H04L 65/611: for multicast or broadcast (systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06) - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream - H04N 21/4668: {for recommending content, e.g. movies} - H04N 21/47815: {Electronic shopping (payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12)} - H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)} - H04N 21/8545: for generating interactive applications - H04N 21/858: Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A system and a method for using the system for targeted commerce in network broadcasting are provided. The system includes an interface device configured to receive a multimedia stream from a network, wherein the multimedia stream includes a close captioning string and wherein the interface device is further configured to process the multimedia stream by providing advertisements in the multimedia stream according to a correlation between the close captioning string and a plurality of vendor keywords; and a viewing device configured to receive the processed multimedia stream and display to a viewer. ## Claims ### Claim 1 A system for targeted commerce in network broadcasting comprising: a non-transitory memory; and one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising: receiving, from a network, a multimedia stream; determining that a closed captioning string of the received multimedia stream includes an identity of a product corresponding to one or more vendor keywords; in response to determining that the closed captioning string includes the identity of the product corresponding to the one or more vendor keywords, identifying and analyzing a plurality of video frames of the received multimedia stream to detect a visual depiction of the product corresponding to the one or more vendor keywords; and in response to both determining that the closed captioning string includes the identity of the product corresponding to the one or more vendor keywords and detecting the visual depiction of the product corresponding to the one or more vendor keywords in the plurality of video frames, modifying the received multimedia stream to include an advertisement corresponding to the product. ### Claim 2 (depends on 1) The system of claim 1, wherein the advertisement included in the multimedia stream is caused to be displayed when the identity of the product appears in the closed captioning string. ### Claim 3 (depends on 1) The system of claim 1, wherein the advertisement included in the multimedia stream is caused to be displayed when the visual depiction of the product appears in the plurality of video frames. ### Claim 4 (depends on 1) The system of claim 1, wherein the advertisement included in the multimedia stream is caused to be displayed when the identity of the product appears in the closed captioning string and the visual depiction of the product appears in the plurality of video frames. ### Claim 5 (depends on 1) The system of claim 1, wherein determining that a closed captioning string of the multimedia stream includes an identity of the product for the detected visual depiction comprises determining a correlation between the closed captioning string and the visual depiction. ### Claim 6 (depends on 5) The system of claim 5, wherein modifying the multimedia stream to include the advertisement comprises: determining one or more advertisements based on the correlation; increasing a probability that one of the one or more advertisements is to be included in the modified multimedia stream based on the correlation; and including one of the one or more advertisements with a highest probability. ### Claim 7 (depends on 1) The system of claim 1, wherein modifying the multimedia stream further comprises modifying the multimedia stream to include a payment tab. ### Claim 8 A non-transitory machine-readable medium comprising a plurality of machine-readable instructions which when executed by one or more processors of an interface device are adapted to cause the interface device to perform a method comprising: receiving, from a network, a multimedia stream; detecting, from closed captioning strings of the multimedia stream, a generic description of a product; determining that the generic description of the product corresponds to one of a plurality of vendor keywords; in response to both detecting that the closed captioning strings include the generic description of the product and that the generic description of the product corresponds to one of the plurality of vendor keywords, identifying and analyzing a plurality of video frames of the received multimedia stream to detect a visual depiction of the product corresponding to one of the plurality of vendor keywords; and in response to each of detecting that the closed captioning strings include the generic description of the product, determining that the generic description of the product corresponds to one of the plurality of vendor keywords, and detecting the visual depiction of the product corresponding to the one of the plurality of vendor keywords in the received multimedia stream, modifying the multimedia stream to include an advertisement. ### Claim 9 (depends on 8) The non-transitory machine-readable medium of claim 8, wherein the advertisement associated with the one of the plurality of vendor keywords is caused to be displayed when the product is detected as being shown in the multimedia stream. ### Claim 10 (depends on 8) The non-transitory machine-readable medium of claim 8, wherein the plurality of vendor keywords is received from one or more vendors. ### Claim 11 (depends on 10) The non-transitory machine-readable medium of claim 10, wherein when the one of the plurality of vendor keywords is associated with two or more vendors, the method further comprises: requesting, from each of the two or more vendors, a bid for the advertisement; and receiving, from each of the two or more vendors, the bid for the advertisement, wherein the advertisement included in the modified multimedia stream corresponds to an advertisement designated by a vendor of the two or more vendors submitting a highest bid. ### Claim 12 (depends on 8) The non-transitory machine-readable medium of claim 8, wherein the method further comprises determining a correlation between the generic description of the product and a purchaser profile, and wherein modifying the multimedia stream to include the advertisement is further based on the determining the correlation between the generic description of the product and the purchaser profile. ### Claim 13 (depends on 12) The non-transitory machine-readable medium of claim 12, wherein a placement of the advertisement included in the multimedia stream is based on the determined correlation between the generic description of the product and a purchaser profile. ### Claim 14 A method for targeted commerce in network broadcasting, the method comprising: receiving, from a network, a multimedia stream; determining that a closed captioning string of the received multimedia stream includes an identity of a product corresponding to one or more vendor keywords; in response to determining that the closed captioning string includes the identity of the product corresponding to the one or more vendor keywords, identifying and analyzing a plurality of video frames of the received multimedia stream to detect a visual depiction of the product corresponding to the one or more vendor keywords; and in response to both determining that the closed captioning string includes the identity of the product corresponding to the one or more vendor keywords and detecting the visual depiction of the product corresponding to the one or more vendor keywords in the plurality of video frames, modifying the received multimedia stream to include an advertisement corresponding to the visual depiction of the product. ### Claim 15 (depends on 14) The method of claim 14, further comprising: determining a first correlation between a purchase profile and the closed captioning string; determining a second correlation between the purchase profile and metadata describing contents of the plurality of video frames; and determining a third correlation between the purchase profile and the one or more vendor keywords. ### Claim 16 (depends on 15) The method of claim 15, wherein the advertisement is identified for display based on the first, second and third correlations. ### Claim 17 (depends on 16) The method of claim 16, wherein the advertisement is identified from a plurality of advertisements by determining a probability that one of the plurality of advertisements has a highest probability for inclusion in the multimedia stream based on the first, second and third correlations. ### Claim 18 (depends on 15) The method of claim 15, wherein the purchase profile is received when a user logs in and discloses an identity of the user. ### Claim 19 (depends on 15) The method of claim 15, wherein the purchase profile is created based on a history of purchases by a user. ### Claim 20 (depends on 15) The method of claim 15, wherein the purchase profile includes one of a list of items that a user is interested in purchasing or a generic list of items.