# US 2014/0156364 A1 · Scheme for determining the locations and timing of advertisements and other insertions in media > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0156364 A1 - Document ID: US20140156364A1 - Status: Published application - Kind: A1 - Application number: 14/175,996 - Filed: 2014-02-07 - Published: 2014-06-05 - Earliest claimed: 2007-03-22 - Assignees: Sony Computer Entertainment America LLC - Applicants: Sony Computer Entertainment America LLC - Inventors: Gary M. Zalewski - Web page: https://patentgrep.com/patent/US20140156364A1 ## CPC - H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement} - G06Q 30/0241: Advertisements - G06V 40/168: {Feature extraction; Face representation} - H04N 5/147: {Scene change detection} - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/43072: {of multiple content streams on the same device} - H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window} - H04N 21/44012: {involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs} - H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)} - H04N 21/8583: {by creating hot-spots} - H04N 21/8586: {by using a URL (processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A non-transitory computer readable storage medium stores one or more computer programs adapted to cause a processor based system to execute steps that include analyzing an image, identifying one or more faces in the image using a face recognition technique, designating at least one of the identified faces collectively as a first area of interest, and determining whether an insertion area exists in the image where additional content can be inserted without obstructing the first area of interest. Another computer program is adapted to cause a processor based system to execute steps that include determining whether the insertion area can be divided into two or more regions based on color. Methods and processor based apparatuses that perform one or more of these steps are also disclosed. ## Claims ### Claim 1 A non-transitory computer readable storage medium storing one or more computer programs adapted to cause a processor based system to execute steps comprising: analyzing an image; identifying one or more faces in the image using a face recognition technique; designating at least one of the identified faces collectively as a first area of interest; and determining whether an insertion area exists in the image where additional content can be inserted without obstructing the first area of interest. ### Claim 2 (depends on 1) The non-transitory computer readable storage medium of claim 1, wherein the step of designating at least one of the identified faces collectively as a first area of interest comprises: determining a size of each of the identified faces; and selecting the at least one of the identified faces to be included in the first area of interest based on the sizes of the identified faces. ### Claim 3 (depends on 2) The non-transitory computer readable storage medium of claim 2, wherein the selecting the at least one of the identified faces to be included in the first area of interest based on the sizes of the identified faces comprises: identifying a largest one of the identified faces; selecting the largest one of the identified faces and only ones of the identified faces that are not smaller than a certain percentage of the largest one of the identified faces. ### Claim 4 (depends on 3) The non-transitory computer readable storage medium of claim 3, wherein the certain percentage is designated by a user. ### Claim 5 (depends on 1) The non-transitory computer readable storage medium of claim 1, wherein the one or more computer programs are further adapted to cause the processor based system to execute steps comprising: inserting the additional content into the insertion area. ### Claim 6 (depends on 1) The non-transitory computer readable storage medium of claim 1, wherein the one or more computer programs are further adapted to cause the processor based system to execute steps comprising: determining whether the insertion area can be divided into two or more regions based on color. ### Claim 7 (depends on 6) The non-transitory computer readable storage medium of claim 6, wherein the one or more computer programs are further adapted to cause the processor based system to execute steps comprising: dividing the insertion area into two or more regions based on color; and inserting the additional content into one of the two or more regions such that the additional content is completely contained within the one of the two or more regions. ### Claim 8 (depends on 1) The non-transitory computer readable storage medium of claim 1, wherein the image comprises one frame in a sequence of frames of content. ### Claim 9 (depends on 8) The non-transitory computer readable storage medium of claim 8, wherein: the identifying one or more faces in the image comprises identifying one or more faces in a scene depicted by the sequence of frames; and the determining whether an insertion area exists in the image where additional content can be inserted without obstructing the first area of interest comprises determining whether an insertion area exists in the scene where additional content can be inserted without obstructing the first area of interest. ### Claim 10 (depends on 9) The non-transitory computer readable storage medium of claim 9, wherein the one or more computer programs are further adapted to cause the processor based system to execute steps comprising: determining an amount of time that the insertion area is available when the sequence of frames is being played. ### Claim 11 (depends on 9) The non-transitory computer readable storage medium of claim 9, wherein the one or more computer programs are further adapted to cause the processor based system to execute steps comprising: determining a size of the insertion area. ### Claim 12 A non-transitory computer readable storage medium storing one or more computer programs adapted to cause a processor based system to execute steps comprising: analyzing a sequence of frames of content; identifying a first area of interest in a scene depicted by the sequence of frames that should not be obstructed from being viewed; determining whether an insertion area exists in the scene where additional content can be inserted without obstructing the first area of interest; and determining whether the insertion area can be divided into two or more regions based on color. ### Claim 13 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein the one or more computer programs are further adapted to cause the processor based system to execute steps comprising: dividing the insertion area into two or more regions based on color; and inserting the additional content into one of the two or more regions such that the additional content is completely contained within the one of the two or more regions. ### Claim 14 (depends on 13) The non-transitory computer readable storage medium of claim 13, wherein the one or more computer programs are further adapted to cause the processor based system to execute steps comprising: selecting the additional content at least partly based on a color of the one of the two or more regions. ### Claim 15 (depends on 13) The non-transitory computer readable storage medium of claim 13, wherein the inserting the additional content into one of the two or more regions such that the additional content is completely contained within the one of the two or more regions comprises: maintaining a buffer area between the additional content and an edge of the one of the two or more regions. ### Claim 16 (depends on 13) The non-transitory computer readable storage medium of claim 13, wherein the inserting the additional content into one of the two or more regions such that the additional content is completely contained within the one of the two or more regions comprises: placing the additional content so that it is fixed in the one of the two or more regions in the scene and is not allowed to move in the scene. ### Claim 17 (depends on 13) The non-transitory computer readable storage medium of claim 13, wherein the inserting the additional content into one of the two or more regions such that the additional content is completely contained within the one of the two or more regions comprises: placing the additional content so that it is allowed to move within the one of the two or more regions in the scene. ### Claim 18 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein the identifying a first area of interest in a scene depicted by the sequence of frames that should not be obstructed from being viewed comprises: identifying one or more faces in the scene using a face recognition technique. ### Claim 19 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein the identifying a first area of interest in a scene depicted by the sequence of frames that should not be obstructed from being viewed comprises: identifying areas in the scene having activity. ### Claim 20 A method, comprising: analyzing a first sequence of frames of content; identifying one or more faces in a scene depicted by the first sequence of frames using a face recognition technique; designating at least one of the identified faces collectively as a first area of interest; and determining whether an insertion area exists in the scene where additional content can be inserted without obstructing the first area of interest. ### Claim 21 (depends on 20) The method of claim 20, further comprising: inserting the additional content into the insertion area in the scene depicted by the first sequence of frames. ### Claim 22 (depends on 21) The method of claim 21, wherein the content further comprises a second sequence of frames following the first sequence of frames. ### Claim 23 (depends on 22) The method of claim 22, further comprising: playing the content so that the first sequence of frames plays and displays the inserted additional content and then the second sequence of frames plays. ### Claim 24 (depends on 23) The method of claim 23, wherein the playing the content further comprises: providing an option to a user during the first sequence of frames to skip the inserted additional content. ### Claim 25 (depends on 24) The method of claim 24, wherein the option to skip the inserted additional content is provided to the user after an elapsed amount of time. ### Claim 26 (depends on 24) The method of claim 24, wherein the option to skip the inserted additional content is provided to the user in response to the user clicking on something on the user's screen. ### Claim 27 (depends on 20) A processor based apparatus receiving and playing the content processed by the method of claim 20. ### Claim 28 (depends on 27) The processor based apparatus of claim 27, wherein the processor based apparatus comprises any one or more of a set-top box, cable box, entertainment system, game console, computer, and television. ### Claim 29 (depends on 21) A processor based apparatus receiving and playing the content processed by the method of claim 21. ### Claim 30 (depends on 29) The processor based apparatus of claim 29, wherein the processor based apparatus comprises any one or more of a set-top box, cable box, entertainment system, game console, computer, and television. ### Claim 31 A method, comprising: analyzing a sequence of frames of content; identifying a first area of interest in a scene depicted by the sequence of frames that should not be obstructed from being viewed; determining whether an insertion area exists in the scene where additional content can be inserted without obstructing the first area of interest; and determining whether the insertion area can be divided into two or more regions based on color. ### Claim 32 (depends on 31) The method of claim 31, further comprising: dividing the insertion area into two or more regions based on color; and inserting the additional content into one of the two or more regions such that the additional content is completely contained within the one of the two or more regions. ### Claim 33 (depends on 31) A processor based apparatus receiving and playing the content processed by the method of claim 31. ### Claim 34 (depends on 33) The processor based apparatus of claim 33, wherein the processor based apparatus comprises any one or more of a set-top box, cable box, entertainment system, game console, computer, and television. ### Claim 35 (depends on 32) A processor based apparatus receiving and playing the content processed by the method of claim 32. ### Claim 36 (depends on 35) The processor based apparatus of claim 35, wherein the processor based apparatus comprises any one or more of a set-top box, cable box, entertainment system, game console, computer, and television. ### Claim 37 A non-transitory computer readable storage medium storing one or more computer programs adapted to cause a processor based system to execute steps comprising: playing a first sequence of frames, wherein the first sequence of frames includes an advertisement inserted into a scene depicted by at least some of the frames in the first sequence of frames; and playing a second sequence of frames after the first sequence of frames; wherein the advertisement was inserted into the scene in the first sequence of frames by a process that includes, identifying one or more faces in the scene using a face recognition technique; designating at least one of the identified faces collectively as a first area of interest; and inserting the advertisement into an insertion area in the scene that does not obstruct the first area of interest. ### Claim 38 (depends on 37) The non-transitory computer readable storage medium of claim 37, wherein the one or more computer programs are further adapted to cause the processor based system to execute steps comprising: providing an option to a user during the playing of the first sequence of frames to skip the advertisement. ### Claim 39 (depends on 38) The non-transitory computer readable storage medium of claim 38, wherein the option to skip the advertisement is provided to the user after an elapsed amount of time. ### Claim 40 (depends on 38) The non-transitory computer readable storage medium of claim 38, wherein the option to skip the advertisement is provided to the user in response to the user clicking on something on the user's screen.