# US 2020/0288209 A1 · Technologies for analyzing and searching for features in image data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0288209 A1 - Document ID: US20200288209A1 - Status: Published application - Kind: A1 - Application number: 16/297,124 - Filed: 2019-03-08 - Published: 2020-09-10 - Earliest claimed: 2019-03-08 - Assignees: FCB Worldwide, Inc. - Applicants: FCB Worldwide, Inc. - Inventors: Michael Fassnacht; Mark Jungwirth; Kerry Hill; Fernando Espejel - Web page: https://patentgrep.com/patent/US20200288209A1 ## CPC - H04N 21/2743: Video hosting of uploaded data from client - H04N 21/4828: {for searching programme descriptors (retrieval of video data G06F16/739)} - G06F 16/24522: {Translation of natural language queries to structured queries} - G06F 16/24573: {using data annotations, e.g. user-defined metadata} - G06F 16/7837: {using objects detected or recognised in the video content} - G06N 20/00: Machine learning - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/234309: {by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo (conversion of standards in analog television systems H04N7/01)} - H04N 21/234336: {by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text} - H04N 21/235: Processing of additional data, e.g. scrambling of additional data or processing content descriptors - H04N 21/26603: {for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method according to one embodiment includes automatically transcoding a video file to generate a plurality of transcoded files, wherein the plurality of transcoded files includes a compressed video file, selecting at least one computer vision system to annotate the compressed video file for searching, transmitting the compressed video file to each computer vision system of the selected at least one computer vision system, and storing a plurality of annotations generated by the selected at least one computer system to a central database in response to transmittal of the compressed video file, wherein each annotation of the plurality of annotations identifies at least one visual characteristic of the compressed video file. ## Claims ### Claim 1 A method, comprising: automatically transcoding a video file to generate a plurality of transcoded files, wherein the plurality of transcoded files includes a compressed video file, and wherein each transcoded file of the plurality of transcoded is of a different file type; selecting at least one computer vision system to annotate the compressed video file for searching; transmitting the compressed video file to each computer vision system of the selected at least one computer vision system; and storing a plurality of annotations generated by the selected at least one computer vision system to a central database in response to transmittal of the compressed video file, wherein each annotation of the plurality of annotations identifies at least one visual characteristic of the compressed video file. ### Claim 2 (canceled) ### Claim 3 (depends on 2) The method of claim 2, wherein the plurality of transcoded files further comprises a plurality of thumbnail images; and further comprising: receiving a user's search terms, via a graphical user interface, for at least one video frame having a desired visual characteristic; and displaying a set of thumbnail images of the plurality of thumbnail images corresponding with the desired visual characteristic. ### Claim 4 (depends on 3) The method of claim 3, further comprising: performing a search for at least one video frame having a desired visual characteristic based on the user's search terms; and identifying a time frame within the video file at which the at least one video frame can be found. ### Claim 5 (canceled) ### Claim 6 (depends on 4) The method of claim 4, wherein the user's search terms comprise natural language search terms; and wherein performing the search comprises applying a machine learning algorithm and translating the user's natural language search terms into structured search terms. ### Claim 7 (canceled) ### Claim 8 (depends on 3) The method of claim 3, further comprising retrieving the video file in response to a user's selection, via the graphical user interface, of a thumbnail image of the set of displayed thumbnail images. ### Claim 9 (depends on 1) The method of claim 1, further comprising displaying, via a graphical user interface, a plurality of computer vision systems for a user to select from to annotate the compressed video file for searching; and wherein selecting the at least one computer vision system to annotate the compressed video file for searching comprises receiving the user's selection, via the graphical user interface, of at least one computer vision system of the plurality of computer vision systems displayed to the user to annotate the compressed video file for searching. ### Claim 10 (depends on 1) The method of claim 1, further comprising generating, by each computer vision system of the at least one computer vision system, a set of annotations based on the compressed video; and wherein the plurality of annotations comprises each set of annotations generated based on the compressed video. ### Claim 11 (depends on 1) The method of claim 1, wherein the video file is of a first file type and the compressed video file is of a second file type different from the first file type. ### Claim 12 A system, comprising: a central database; a shared data storage; at least one processor; and at least one memory comprising a plurality of instructions stored thereon that, in response to execution by the at least one processor, causes the system to: automatically transcode a video file to generate a plurality of transcoded files, wherein the plurality of transcoded files includes a compressed video file, wherein each transcoded file of the plurality of transcoded is of a different file type; select at least one computer vision system to annotate the compressed video file for searching; transmit the compressed video file to each computer vision system of the selected at least one computer vision system; and store a plurality of annotations generated by the selected at least one computer system to the central database in response to transmittal of the compressed video file, wherein each annotation of the plurality of annotations identifies at least one visual characteristic of the compressed video file. ### Claim 13 (depends on 12) The system of claim 12, wherein the plurality of transcoded files further comprises a plurality of thumbnail images; and wherein the plurality of instructions further causes the system to: receive a user's search terms, via a graphical user interface, for at least one video frame having a desired visual characteristic; display a set of thumbnail images of the plurality of thumbnail images corresponding with the desired visual characteristic; and retrieve the video file in response to a user's selection, via the graphical user interface, of a thumbnail image of the set of displayed thumbnail images. ### Claim 14 (depends on 13) The system of claim 13, wherein the plurality of instructions further causes the system to: perform a search for at least one video frame having a desired visual characteristic based on the user's search terms; and identify a time frame within the video file at which the at least one video frame can be found; wherein the user's search terms comprise natural language search terms; and wherein to perform the search comprises to translate the user's natural language search terms into structured search terms. ### Claim 15 (canceled) ### Claim 16 (canceled) ### Claim 17 (depends on 12) The system of claim 12, wherein to select the at least one computer vision system to annotate the compressed video file for searching comprises to receive a user's selection, via a graphical user interface, of at least one computer vision system to annotate the compressed video file for searching, wherein the at least one computer vision system is selected by the user from a plurality of computer vision systems displayed to the user via the graphical user interface. ### Claim 18 (canceled) ### Claim 19 (depends on 12) The system of claim 12, wherein the video file is of a first file type and the compressed video file is of a second file type different from the first file type. ### Claim 20 A system, comprising: a central database; a shared data storage; a transcode system that automatically transcodes a video file to generate a plurality of transcoded files including a compressed video file, wherein each transcoded file of the plurality of transcoded is of a different file type; a computing device that (i) provides a first graphical user interface through which a user selects at least one computer vision system to annotate the compressed video file for searching and (ii) transmits the compressed video file to each computer vision system of the selected at least one computer vision system; a computer vision system that (i) receives the compressed video file, (ii) generates a plurality of annotations based on the compressed video file, wherein each annotation of the plurality of annotations identifies at least one visual characteristic of the compressed file, and (iii) store the plurality of annotations to the central database; and a client device that (i) provides a second graphical user interface through which a user enters search terms for at least one video frame having a desired visual characteristic and (ii) displays, via the second graphical user interface, a set of thumbnail images that correspond with the desired visual characteristic. ### Claim 21 (depends on 11) The method of claim 11, wherein the video file includes raw grayscale images and the compressed video file includes color-added images. ### Claim 22 (depends on 1) The method of claim 1, wherein the plurality of transcoded files further comprises a plurality of thumbnail images corresponding with particular frames within the video file, and further comprising: receiving a user's search terms, via a graphical user interface, for at least one video frame having a desired visual characteristic; performing a search for at least one video frame having the desired visual characteristic based on the user's search terms; displaying a set of thumbnail images of the plurality of thumbnail images; and displaying a level of confidence for each thumbnail image of the set of the thumbnail images that the thumbnail image corresponds with the desired visual characteristic. ### Claim 23 (depends on 1) The method of claim 1, further comprising storing a raw grayscale version of the video file to a shared data storage prior to transmitting the compressed video file to each computer vision system; and editing the raw grayscale version of the video file prior to receiving the plurality of annotations generated by the selected at least one computer vision system. ### Claim 24 (depends on 1) The method of claim 1, wherein selecting the at least one computer vision system comprises selecting a first computer vision system and a second computer vision system to annotate the compressed video file for searching; wherein storing the plurality of annotations generated by the selected at least one computer vision system comprises storing a first plurality of annotations generated by the first computer vision system and a second plurality of annotations generated by the second computer vision system. ### Claim 25 (depends on 1) The method of claim 1, further comprising organizing the video file, the plurality of transcoded files, and the plurality of annotations for download into a video editing application. ### Claim 26 (depends on 1) The method of claim 1, wherein the plurality of transcoded files includes a first transcoded file having an mp4 file type and a second transcoded file having an mp3 file type.