# US 10,122,983 B1 · Creating a video for an audio file > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,122,983 B1 - Document ID: US10122983B1 - Status: Granted - Kind: B1 - Application number: 13/786,395 - Filed: 2013-03-05 - Granted: 2018-11-06 - Earliest claimed: 2013-03-05 - Assignees: Google Inc. - Applicants: Google Inc. - Inventors: David Ross; Hrishikesh Aradhye; Douglas Eck; Christopher Tim Althoff - Web page: https://patentgrep.com/patent/US10122983B1 ## CPC - H04N 9/802: involving processing of the sound signal (H04N9/806, H04N9/835 take precedence) - G06F 16/538: Presentation of query results - G06F 16/60: of audio data - G06F 16/63: Querying - G06F 16/68: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually - G06F 16/683: using metadata automatically derived from the content - G06F 16/686: {using information manually generated, e.g. tags, keywords, comments, title or artist information, time, location or usage information, user ratings} - G11B 27/00: Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel - G11B 27/022: Electronic editing of analogue information signals, e.g. audio or video signals - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel - H04N 5/765: Interface circuits between an apparatus for recording and another apparatus - H04N 9/8205: {involving the multiplexing of an additional signal and the colour video signal} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) ## Abstract Systems and methods described herein relate to automation of video creation for an associated audio file or musical composition. In particular, a video can be generated for the audio file that includes images and videos that are compelling and contextually relevant to, and technically compatible with, the audio file. ## Claims ### Claim 1 A system comprising: a memory; and a processor, coupled to the memory, the processor to: determine metadata associated with an audio track; identify entity data that is associated with the metadata of the audio track, wherein entity data comprises a plurality of categories related to the audio track; determine a ranking for the categories of the entity data, wherein the ranking is indicative of a relevance of a particular category to the audio track and is determined irrespective of a comparison of words recorded in the audio track and the particular category; perform a media query to identify visual media related to the audio track based on the ranked categories; and generate a video for the audio track by selecting one or more of the visual media to include in the video. ### Claim 2 (depends on 1) The system of claim 1, wherein the visual media comprises at least one of an image or a video. ### Claim 3 (depends on 1) The system of claim 1, wherein the metadata comprises at least one of a title of the audio track or a name of an artist associated with the audio track. ### Claim 4 (depends on 1) The system of claim 1, wherein the categories of the entity data comprise at least one of an article about an artist; a title and release date of an album that includes the audio track; a genre for the audio track, the album, or the artist; an award associated with the audio track, the album, or the artist; biographical information associated with the artist; inspiration for the audio track, the album, or the artist; an influence associated with the audio track, the album, or the artist; an instrument associated with the artist; information associated with a related artist; lyrics associated with the audio track; or data associated with other media in which the audio track is presented. ### Claim 5 (depends on 1) The system of claim 1, wherein the processor is further to filter the visual media, wherein the video is generated using the filtered visual media. ### Claim 6 (depends on 5) The system of claim 5, wherein the processor is further to filter the visual media based on at least one of a quality of the visual media, a disambiguation associated with the visual media with respect to the media query, or near-duplicate content the visual media. ### Claim 7 (depends on 1) The system of claim 1, wherein the processor is further to filter the visual media based on inferences associated with a machine-learning technique trained based upon human generated music videos. ### Claim 8 (depends on 1) The system of claim 1, wherein to generate the video for the audio track, the processor to utilize a semantic structure in connection with at least one of selection or ordering of the one or more visual media in to generate the video, wherein the semantic structure describes at least one of a chorus, a repeating portion, or a solo performance associated with the audio track. ### Claim 9 (depends on 1) The system of claim 1, the processor further to determine a descriptive caption to include with a portion of the video. ### Claim 10 A method comprising: determining metadata associated with an audio track; identifying entity data that is associated with the metadata of the audio track, wherein entity data comprises a plurality of categories related to the audio track; determining, by a processor, a ranking for the categories of the entity data, wherein the ranking is indicative of a relevance of a particular category to the audio track and is determined irrespective of a comparison of words recorded in the audio track and the particular category; performing a media query to identify visual media related to the audio track based on the ranked categories; and generating a video for the audio track by selecting one or more of the visual media to include in the video. ### Claim 11 (depends on 10) The method of claim 10, wherein the visual media comprises at least one of an image or video. ### Claim 12 (depends on 10) The method of claim 10, further comprising: filtering the visual media, wherein the video is generated using the filtered visual media. ### Claim 13 (depends on 12) The method of claim 12, wherein filtering the visual media further comprises filtering the visual media based on at least one of a quality of the visual media, a disambiguation associated with the visual media, or near-duplicate content of the visual media. ### Claim 14 (depends on 12) The method of claim 12, wherein filtering the visual media comprises filtering the visual media utilizing a machine-learning technique trained based upon human generated music videos. ### Claim 15 (depends on 10) The method of claim 10, further comprising determining a caption to include with a portion of the video. ### Claim 16 A non-transitory computer-readable storage medium comprising instructions that, responsive to execution by a processor, cause the processor to perform operations comprising: determining metadata associated with an audio track; identifying entity data that is associated with the metadata of the audio track, wherein entity data comprises a plurality of categories related to the audio track; determining, by the processor, a ranking for the categories of the entity data, wherein the ranking is indicative of a relevance of a particular category to the audio track and is determined irrespective of a comparison of words recorded in the audio track and the particular category; performing a media query to identify visual media related to the audio track based on the ranked categories; and generating a video for the audio track by selecting one or more of the visual media to include in the video. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the visual media comprises at least one of an image or video. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium of claim 16, the operations further comprising determining a caption to include with a portion of the video.