# US 9,519,644 B2 · Methods and devices for generating media items > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,644 B2 - Document ID: US9519644B2 - Status: Granted - Kind: B2 - Application number: 14/608,108 - Filed: 2015-01-28 - Published: 2015-10-08 - Granted: 2016-12-13 - Earliest claimed: 2014-04-04 - Assignees: Facebook, Inc. - Applicants: Facebook, Inc. - Inventors: Scott Snibbe; Graham McDermott; Justin Ponczec; Spencer Schoeben; Jesse Fulton - Web page: https://patentgrep.com/patent/US9519644B2 ## CPC - G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel - G06F 16/178: Techniques for file synchronisation in file systems - G06F 16/433: {using audio data} - G06F 16/434: {using image data, e.g. images, photos, pictures taken by a user} - G06F 16/4393: {Multimedia presentations, e.g. slide shows, multimedia albums} - G06F 16/48: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/235: Processing of additional data, e.g. scrambling of additional data or processing content descriptors - H04N 21/2393: {involving handling client requests (scheduling or organising the servicing of application requests H04L67/60)} - H04N 21/8547: involving timestamps for synchronizing content - H04N 21/858: Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A server system with one or more processors and memory receives, from a client device, information including one or more criteria for a media item to be generated. In some embodiments, the one or more criteria include one or more audio tracks for the media item to be generated. In some embodiments, the one or more criteria include one or more keywords for the media item to be generated. The server system identifies one or more media files in a database of media files for the media item to be generated based at least in part on the one or more criteria. The server system sends, to the client device, first information identifying the one or more media files. In some embodiments, the server system also sends, to the client device, synchronization information for synchronizing one or more audio tracks with the one or more identified media files. ## Claims ### Claim 1 A method of generating a media item, the method comprising: at a server system with one or more processors and memory: receiving from a client device one or more user visual media files; generating one or more visual media characteristics for the one or more user visual media files; receiving from the client device a request to generate the media item using the one or more user visual media files, the request including one or more criteria for the media item to be generated; and in response to receiving the request, automatically and without user interaction, generating the media item, including: identifying one or more audio tracks based at least in part on the one or more criteria and the one or more visual media characteristics, and identifying one or more sources for the identified one or more audio tracks; and generating synchronization information for synchronizing the one or more audio tracks with the one or more visual media files during playback of the media item; storing the generated media item at a new genesis node in the server system, including storing information identifying: (1) the one or more user visual media items, (2) the identified one or more audio tracks, (3) the identified one or more sources for the identified one or more audio tracks, and (4) the generated synchronization information; and enabling playback of the media item at the client device by sending a link for the stored media item to the client device. ### Claim 2 (depends on 1) The method of claim 1, wherein the one or more criteria include identification information for the one or more audio tracks. ### Claim 3 (depends on 1) The method of claim 1, wherein the one or more user visual media files include at least one video and at least one image. ### Claim 4 (depends on 1) The method of claim 1, further including: analyzing the one or more audio tracks so as to generate one or more characteristic fingerprints for the one or more audio tracks; and wherein identifying the one or more audio tracks includes matching, within a predefined degree of accuracy, at least one of the one or more characteristic fingerprints for the one or more audio tracks to the one or more visual media characteristics for the one or more user visual media files. ### Claim 5 (depends on 1) The method of claim 1, further including: after receiving the request, obtaining a user profile associated with the user of the client device; and wherein identifying the one or more audio tracks includes identifying the one or more audio tracks based at least in part on the one or more criteria and one or more parameters in the user profile. ### Claim 6 (depends on 1) The method of claim 1, wherein the received request further includes one or more environmental parameters associated with the client device or a user of the client device; and wherein identifying the one or more audio tracks includes identifying the one or more audio tracks based at least in part on the one or more criteria and the one or more environmental parameters. ### Claim 7 (depends on 1) The method of claim 1, wherein identifying the one or more audio tracks includes identifying at least a subset of the one or more audio tracks based on metadata annotation of the at least a subset of the one or more user visual media files. ### Claim 8 (depends on 1) The method of claim 1, wherein identifying the one or more audio tracks includes identifying at least a subset of the one or more audio tracks by applying a matching algorithm to match the one or more criteria with the at least a subset of the one or more audio tracks based on keywords. ### Claim 9 (depends on 1) The method of claim 1, wherein identifying the one or more audio tracks includes identifying at least a subset of the one or more audio tracks by applying a matching algorithm to match the one or more criteria with extractable characteristics of the at least a subset of the one or more audio tracks. ### Claim 10 (depends on 1) The method of claim 1, further comprising storing the one or more user visual media files in a database of media files at the server system, wherein a respective user visual media file in the database of media files is indexed according to a plurality of parameters. ### Claim 11 (depends on 10) The method of claim 10, wherein the plurality of parameters includes one or more of the set of: keywords and hashtags corresponding to the respective visual media file; metadata corresponding to the respective visual media file; and one or more extractable characteristics of the respective visual media files. ### Claim 12 (depends on 1) The method of claim 1, wherein the one or more user visual media files include at least one of: one or more visual media files captured by a user of the client device; and one or more visual media files captured by friends of the user of the client device. ### Claim 13 (depends on 1) The method of claim 1, further including sending, to the client device, effect information for combining the one or more user visual media files with one or more transition effects. ### Claim 14 (depends on 13) The method of claim 13, further including, prior to sending the effect information, determining the one or more transition effects based at least in part on measured characteristics of the one or more audio tracks. ### Claim 15 (depends on 1) The method of claim 1, further including: receiving, from the client device, feedback as to the accuracy of the generated media item; and modifying a matching algorithm based on the feedback, the matching algorithm for identifying the one or more audio tracks. ### Claim 16 A server system, comprising: one or more processors; and memory storing one or more programs to be executed by the one or more processors, the one or more programs comprising instructions for: receiving from a client device one or more user visual media files; generating one or more visual media characteristics for the one or more user visual media files; receiving from the client device a request to generate the media item using the one or more user visual media files, the request including one or more criteria for the media item to be generated; and in response to receiving the request, automatically and without user interaction, generating the media item, including: identifying one or more audio tracks based at least in part on the one or more criteria and the one or more visual media characteristics, and identifying one or more sources for the identified one or more audio tracks; and generating synchronization information for synchronizing the one or more audio tracks with the one or more visual media files during playback of the media item; storing the generated media item at a new genesis node in the server system, including storing information identifying: (1) the one or more user visual media items, (2) the identified one or more audio tracks, (3) the identified one or more sources for the identified one or more audio tracks, and (4) the generated synchronization information; and enabling playback of the media item at the client device by sending a link for the stored media item to the client device. ### Claim 17 A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by a server system with one or more processors, cause the server system to: receive from a client device one or more user visual media files; generate one or more visual media characteristics for the one or more user visual media files; receive from the client device a request to generate the media item using the one or more user visual media files, the request including one or more criteria for the media item to be generated; and in response to receiving the request, automatically and without user interaction, generate the media item, including: identifying one or more audio tracks based at least in part on the one or more criteria and the one or more visual media characteristics, and identifying one or more sources for the identified one or more audio tracks; and generating synchronization information for synchronizing the one or more audio tracks with the one or more visual media files during playback of the media item; store the generated media item at a new genesis node in the server system, including storing information identifying: (1) the one or more user visual media items, (2) the identified one or more audio tracks, (3) the identified one or more sources for the identified one or more audio tracks, and (4) the generated synchronization information; and enable playback of the media item at the client device by sending a link for the stored media item to the client device. ### Claim 18 (depends on 1) The method of claim 1, further including storing the one or more user visual media files at a first location in a database of media files at the server system; and wherein storing the generated media item at the new genesis node in the server system includes storing identification of the first location in the database of media files. ### Claim 19 (depends on 1) The method of claim 1, further including: receiving a playback request from a second client device to playback the media item, wherein the second client device is distinct from the client device; and in response to receiving the playback request, enabling playback of the media item at the second client device by sending the link to the stored media item to the second client device. ### Claim 20 (depends on 1) The method of claim 1, further including: receiving a modification request from a second client device to store a modified version of the media item; in response to receiving the modification request, storing the modified version of the media item at a new leaf node associated with the new genesis node in the server system, wherein storing the modification version of the media item includes storing an association between the modified version of the media item and media item; and enabling playback of the modified version of the media item by sending a link for the modified version of the media item to the second client device.