# US 10,484,728 B2 · Information processing apparatus, method, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,484,728 B2 - Document ID: US10484728B2 - Status: Granted - Kind: B2 - Application number: 15/029,600 - Filed: 2014-10-10 - Published: 2016-09-15 - Granted: 2019-11-19 - Earliest claimed: 2013-10-21 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Akira Inoue; Shusuke Takahashi; Masayuki Nishiguchi - Web page: https://patentgrep.com/patent/US10484728B2 ## CPC - H04N 21/242: Synchronisation processes, e.g. processing of PCR [Programme Clock References] {(arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18)} - G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel - H04N 5/04: Synchronising (for television systems using pulse code modulation H04N7/56) - H04N 5/60: for the sound signals - H04N 21/23602: {Multiplexing isochronously with the video sync, e.g. according to bit-parallel or bit-serial interface formats, as SDI} - H04N 21/4122: {additional display device, e.g. video projector (digital output for controlling a plurality of local displays G06F3/1423)} - H04N 21/4342: {Demultiplexing isochronously with video sync, e.g. according to bit-parallel or bit-serial interface formats, as SDI} - H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/8547: involving timestamps for synchronizing content - G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) ## Abstract An information processing apparatus for reproducing second content in synchronization with reproduction of first content by a second information processing apparatus different from the information processing apparatus, the first content comprising audio content. The information processing apparatus comprising circuitry configured to: extract a first feature from the audio content; obtain a second feature of the audio content, the second feature being together with the second content; compare the first feature with the second feature; and generate, based on results of the comparing, synchronization information used for reproducing the second content in synchronization with the first content. ## Claims ### Claim 1 An information receiving apparatus for reproducing a second media content in synchronization with reproduction of a first media content by a second information receiving apparatus different from the information receiving apparatus, the first media content comprising audio content, the information receiving apparatus comprising: circuitry configured to: extract an audio-derived feature from an audio signal generated via reproduction of the audio content by the second information receiving apparatus; obtain, from an information providing apparatus different from the second information receiving apparatus, the second media content together with a synchronization feature, the synchronization feature being extracted from the audio content of the first media content prior to being transmitted by the information providing apparatus; compare the audio-derived feature with the synchronization feature; generate, based on results of the comparing, synchronization information; and reproduce, using the generated synchronization information and the information receiving apparatus, the second media content in synchronization with reproduction of the first media content by the second information receiving apparatus. ### Claim 2 (depends on 1) The information receiving apparatus of claim 1, wherein the second media content and the synchronization feature are transmitted to the information processing apparatus at a time determined based on a difference between an arrival time of the first media content and an arrival time of the second media content and the synchronization feature. ### Claim 3 (depends on 1) The information receiving apparatus of claim 1, wherein the circuitry is configured to: request transmission of the second media content and the synchronization feature; and receive the second media content and the synchronization feature which are transmitted in response to the request. ### Claim 4 (depends on 3) The information receiving apparatus of claim 3, wherein the circuitry is configured to: request transmission of the first media content; and receive the first media content which is transmitted in response to the request for transmission of the first media content. ### Claim 5 (depends on 1) The information receiving apparatus of claim 1, wherein the circuitry is configured to: extract a first plurality of features from the audio signal, the first plurality of features including the audio-derived feature; obtain a second plurality of features together with the second media content, the second plurality of features including the synchronization feature; compare the first plurality of features with the second plurality of features; and generate, based on results of comparing the first plurality of features with the second plurality of features, the synchronization information used for reproducing the second media content in synchronization with the first media content. ### Claim 6 (depends on 1) The information receiving apparatus of claim 1, wherein the circuitry is configured to control reproduction of the second media content. ### Claim 7 (depends on 6) The information receiving apparatus of claim 6, wherein the circuitry is configured to correct a reproduction position of the second media content based on the synchronization information. ### Claim 8 (depends on 1) The information receiving apparatus of claim 1, wherein the circuitry is configured to: acquire presentation time information associated with the first media content; and obtain a range of time based on results of comparing the presentation time information associated with the first media content and presentation time information associated with the second media content, wherein the circuitry is configured to generate the synchronization information at least in part by comparing the synchronization feature with one or more features extracted from one or more portions of the first media content appearing in the first media content in the range of time. ### Claim 9 (depends on 1) The information receiving apparatus of claim 1, wherein the circuitry is configured to compare the audio-derived feature and the synchronization feature after performing frame rate conversion on at least one of the audio-derived feature and the synchronization feature such that a frame rate of the audio-derived feature coincides with a frame rate of the synchronization feature. ### Claim 10 (depends on 1) The information receiving apparatus of claim 1, wherein the second media content includes the synchronization feature. ### Claim 11 An information processing method for reproducing by an information receiving apparatus a second media content in synchronization with reproduction of a first media content, the first media content comprising audio content, the information processing method comprising: extracting an audio-derived feature from an audio signal generated via reproduction of the audio content by a second information receiving apparatus; obtaining, from an information providing apparatus different from the second information receiving apparatus, the second media content together with a synchronization feature, the synchronization feature being extracted from the audio content of the first media content prior to being transmitted by the information providing apparatus; comparing the audio-derived feature with the synchronization feature; generating, based on results of the comparing, synchronization information; and reproducing, using the generated synchronization information and by the information receiving apparatus, the second media content in synchronization with reproduction of the first media content by the second information receiving apparatus. ### Claim 12 At least one non-transitory computer readable storage medium storing processor-executable instructions that, when executed by at least one computer, causes the at least one computer to perform an information processing method for reproducing by an information receiving apparatus a second media content in synchronization with reproduction of a first media content, the first media content comprising audio content, the information processing method comprising: extracting an audio-derived feature from an audio signal generated via reproduction of the audio content by a second information receiving apparatus; obtaining, from an information providing apparatus different from the second information receiving apparatus, the second media content together with a synchronization feature, the synchronization feature being extracted from the audio content prior to the first media content being transmitted by the information providing apparatus; comparing the audio-derived feature with the synchronization feature; generating, based on results of the comparing, synchronization information; and reproducing using the generated synchronization information and by the information receiving apparatus the second media content in synchronization with reproduction of the first media content by the second information receiving apparatus. ### Claim 13 An information providing apparatus for transmitting a first media content and a second media content to one or more other devices, the first media content comprising audio content, the information providing apparatus comprising: circuitry configured to: extract a synchronization feature from the audio content of the first media content prior to transmission of the first media content to the one or more other devices; associate, in synchronization with the first media content, the synchronization feature with the second media content that has a time-synchronized relationship with the first media content; transmit the first media content to a first receiving device; and transmit the second media content together with the synchronization feature to a second receiving device different from the first receiving device. ### Claim 14 (depends on 13) The information providing apparatus of claim 13, wherein the circuitry is configured to transmit the second media content and the synchronization feature at a time determined based on a difference between arrival time of the first media content and the second media content. ### Claim 15 (depends on 13) The information providing apparatus of claim 13, wherein the circuitry is configured to transmit the second media content and the synchronization feature in response to a request for their transmission. ### Claim 16 (depends on 15) The information providing apparatus of claim 15, wherein the circuitry is configured to transmit the first media content in response to a request for its transmission. ### Claim 17 (depends on 13) The information providing apparatus of claim 13, wherein the circuitry is configured to: extract a plurality of features from a plurality of portions of the audio content of the first media content prior to transmission of the first media content to the one or more other devices, the plurality of features including the synchronization feature; and associate and transmit the extracted features with the second media content. ### Claim 18 (depends on 13) The information providing apparatus of claim 13, wherein the circuitry is configured to: downsample the synchronization feature to obtain a downsampled synchronization feature; and transmit the second media content and the downsampled synchronization feature. ### Claim 19 (depends on 13) The information receiving apparatus of claim 13, wherein the second media content includes the audio-derived feature. ### Claim 20 An information processing method for transmitting by an information providing apparatus a first media content and a second media content to one or more other devices, the first content comprising audio content, the method comprising: extracting a synchronization feature from the audio content of the first media content prior to transmission of the first media content to the one or more other devices; associating, in synchronization with the first media content, the synchronization feature with second media content that has a time-synchronized relationship with the first media content; transmitting the first media content to a first receiving device; and transmitting the second media content together with the synchronization feature to a second receiving device different from the first receiving device. ### Claim 21 At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer, causes the at least one computer to perform an information processing method for transmitting by an information providing apparatus a first media content and a second media content to one or more other devices, the first media content comprising audio content, the information processing method comprising: extracting a synchronization feature from the audio content of the first media content prior to transmission of the first media content to the one or more other devices; associating, in synchronization with the first media content, the synchronization feature with second media content that has a time-synchronized relationship with the first media content; transmitting the first media content to a first receiving device; and transmitting the second media content together with the synchronization feature to a second receiving device different from the first receiving device.