# US 2021/0344815 A1 · Information synchronization device, information synchronization method, and information synchronization program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0344815 A1 - Document ID: US20210344815A1 - Status: Published application - Kind: A1 - Application number: 17/280,231 - Filed: 2019-09-13 - Published: 2021-11-04 - Earliest claimed: 2018-09-28 - Assignees: Nippon Telegraph and Telephone Corporation - Applicants: Nippon Telegraph and Telephone Corporation - Inventors: Keisuke Hasegawa; Masato Ono; Koji Namba; Takahide Hoshide; Tetsuya Yamaguchi; Akira Ono - Web page: https://patentgrep.com/patent/US20210344815A1 ## CPC - H04N 5/06: Generation of synchronising signals - H04N 21/816: {involving special video data, e.g 3D video} - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - H04N 5/38: Transmitter circuitry {for the transmission of television signals according to analogue transmission standards} (H04N5/14 takes precedence) - H04N 21/242: Synchronisation processes, e.g. processing of PCR [Programme Clock References] {(arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18)} - H04N 21/8547: involving timestamps for synchronizing content - G06T 2207/10016: Video; Image sequence - H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Matching timings of transmitting pieces of information for reproducing video and audio with presence and handling loss are provided. An information synchronization device is configured to synchronize, to a reference timestamp, data in which one or more labels for identifying respective one or more subjects on a video and one or more pieces of location information for identifying respective locations of the one or more subjects are respectively associated. The information synchronization device includes a reception unit for receiving the data at each time, a buffer for storing the received data, and a location information interpolation unit for generating, when pieces of the location information at times before and after the reference timestamp is stored into the buffer for any of the labels, location information of the label at the reference timestamp by interpolation using the pieces of the location information. ## Claims ### Claim 1 An information synchronization device configured to synchronize, to a reference timestamp, data in which one or more labels for identifying respective one or more subjects on a video and one or more pieces of location information for identifying respective locations of the one or more subjects are respectively associated, the information synchronization device comprising: a reception unit, including one or more processors, configured to receive the data at each time; a buffer configured to store the received data; and a location information interpolation unit, including one or more processors, configured to generate, upon pieces of the location information at times before and after the reference timestamp being stored into the buffer for any of the labels, location information of the label at the reference timestamp by interpolation using the pieces of the location information. ### Claim 2 (depends on 1) The information synchronization device according to claim 1, comprising a frame rate conversion unit, including one or more processors, configured to convert and transmit a frame rate of the location information at the reference timestamp. ### Claim 3 (depends on 1) The information synchronization device according to claim 1, further comprising: a location information synchronization unit, including one or more processors, configured to allocate the location information to the reference timestamp at a time within a short period close to or including a timestamp of the location information, wherein the location information synchronization unit allocates, when another piece of location information has already been allocated to the reference timestamp to which the location information is to be allocated, no location information to the reference timestamp, and allocates the location information to the reference timestamp to which no location information has been allocated among reference timestamps within a predetermined period close to or including the timestamp of the location information. ### Claim 4 (depends on 3) The information synchronization device according to claim 3, wherein the location information synchronization unit is configured to allocate, when no location information has been allocated to two reference timestamps at a time before the timestamp of the location information, the location information to one of the two reference timestamps. ### Claim 5 An information synchronization method of synchronizing, to a reference timestamp, data in which one or more labels for identifying respective one or more subjects on a video and one or more pieces of location information for identifying respective locations of the one or more subjects are respectively associated, the method comprising: receiving, by a computer, the data at each time; storing, by the computer, the received data into a buffer; and generating, by the computer, upon pieces of the location information at times before and after the reference timestamp being stored into the buffer for any of the labels, location information of the label at the reference timestamp by interpolation using the pieces of the location information. ### Claim 6 (depends on 5) The information synchronization method according to claim 5, further comprising: allocating, by the computer, the location information to the reference timestamp at a time within a short period close to or including a timestamp of the location information, wherein allocating the location information comprises allocating, when another piece of location information has already been allocated to the reference timestamp to which the location information is to be allocated, no location information to the reference timestamp, and allocating the location information to the reference timestamp to which no location information has been allocated among reference timestamps within a predetermined period close to or including the timestamp of the location information. ### Claim 7 A non-transitory computer readable medium storing one or more instructions causing a computer to operate as an information synchronization device configured to synchronize, to a reference timestamp, data in which one or more labels for identifying respective one or more subjects on a video and one or more pieces of location information for identifying respective locations of the one or more subjects are respectively associated, the one or more instructions causing the computer to execute: receiving, by the computer, the data at each time; storing, by the computer, the received data into a buffer; and generating, by the computer, upon pieces of the location information at times before and after the reference timestamp being stored into the buffer for any of the labels, location information of the label at the reference timestamp by interpolation using the pieces of the location information. ### Claim 8 (depends on 5) The information synchronization method according to claim 5, further comprising: converting, by the computer, and transmit a frame rate of the location information at the reference timestamp. ### Claim 9 (depends on 6) The information synchronization method according to claim 6, further comprising: allocating, by the computer when no location information has been allocated to two reference timestamps at a time before the timestamp of the location information, the location information to one of the two reference timestamps. ### Claim 10 (depends on 7) The non-transitory computer readable medium according to claim 7, wherein the one or more instructions further cause the computer to execute: converting, by the computer, and transmit a frame rate of the location information at the reference timestamp. ### Claim 11 (depends on 7) The non-transitory computer readable medium according to claim 7, wherein the one or more instructions further cause the computer to execute: allocating, by the computer, the location information to the reference timestamp at a time within a short period close to or including a timestamp of the location information, wherein allocating the location information comprises allocating, when another piece of location information has already been allocated to the reference timestamp to which the location information is to be allocated, no location information to the reference timestamp, and allocating the location information to the reference timestamp to which no location information has been allocated among reference timestamps within a predetermined period close to or including the timestamp of the location information. ### Claim 12 (depends on 11) The non-transitory computer readable medium according to claim 11, wherein the one or more instructions further cause the computer to execute: allocating, by the computer when no location information has been allocated to two reference timestamps at a time before the timestamp of the location information, the location information to one of the two reference timestamps.