# US 12,003,762 B2 · Information completion apparatus, method and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,003,762 B2 - Document ID: US12003762B2 - Status: Granted - Kind: B2 - Application number: 17/613,464 - Filed: 2019-05-28 - Published: 2022-07-21 - Granted: 2024-06-04 - Earliest claimed: 2019-05-28 - Assignees: Nippon Telegraph and Telephone Corporation - Applicants: Nippon Telegraph and Telephone Corporation - Inventors: Mariko Isogawa; Dan Mikami; Kosuke Takahashi; Yoshinori Kusachi - Web page: https://patentgrep.com/patent/US12003762B2 ## CPC - H04N 19/577: Motion compensation with bidirectional frame interpolation, i.e. using B-pictures - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06V 2201/07: Target detection - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A technique for interpolating positional information of a target in an image of a frame in which positional information of the target has not been acquired. An information interpolation device includes: a target information acquisition unit 4 acquiring target information that is information related to a target in an image of each frame composing an input video; an indicator determination unit 5 determining, based on the target information, an indicator indicating validity as a starting frame for each frame, the starting frame starting predetermined image processing on a target; a starting frame determination unit 6 determining a starting frame based on the determined indicator; and a target information interpolation unit 7 interpolating, when a frame in which positional information of a target is not included in target information exists among frames within a predetermined number from the determined starting frame, positional information of the target of the frame that does not include the positional information of the target by using positional information of the target included in target information of a frame other than the frame that does not include the positional information of the target. ## Claims ### Claim 1 An information interpolation device comprising: processing circuitry configured to: acquire target information that is information related to a target in an image of each frame composing an input video; performing either one of: when a frame next to a frame in which target information includes information indicating that a target does not exist is a frame in which target information includes information indicating that the target exists, determine, based on the target information, an indicator indicating high validity as a starting frame for each frame, the starting frame starting predetermined image processing on a target, the starting frame represents a frame to start predetermined image processing on a target as an indicator of the frame in which the target information includes the information indicating that the target exists; or calculate the indicator based on at least one of (i) information on whether or not a target exists, (ii) an evaluation value indicating a size of an overlapping region of a target and an object other than the target, (iii) an evaluation value representing how a target is close to a predetermined shape, and (iv) an evaluation value representing a magnitude of variation of a pattern in a target, that are included in target information or calculated from target information, wherein the evaluation value representing the magnitude of variation of the pattern in the target is an indicator representing proximity between an image of the target in each frame and a predetermined template image, or a sum of magnitudes of differences in images of the target in consecutive frames; determine a starting frame based on the indicator; and interpolate, when a frame in which positional information of a target is not included in target information exists among frames within a predetermined number from the determined starting frame, positional information of the target of the frame that does not include the positional information of the target by using positional information of the target included in target information of a frame other than the frame that does not include the positional information of the target. ### Claim 2 (depends on 1) The information interpolation device according to claim 1, wherein the processing circuitry is configured to determine the indicator only for a frame necessary for processing of the starting frame determination unit, and the processing circuitry is configured to acquire target information only for a frame necessary for processing of the processing circuitry. ### Claim 3 An information interpolation method comprising: acquiring target information that is information related to a target in an image of each frame composing an input video; performing either one of: when a frame next to a frame in which target information includes information indicating that a target does not exist is a frame in which target information includes information indicating that the target exists, determining, based on the target information, an indicator indicating high validity as a starting frame for each frame, the starting frame starting predetermined image processing on a target, the starting frame represents a frame to start predetermined image processing on a target as an indicator of the frame in which the target information includes the information indicating that the target exists; or calculate the indicator based on at least one of (i) information on whether or not a target exists, (ii) an evaluation value indicating a size of an overlapping region of a target and an object other than the target, (iii) an evaluation value representing how a target is close to a predetermined shape, and (iv) an evaluation value representing a magnitude of variation of a pattern in a target, that are included in target information or calculated from target information, wherein the evaluation value representing the magnitude of variation of the pattern in the target is an indicator representing proximity between an image of the target in each frame and a predetermined template image, or a sum of magnitudes of differences in images of the target in consecutive frames; determining a starting frame based on the indicator; and interpolating, when a frame in which positional information of a target is not included in target information exists among frames within a predetermined number from the determined starting frame, positional information of the target of the frame that does not include the positional information of the target by using positional information of the target included in target information of a frame other than the frame that does not include the positional information of the target. ### Claim 4 (depends on 3) A non-transitory computer readable medium that stores a program causing a computer to perform each step of the information interpolation method according to claim 3. ### Claim 5 An information interpolation device comprising: processing circuitry configured to: acquire target information that is information related to a target in an image of each frame composing an input video; calculate an indicator based on at least one of (i) information on whether or not a target exists, (ii) an evaluation value indicating a size of an overlapping region of a target and an object other than the target, (iii) an evaluation value representing how a target is close to a predetermined shape, and (iv) an evaluation value representing a magnitude of variation of a pattern in a target, that are included in target information or calculated from target information, wherein the evaluation value representing the magnitude of variation of the pattern in the target is an indicator representing proximity between an image of the target in each frame and a predetermined template image, or a sum of magnitudes of differences in images of the target in consecutive frames; determine a starting frame based on the calculated indicator; and interpolate, when a frame in which positional information of a target is not included in target information exists among frames within a predetermined number from the determined starting frame, positional information of the target of the frame that does not include the positional information of the target by using positional information of the target included in target information of a frame other than the frame that does not include the positional information of the target.