Composite video frame replacement
AssigneeInternational Business Machines Corporation
InventorsZheng Ping AC Chu, Fan Xiao Xin, Yuan Jin, Xi Xi Liu
Life of this patent
Estimating the term…
Abstract
One or more frames within a video are segmented and classified according to information type. At least a portion of the information in the one or more frames is determined to be obscured. In response to this determination, information from audio data and the unobscured portion of the one or more frames is extracted. Using this extracted information, a set of replacement data is generated, according to the information type, for the obscured information in the one or more frames. The set of replacement data and the unobscured portion of the one or more frames is merged into one or more composite frames. These composite frames are displayed to a user.
Claims14
A computer-implemented method for reconstructing obscured video information, the method comprising:
- for one or more frames within a video generated by a camera, segmenting and classifying the one or more frames according to visual information type;
- determining at least a portion of the information in the one or more frames is obscured by an obstruction between the camera and the information;
- in response to the determination, extracting information from audio data and the unobscured portion of the one or more frames;
- linking, via a relational database, the extracted audio data to the obscured information in the one or more frames;
- using the extracted information and the relational database, generating a set of replacement data, according to visual information type, for the obscured information in the one or more frames, wherein metadata associated with the one or more frames is used to generate at least a portion of the set of replacement data, and wherein the linked extracted audio data is transcribed into a word-for-word replacement for a second portion of the set of replacement data;
- merging the set of replacement data and the unobscured portion of the one or more frames into one or more composite frames;
- displaying the one or more composite frames to a user;
- receiving, from the user, an indication to move an obstruction within the video to a target location in the one or more composite frames, wherein the obstruction is a presenter;
- and
- displaying, to the user, the one or more composite frames, wherein the obstruction is displayed at the target location, and wherein the replacement data is displayed at the original location of the obstruction.
A system for reconstructing obscured video information, comprising:
- a memory with program instructions included thereon;
- and
- a processor in communication with the memory, wherein the program instructions cause the processor to:
- for one or more frames within a video generated by a camera, segment and classify the one or more frames according to visual information type;
- determine at least a portion of the information in the one or more frames is obscured by an obstruction between the camera and the information;
- in response to the determination, extracting information from audio data and the unobscured portion of the one or more frames;
- linking, via a relational database, the extracted audio data to the obscured information in the one or more frames;
- using the extracted information and the relational database, generating a set of replacement data, according to visual information type, for the obscured information in the one or more frames, wherein metadata associated with the one or more frames is used to generate at least a portion of the set of replacement data, and wherein the linked extracted audio data is transcribed into a word-for-word replacement for a second portion of the set of replacement data;
- merging the set of replacement data and the unobscured portion of the one or more frames into one or more composite frames;
- displaying the one or more composite frames to a user;
- receiving, from the user, an indication to move an obstruction within the video to a target location in the one or more composite frames, wherein the obstruction is a presenter;
- and
- displaying, to the user, the one or more composite frames, wherein the obstruction is displayed at the target location, and wherein the replacement data is displayed at the original location of the obstruction.
A computer program product for reconstructing obscured video information, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a device to cause the device to:
- for one or more frames within a video generated by a camera, segment and classify the one or more frames according to visual information type;
- determine at least a portion of the information in the one or more frames is obscured by an obstruction between the camera and the information;
- in response to the determination, extracting information from audio data and the unobscured portion of the one or more frames;
- linking, via a relational database, the extracted audio data to the obscured information in the one or more frames;
- using the extracted information and the relational database, generating a set of replacement data, according to visual information type, for the obscured information in the one or more frames, wherein metadata associated with the one or more frames is used to generate at least a portion of the set of replacement data, and wherein the linked extracted audio data is transcribed into a word-for-word replacement for a second portion of the set of replacement data;
- merging the set of replacement data and the unobscured portion of the one or more frames into one or more composite frames;
- displaying the one or more composite frames to a user;
- receiving, from the user, an indication to move an obstruction within the video to a target location in the one or more composite frames, wherein the obstruction is a presenter;
- and
- displaying, to the user, the one or more composite frames, wherein the obstruction is displayed at the target location, and wherein the replacement data is displayed at the original location of the obstruction.
Dependencies are inferred from the claim text.