Determining video provenance utilizing deep learning
AssigneesAdobe Inc., University of Surrey
InventorsAlexander Black, Van Tu Bui, John Collomosse, Simon Jenni, Viswanathan Swaminathan
Life of this patent
Estimating the term…
Abstract
The present disclosure relates to systems, methods, and non-transitory computer readable media that utilize deep learning to map query videos to known videos so as to identify a provenance of the query video or identify editorial manipulations of the query video relative to a known video. For example, the video comparison system includes a deep video comparator model that generates and compares visual and audio descriptors utilizing codewords and an inverse index. The deep video comparator model is robust and ignores discrepancies due to benign transformations that commonly occur during electronic video distribution.
Claims20
A computer-implemented method comprising:
- receiving a query to determine provenance information indicating a known video stored in a database for a query video, the query video comprising a modification from the known video;
- and
- determining, from a plurality of known videos, the known video that has been manipulated to generate the query video by:
- sub-dividing the query video into visual segments and audio segments;
- generating visual descriptors for the visual segments of the query video utilizing a visual neural network encoder;
- generating audio descriptors for the audio segments of the query video utilizing an audio neural network encoder;
- and
- determining that video segments from the known video are similar to the query video based on the visual descriptors and audio descriptors.
A non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
- receiving a query to determine provenance information indicating a known video stored in a database for a query video, the query video comprising a modification from the known video;
- and
- determining, from a plurality of known videos, the known video that has been manipulated to generate the query video by:
- sub-dividing the query video into visual segments and audio segments;
- generating visual descriptors for the visual segments of the query video utilizing a visual neural network encoder;
- generating audio descriptors for the audio segments of the query video utilizing an audio neural network encoder;
- and
- determining that video segments from the known video are similar to the query video based on the visual descriptors and audio descriptors.
A system comprising:
- one or more memory devices;
- and
- one or more processors coupled to the one or more memory devices, wherein the one or more processors cause the system to perform operations comprising:
- receiving a query to determine provenance information indicating a known video stored in a database for a query video, the query video comprising a modification from the known video;
- determining, from a plurality of known videos, the known video that has been manipulated to generate the query video;
- generating one or more visual indicators identifying visual changes in the query video relative to the known video;
- and
- displaying the query video with the one or more visual indicators overlaid on frames of the query video to indicate locations of the visual changes in the query video relative to the known video.
Dependencies are inferred from the claim text.