Fine-grained video understanding via external memory using neural sampling
AssigneeSamsung Electronics Co., Ltd.
InventorSaket Gurukar
Life of this patent
Estimating the term…
Abstract
A method includes receiving a query at a query module and producing a query module output. The method also includes receiving a video at an external memory module. The method also includes generating a pool of video tokens from the video. The method also includes performing neural sampling to sample the pool of video tokens using a neural sampler in the memory sampling module. The method also includes storing the sampled video tokens in the external memory module. The method also includes providing a response to the query based on the sampled video tokens stored in the external memory module.
Claims20
A method comprising:
- receiving a query at a query module and producing a query module output;
- receiving a video at an external memory module;
- generating a pool of video tokens from the video;
- performing neural sampling to sample the pool of video tokens using a neural sampler in the memory sampling module;
- storing the sampled video tokens in the external memory module;
- and
- providing a response to the query based on the sampled video tokens stored in the external memory module.
An electronic device, comprising:
- at least one processing device configured to:
- receive a query at a query module and produce a query module output;
- receive a video at an external memory module;
- generate a pool of video tokens from the video;
- perform neural sampling to sample the pool of video tokens using a neural sampler in the memory sampling module;
- store the sampled video tokens in the external memory module;
- and
- provide a response to the query based on the sampled video tokens stored in the external memory module.
A non-transitory machine readable medium comprising instructions that when executed by at least one processor of an electronic device, causes the electronic device to:
- receive a query at a query module and producing a query module output;
- receive a video at an external memory module;
- generate a pool of video tokens from the video;
- perform neural sampling to sample the pool of video tokens using a neural sampler in the memory sampling module;
- store the sampled video tokens in the external memory module;
- and
- provide a response to the query based on the sampled video tokens stored in the external memory module.
Dependencies are inferred from the claim text.