Shuffle, attend, and adapt: video domain adaptation by clip order prediction and clip attention alignment
AssigneeNEC Laboratories America, Inc.
InventorsGaurav Sharma, Samuel Schulter, Jinwoo Choi
Life of this patent
Estimating the term…
Abstract
A method for performing video domain adaptation for human action recognition is presented. The method includes using annotated source data from a source video and unannotated target data from a target video in an unsupervised domain adaptation setting, identifying and aligning discriminative clips in the source and target videos via an attention mechanism, and learning spatial-background invariant human action representations by employing a self-supervised clip order prediction loss for both the annotated source data and the unannotated target data.
Claims20
A computer-implemented method executed on a processor for performing video domain adaptation for human action recognition, the method comprising:
- using annotated source data from a source video and unannotated target data from a target video in an unsupervised domain adaptation setting;
- identifying and aligning discriminative clips in the source and target videos via an attention mechanism;
- and
- learning spatial-background invariant human action representations by employing a self-supervised clip order prediction loss for both the annotated source data and the unannotated target data.
A non-transitory computer-readable storage medium comprising a computer-readable program for performing video domain adaptation for human action recognition, wherein the computer-readable program when executed on a computer causes the computer to perform the steps of:
- using annotated source data from a source video and unannotated target data from a target video in an unsupervised domain adaptation setting;
- identifying and aligning discriminative clips in the source and target videos via an attention mechanism;
- and
- learning spatial-background invariant human action representations by employing a self-supervised clip order prediction loss for both the annotated source data and the unannotated target data.
A system for performing video domain adaptation for human action recognition, the system comprising:
- a memory;
- and
- one or more processors in communication with the memory configured to:
- use annotated source data from a source video and unannotated target data from a target video in an unsupervised domain adaptation setting;
- identify and align discriminative clips in the source and target videos via an attention mechanism;
- and
- learn spatial-background invariant human action representations by employing a self-supervised clip order prediction loss for both the annotated source data and the unannotated target data.
Dependencies are inferred from the claim text.