Mobile device with activity recognition
AssigneeNEC Corporation
InventorsKihyuk Sohn, Manmohan Chandraker, Xiang Yu
Life of this patent
Estimating the term…
Abstract
A computer-implemented method, system, and computer program product are provided for activity recognition in a mobile device. The method includes receiving a plurality of unlabeled videos from one or more cameras. The method also includes generating a classified video for each of the plurality of unlabeled videos by classifying an activity in each of the plurality of unlabeled videos. The method additionally includes storing the classified video in a location in a memory designated for videos of the activity in each of the classified videos.
Claims16
A mobile device utilizing activity recognition, the mobile device comprising:
- one or more cameras;
- a processor and memory coupled to the processor, the processing system programmed to:
- receive a plurality of unlabeled videos from the one or more cameras;
- generate a classified video for each of the plurality of unlabeled videos by classifying an activity in each of the plurality of unlabeled videos;
- and
- store the classified video in a location in the memory designated for videos of the activity in each of the classified videos,
- wherein the processing system is further programmed to utilize domain adversarial training system to generate the classified video, and
- wherein the domain adversarial training system employs a frame-wise discriminator to learn differences between classified videos and the unlabeled videos at the frame level and to backpropagate the differences into a feature extraction CNN and a feature aggregation system.
A computer program product for a mobile device with activity recognition, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
- receiving a plurality of unlabeled videos from one or more cameras;
- generating a classified video for each of the plurality of unlabeled videos by classifying an activity in each of the plurality of unlabeled videos;
- and
- storing the classified video in a location in a memory designated for videos of the activity in each of the classified videos,
- wherein the plurality of unlabeled videos includes first-person videos, and
- wherein the generating determines an average of the frame features, determines a weighted average of the frame features responsive to a weighting factor, and employs both a short-term memory neural network and a gated recurrent unit network.
A computer-implemented method for activity recognition in a mobile device, the method comprising:
- receiving a plurality of unlabeled videos from one or more cameras;
- generating a classified video for each of the plurality of unlabeled videos by classifying an activity in each of the plurality of unlabeled videos;
- and
- storing the classified video in a location in a memory designated for videos of the activity in each of the classified videos,
- wherein the plurality of unlabeled videos includes first-person videos, and
- wherein the generating determines an average of the frame features, determines a weighted average of the frame features responsive to a weighting factor, and employs both a short-term memory neural network and a gated recurrent unit network.
Dependencies are inferred from the claim text.