Action recognition apparatus and method
AssigneesHyundai Motor Company, Kia Corporation
InventorSohee Kim
Life of this patent
Estimating the term…
Abstract
An action recognition method of an action recognition apparatus operated by a processor includes, receiving a video image; detecting position values and joint prediction values of each of a plurality of joints for each of the plurality of image frames constituting the video image; calculating a position difference of each of the plurality of joints for each of the plurality of image frames; generating input data including a position value, a predicted value, and a position difference of each of the plurality of joints for each of the plurality of image frames; and detecting a type of action by inputting the input data into an action recognition model.
Claims17
An action recognition method comprising:
- receiving, by an action recognition apparatus operated by a processor, a video image;
- detecting, by a data obtainer, both a position value and a joint prediction value of each joint among a plurality of joints for each image frame among a plurality of image frames constituting the video image;
- calculating, by an input data generator, a position difference of the each joint for the each image frame;
- generating, by the input data generator, input data including the position value, the joint prediction value, and the position difference of the each joint for the each image frame;
- and
- detecting, by an action predictor including an action recognition model, a type of action by inputting the input data into the action recognition model,
- wherein the action recognition model is a convolutional neural network (CNN),
- wherein the input data includes a first input data and a second input data,
- wherein the first input data includes:
- 2D image data of a first channel including an x-coordinate of the each joint for the plurality of image frames;
- 2D image data of a second channel including a y-coordinate of the each joint for the plurality of image frames;
- and
- 2D image data of a third channel including the joint prediction value of the each joint for the plurality of image frames, and
- wherein the second input data includes:
- 2D image data of a fourth channel including an x-coordinate position difference of the each joint for the plurality of image frames and a y-coordinate position difference of the each joint for the plurality of image frames.
An action recognition apparatus that recognizes actions from an input video image, the action recognition apparatus operated by a processor and comprising:
- a data obtainer configured to detect both a position value and a joint prediction value of each joint among a plurality of joints for each image frame among a plurality of image frames constituting the input video image;
- an input data generator configured to calculate a position difference of the each joint for the each image frame and generate an input data including the position value, the joint prediction value, and the position difference of the each joint for the each image frame;
- and
- an action predictor configured to include an action recognition model and recognize a type of action based on results output by inputting the input data into the action recognition model,
- wherein the action recognition model is a convolutional neural network (CNN),
- wherein the input data includes a first input data and a second input data,
- wherein the first input data includes:
- 2D image data of a first channel including an x-coordinate of the each joint for the plurality of image frames;
- 2D image data of a second channel including a y-coordinate of the each joint for the plurality of image frames;
- and
- 2D image data of a third channel including the joint prediction value of the each joint for the plurality of image frames, and
- wherein the second input data includes:
- 2D image data of a fourth channel including an x-coordinate position difference of the each joint for the plurality of image frames and a y-coordinate position difference of the each joint for the plurality of image frames.
Dependencies are inferred from the claim text.