Automated gesture identification using neural networks

An artificial intelligence system for identifying a gesture communicated by a subject, the system comprising:

• at least one processor;
• and
• non-transitory memory including processor executable code, wherein the processor executable code upon execution by the at least one processor causes the at least one processor to:
• receive a plurality of images associated with the gesture;
• provide the plurality of images to a first 3-dimensional convolutional neural network (3D CNN) and a second 3D CNN, wherein the first 3D CNN is operable to produce motion information, wherein the second 3D CNN is operable to produce pose and color information, wherein the first 3D CNN is operable to implement an optical flow algorithm to detect the gesture;
• fuse the motion information and the pose and color information to produce an identification of the gesture;
• and
• determine whether the identification corresponds to a singular gesture across the plurality of images using a recurrent neural network (RNN) that comprises one or more long short-term memory (LTSM) units.

A method for identifying gestures communicated by a subject, comprising:

• receiving a plurality of images associated with the gesture;
• providing the plurality of images to a first 3-dimensional convolutional neural network (3D CNN) and a second 3D CNN, wherein the first 3D CNN is operable to produce motion information, wherein the second 3D CNN is operable to produce pose and color information, wherein the first 3D CNN is operable to implement an optical flow algorithm to detect the gesture;
• fusing the motion information and the pose and color information to produce an identification of the gesture;
• and
• determining whether the identification corresponds to a singular gesture across the plurality of images using a recurrent neural network (RNN) that comprises one or more long short-term memory (LTSM) units.