Systems and methods for behavior detection using 3D tracking and machine learning

A behavioral classification system, comprising:

• a plurality of 3D imaging systems;
• and
• a behavioral classification computer system comprising at least one memory and at least one microprocessor directed by at least a classification application stored in the at least one memory to:
• control the plurality of 3D imaging systems to each acquire a sequence of frames of image data comprising depth information;
• store at least a portion of each of the sequences of frames of image data comprising depth information in the at least one memory;
• and
• for each of the sequences of frames of image data:
• identify at least a primary subject interacting with a secondary subject within a given sequence of frames of image data comprising depth information;
• determine poses for at least the primary subject and the secondary subject within a plurality of frames from the given sequence of frames of image data;
• extract a set of parameters describing the poses and movement of at least the primary and secondary subjects from the plurality of frames from the given sequence of frames of image data;
• detect a social behavior performed by at least the primary subject and involving at least the secondary subject, wherein the primary subject occludes at least a portion of the secondary subject, using a classifier trained to discriminate between a plurality of social behaviors based upon the set of parameters describing poses and movement of a plurality of subjects extracted from a plurality of frames of image data comprising depth information;
• and
• store the detected social behavior and an association with the primary subject in the at least one memory.

An behavior classification system, comprising:

• a microprocessor;
• and
• memory containing a classification application;
• wherein the classification application directs the microprocessor to:
• identify at least a primary subject interacting with a secondary subject within a sequence of frames of image data comprising depth information, where the sequence of frames of image data are captured from a viewpoint of the secondary subject;
• determine poses for at least the primary subject within a plurality of frames from the sequence of frames of image data;
• extract a set of parameters describing the poses and movement of at least the primary subject from the plurality of frames from the sequence of frames of image data;
• and
• detect a social behavior performed by the primary subject and involving at least the secondary subject, wherein the primary subject occludes at least a portion of the secondary subject, using a classifier trained to discriminate between a plurality of social behaviors based upon the set of parameters describing poses and movement of a first subject with respect to at least a second subject extracted from a plurality of frames of image data comprising depth information.

A behavioral classification system, comprising:

• a microprocessor;
• memory containing a classification application;
• wherein the classification application directs the microprocessor to:
• identify a primary subject and a secondary subject within a sequence of frames of image data comprising depth information;
• determine poses of the primary subject and the secondary subject within a plurality of frames from the sequence of frames of image data;
• extract a set of parameters describing the poses and movement of the primary subject and the secondary subject from the plurality of frames from the sequence of frames of image data;
• detect a behavior performed by at least the primary subject and involving at least the secondary subject, wherein the primary subject occludes at least a portion of the secondary subject, using a classifier trained to discriminate between a plurality of behaviors based upon the set of parameters describing poses and movement of a subject extracted from a plurality of frames of image data comprising depth information;
• and
• infer a genotype for the primary subject based upon behavior including the detected behavior performed by the primary subject.