System and method for automatically discovering, characterizing, classifying and semi-automatically labeling animal behavior and quantitative phenotyping of behaviors in animals

An apparatus for automatically discovering, characterizing and classifying the behavior of an animal in an experimental area, comprising:

• a 3D depth camera that generates a video stream having a plurality of images of the experimental area with the animal in the experimental area, the images having both area and depth information;
• a data processing system having a processor and a memory containing program code which, when executed:
• (a) removes background noise from each of the plurality of images to generate processed images having light and dark areas, wherein the light areas comprise depth values above a threshold of a subtracted background and wherein the dark areas comprise depth values at or below the threshold of the subtracted background;
• (b) determines contours of the light areas in the plurality of processed images;
• (c) extracts at least one morphometric parameter from both area and depth image information within the contours to form a plurality of multi-dimensional data points, each data point representing the posture of the animal at a specific time, wherein one of the at least one morphometric parameters comprises a cropped three dimensional image of the animal;
• and
• (d) clusters the data points to output a set of clusters that each represent an animal behavior;
• (e) assigns each cluster a label that represents an animal behavior;
• and
• (f) outputs a visual representation of the set of clusters and corresponding labels.