Object tracking in video with visual constraints

A system for tracking an object in a frame of a video, comprising:

• a pose constraint module that determines a pose constraint value based on a pose model and at least a portion the frame;
• an alignment constraint module that determines an alignment confidence score based on an alignment model and at least a portion the frame;
• an adaptive module that determines an adaptive term value based on an adaptive model and at least a portion of a frame in the video;
• a visual constrainer that determines an energy value based on the adaptive term value, the pose constraint value, and the alignment confidence score;
• and
• a tracker that determines a resultant tracking state based on the energy value, the resultant tracking state defining a likely position of the object in the frame given the object's likely position in a set of previous frames in the video,
• wherein the adaptive model is determined based on at least one previous frame of the video,
• wherein the pose model is determined based on different possible appearances of the object, and
• wherein the alignment model detects misalignments of the tracked object.