Method and system for motion vector-based video monitoring and event categorization

A method of processing a video stream, comprising:

• processing the video stream to detect a start of a first motion event candidate in the video stream;
• and
• in response to detecting the start of the first motion event candidate in the video stream, initiating event recognition processing on a first video segment associated with the start of the first motion event candidate, wherein initiating the event recognition processing further comprises:
• determining a motion track of a first object identified in the first video segment;
• generating a representative motion vector for the first motion event candidate based on the motion track of the first object;
• and
• sending the representative motion vector for the first motion event candidate to an event categorizer, wherein the event categorizer assigns a respective motion event category to the first motion event candidate based on the representative motion vector of the first motion event candidate,
• wherein determining the motion track of the object identified in the first video segment further comprises, based on a frame sequence of the first video segment:
• performing background estimation to obtain a background for the first video segment;
• performing object segmentation to identify one or more foreground objects in the first video segment by subtracting the obtained background from the frame sequence, the one or more foreground object including the object;
• and
• establishing a respective motion track for each of the one or more foreground objects by associating respective motion masks of the foreground object across multiple frames of the frame sequence, and
• wherein performing the object segmentation to identify one or more foreground objects and establishing the respective motion track for each of the one or more foreground objects further comprises:
• building a histogram of foreground pixels identified in the frame sequence of the first video segment, wherein the histogram specifies a frame count for each pixel location in a scene of the first video segment;
• filtering the histogram to remove regions below a predetermined frame count;
• segmenting the filtered histogram into the one or more motion regions;
• and
• selecting one or more dominant motion regions from the one or more motion regions based on a predetermined dominance criterion, where each dominant motion region corresponds to the respective motion track of a corresponding one of the one or more foreground objects.

A computing system for processing a video stream, comprising:

• one or more processors;
• and
• memory having instructions stored thereon, the instructions, when executed by the one or more processors, cause the processors to perform operations comprising:
• processing the video stream to detect a start of a first motion event candidate in the video stream, wherein processing comprises:
• obtaining a profile of a total motion pixel count for a current frame sequence in the video stream;
• in response to determining that the obtained profile meets a predetermined trigger criterion, determining that the current frame sequence includes a motion event candidate;
• identifying a beginning time for a portion of the profile meeting the predetermined trigger criterion;
• and
• designating the identified beginning time to be the start of the first motion event candidate;
• and
• in response to detecting the start of the first motion event candidate in the video stream, initiating event recognition processing on a first video segment associated with the start of the first motion event candidate, wherein initiating the event recognition processing further comprises:
• determining a motion track of a first object identified in the first video segment;
• generating a representative motion vector for the first motion event candidate based on the motion track of the first object;
• and
• sending the representative motion vector for the first motion event candidate to an event categorizer, wherein the event categorizer assigns a respective motion event category to the first motion event candidate based on the representative motion vector of the first motion event candidate.

A computing system for processing a video stream, comprising:

• one or more processors;
• and
• memory having instructions stored thereon, the instructions, when executed by the one or more processors, cause the processors to perform operations comprising:
• processing the video stream to detect a start of a first motion event candidate in the video stream;
• and
• in response to detecting the start of the first motion event candidate in the video stream, initiating event recognition processing on a first video segment associated with the start of the first motion event candidate, wherein initiating the event recognition processing further comprises:
• determining a motion track of a first object identified in the first video segment;
• generating a representative motion vector for the first motion event candidate based on the motion track of the first object;
• and
• sending the representative motion vector for the first motion event candidate to an event categorizer, wherein the event categorizer assigns a respective motion event category to the first motion event candidate based on the representative motion vector of the first motion event candidate,
• wherein determining the motion track of the object identified in the first video segment further comprises, based on a frame sequence of the first video segment:
• performing background estimation to obtain a background for the first video segment;
• performing object segmentation to identify one or more foreground objects in the first video segment by subtracting the obtained background from the frame sequence, the one or more foreground object including the object;
• and
• establishing a respective motion track for each of the one or more foreground objects by associating respective motion masks of the foreground object across multiple frames of the frame sequence, and
• wherein performing the object segmentation to identify one or more foreground objects and establishing the respective motion track for each of the one or more foreground objects further comprises:
• building a histogram of foreground pixels identified in the frame sequence of the first video segment, wherein the histogram specifies a frame count for each pixel location in a scene of the first video segment;
• filtering the histogram to remove regions below a predetermined frame count;
• segmenting the filtered histogram into the one or more motion regions;
• and
• selecting one or more dominant motion regions from the one or more motion regions based on a predetermined dominance criterion, where each dominant motion region corresponds to the respective motion track of a corresponding one of the one or more foreground objects.