Application-specific object-based segmentation and recognition system

A video detection apparatus for detecting and following movement of a predefined object of interest within the video data of the output of a video camera encoded in frame-wise data, each frame occurring at sequential time intervals, and including the segmentation of at least one portion of the frame-wise data, an object segment, as potentially including the preselected object of interest, and including vertical edge pixels and horizontal edge pixels detected to be present within the object segment, comprising:

• an edge symmetry detection means comprising:
• a vertical symmetry detection means for computing the vertical symmetry of at least one of the set of vertical edge pixels and the set of horizontal edge pixels in each column about a horizontal axis passing through the object segment, and producing a vertical symmetry value for columns within the object segment;
• a horizontal symmetry detection means for computing the horizontal symmetry of at least one of the set of vertical edge pixels and set of horizontal edge pixels in each row about a vertical axis passing through the object segment, and producing a horizontal symmetry value for rows within the object segment;
• and,
• an edge symmetry feature set computation means for computing an edge symmetry feature set for the object segment based upon the vertical symmetry values and the horizontal symmetry values;
• and,
• a background modeling means comprising:
• a background modeling means for isolating from a current frame at least one segment of the current frame, each of the at least one segments containing a possible object of interest within the current frame, on a frame-wise basis, and providing as an output each of the at least one segments;
• an object of interest classifying means for determining whether or not any segment of the at least one segments output from the background modeling means includes a object of interest and for characterizing any such segment as an object segment, utilizing at least in part the edge symmetry feature set;
• an object segment tracking means for tracking the location within the current frame of any object segment, and determining a projected location of the object segment in a subsequent frame, and providing the background modeling means with the projected location in the subsequent frame of the object segment;
• and
• wherein the background modeling means includes means for providing as its output the projected location of the object segment in the subsequent frame in lieu of a segment from the subsequent frame determined by the background modeling means to contain a possible object of interest or in the event that the background modeling means is unable in the subsequent frame to locate any segment containing a possible object of interest.

A video detection method for detecting and following movement of a predefined object of interest within the video data of the output of a video camera encoded in frame-wise data, each frame occurring at sequential time intervals, and including the segmentation of at least one portion of the frame-wise data, an object segment, as potentially including the preselected object of interest, and including vertical edge pixels and horizontal edge pixels detected to be present within the object segment, comprising:

• utilizing an edge symmetry detection unit:
• computing the vertical symmetry of at least one of the set of vertical edge pixels and the set of horizontal edge pixels in each column about a horizontal axis passing through the object segment, and producing a vertical symmetry value for columns within the object segment;
• computing the horizontal symmetry of at least one of the set of vertical edge pixels and set of horizontal edge pixels in each row about a vertical axis passing through the object segment, and producing a horizontal symmetry value for rows within the object segment;
• and,
• computing an edge symmetry feature set for the object segment based upon the vertical symmetry values and the horizontal symmetry values;
• and,
• utilizing a background modeling unit comprising:
• isolating from a current frame at least one segment of the current frame, each of the at least one segments containing a possible object of interest within the current frame, on a frame-wise basis, and providing as an output each of the at least one segments;
• determining whether or not any segment of the at least one segments output from the background modeling means includes a object of interest and for characterizing any such segment as an object segment, utilizing at least in part the edge symmetry feature set;
• tracking the location within the current frame of any object segment, and determining a projected location of the object segment in a subsequent frame, and providing the background modeling means with the projected location in the subsequent frame of the object segment;
• and
• wherein the background modeling unit includes means for providing as its output the projected location of the object segment in the subsequent frame in lieu of a segment from the subsequent frame determined by the background modeling means to contain a possible object of interest or in the event that the background modeling means is unable in the subsequent frame to locate any segment containing a possible object of interest.

A video detection apparatus adapted to detect and follow movement of a predefined object of interest within the video data of the output of a video camera encoded in frame-wise data, each frame occurring at sequential time intervals, and including the segmentation of at least one portion of the frame-wise data, an object segment, as potentially including the preselected object of interest, and including vertical edge pixels and horizontal edge pixels detected to be present within the object segment, comprising:

• an edge symmetry detection unit comprising:
• a vertical symmetry detector adapted to compute the vertical symmetry of the set of vertical edge pixels and the set of horizontal edge pixels in each column about a horizontal axis passing through the object segment, and to produce a vertical symmetry value for columns within the object segment;
• a horizontal symmetry detector adapted to compute the horizontal symmetry of the set of vertical edge pixels and set of horizontal edge pixels in each row about a vertical axis passing through the object segment, and to produce a horizontal symmetry value for rows within the object segment;
• and,
• an edge symmetry feature set computation apparatus adapted to compute an edge symmetry feature set for the object segment based upon the vertical symmetry values and the horizontal symmetry values;
• and,
• a background modeling unit comprising:
• a background modeling apparatus adapted to isolate from a current frame at least one segment of the current frame, each of the at least one segments containing a possible object of interest within the current frame, on a frame-wise basis, and to provide as an output each of the at least one segments;
• an object of interest classifying apparatus adapted to determine whether or not any segment of the at least one segments output from the background modeling apparatus includes a object of interest and to characterize any such segment as an object segment, utilizing at least in part the edge symmetry feature set;
• an object segment tracking apparatus adapted to track the location within the current frame of any object segment, and to determine a projected location of the object segment in a subsequent frame, and to provide the background modeling apparatus with the projected location in the subsequent frame of the object segment;
• wherein the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in lieu of a segment from the subsequent frame determined by the background modeling apparatus to contain a possible object of interest or in the event that the background modeling apparatus is unable in the subsequent frame to locate any segment containing a possible object of interest;
• and,
• the object segment tracking apparatus further comprising:
• a matching apparatus adapted to match the location of an object segment in the current frame to one of a plurality of projected locations of the object segment in the current frame, which projections are based upon the location of the respective object segment in at least one prior frame;
• and,
• a track provider adapted to receive and store the location of an object segment in the current frame and over a plurality of prior frames and adapted to provide the projections of the location of the object segment in a subsequent frame based upon the locations of the object segments in the current frame and the plurality of prior frames.

A video detection apparatus adapted to detect and follow movement of a predefined object of interest within the video data of the output of a video camera encoded in frame-wise data, each frame occurring at sequential time intervals, and including the segmentation of at least one portion of the frame-wise data, an object segment, as potentially including the preselected object of interest, and including vertical edge pixels and horizontal edge pixels detected to be present within the object segment, comprising:

• an edge symmetry detection unit comprising:
• a vertical symmetry detector adapted to compute the vertical symmetry of the set of vertical edge pixels and the set of horizontal edge pixels in each column about a horizontal axis passing through the object segment, and to produce a vertical symmetry value for columns within the object segment;
• a horizontal symmetry detector adapted to compute the horizontal symmetry of the set of vertical edge pixels and set of horizontal edge pixels in each row about a vertical axis passing through the object segment, and to produce a horizontal symmetry value for rows within the object segment;
• and,
• an edge symmetry feature set computation apparatus adapted to compute an edge symmetry feature set for the object segment based upon the vertical symmetry values and the horizontal symmetry values;
• and,
• a background modeling unit comprising:
• a background modeling apparatus adapted to isolate from a current frame at least one segment of the current frame, each of the at least one segments containing a possible object of interest within the current frame, on a frame-wise basis, and to provide as an output each of the at least one segments;
• an object of interest classifying apparatus adapted to determine whether or not any segment of the at least one segments output from the background modeling apparatus includes a object of interest and to characterize any such segment as an object segment, utilizing at least in part the edge symmetry feature set;
• an object segment tracking apparatus adapted to track the location within the current frame of any object segment, and to determine a projected location of the object segment in a subsequent frame, and to provide the background modeling apparatus with the projected location in the subsequent frame of the object segment;
• wherein the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in lieu of a segment from the subsequent frame determined by the background modeling apparatus to contain a possible object of interest or in the event that the background modeling apparatus is unable in the subsequent frame to locate any segment containing a possible object of interest;
• and,
• the object segment tracking apparatus further comprising:
• a matching apparatus adapted to match the location of an object segment in the current frame to one of a plurality of projected locations of the object segment in the current frame, which projections are based upon the location of the respective object segment in at least one prior frame;
• a track provider adapted to receive and store the location of an object segment in the current frame and over a plurality of prior frames and adapted to provide the projections of the location of the object segment in a subsequent frame based upon the locations of the object segments in the current frame and the plurality of prior frames;
• and,
• the vertical symmetry detector is adapted to compute the vertical symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given column i as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each column i on opposing sides of the horizontal axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the horizontal axis;
• and,
• the horizontal symmetry detector is adapted to compute the horizontal symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given row j as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each row j on opposing sides of the vertical axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the vertical axis.

A video detection apparatus adapted to detect and follow movement of a predefined object of interest within the video data of the output of a video camera encoded in frame-wise data, each frame occurring at sequential time intervals, and including the segmentation of at least one portion of the frame-wise data, an object segment, as potentially including the preselected object of interest, and including vertical edge pixels and horizontal edge pixels detected to be present within the object segment, comprising:

• an edge symmetry detection unit comprising:
• a vertical symmetry detector adapted to compute the vertical symmetry of the set of vertical edge pixels and the set of horizontal edge pixels in each column about a horizontal axis passing through the object segment, and to produce a vertical symmetry value for columns within the object segment;
• a horizontal symmetry detector adapted to compute the horizontal symmetry of the set of vertical edge pixels and set of horizontal edge pixels in each row about a vertical axis passing through the object segment, and to produce a horizontal symmetry value for rows within the object segment;
• and,
• an edge symmetry feature set computation apparatus adapted to compute an edge symmetry feature set for the object segment based upon the vertical symmetry values and the horizontal symmetry values;
• and,
• a background modeling unit comprising:
• a background modeling apparatus adapted to isolate from a current frame at least one segment of the current frame, each of the at least one segments containing a possible object of interest within the current frame, on a frame-wise basis, and to provide as an output each of the at least one segments;
• an object of interest classifying apparatus adapted to determine whether or not any segment of the at least one segments output from the background modeling apparatus includes a object of interest and to characterize any such segment as an object segment, utilizing at least in part the edge symmetry feature set;
• an object segment tracking apparatus adapted to track the location within the current frame of any object segment, and to determine a projected location of the object segment in a subsequent frame, and to provide the background modeling apparatus with the projected location in the subsequent frame of the object segment;
• wherein the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in lieu of a segment from the subsequent frame determined by the background modeling apparatus to contain a possible object of interest or in the event that the background modeling apparatus is unable in the subsequent frame to locate any segment containing a possible object of interest;
• and,
• the object segment tracking apparatus further comprising:
• a matching apparatus adapted to match the location of an object segment in the current frame to one of a plurality of projected locations of the object segment in the current frame, which projections are based upon the location of the respective object segment in at least one prior frame;
• a track provider adapted to receive and store the location of an object segment in the current frame and over a plurality of prior frames and adapted to provide the projections of the location of the object segment in a subsequent frame based upon the locations of the object segments in the current frame and the plurality of prior frames;
• and,
• the vertical symmetry detector is adapted to compute the vertical symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given column i as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each column i on opposing sides of the horizontal axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the horizontal axis;
• and,
• the horizontal symmetry detector is adapted to compute the horizontal symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given row j as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each row j on opposing sides of the vertical axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the vertical axis;
• and,
• the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in order to override any conflict in the labeling of pixels in the subsequent frame as part of a segment containing a possible object of interest.

A video detection apparatus adapted to detect and follow movement of a predefined object of interest within the video data of the output of a video camera encoded in frame-wise data, each frame occurring at sequential time intervals, and including the segmentation of at least one portion of the frame-wise data, an object segment, as potentially including the preselected object of interest, and including vertical edge pixels and horizontal edge pixels detected to be present within the object segment, comprising:

• an edge symmetry detection unit comprising:
• a vertical symmetry detector adapted to compute the vertical symmetry of the set of vertical edge pixels and the set of horizontal edge pixels in each column about a horizontal axis passing through the object segment, and to produce a vertical symmetry value for columns within the object segment;
• a horizontal symmetry detector adapted to compute the horizontal symmetry of the set of vertical edge pixels and set of horizontal edge pixels in each row about a vertical axis passing through the object segment, and to produce a horizontal symmetry value for rows within the object segment;
• and,
• an edge symmetry feature set computation apparatus adapted to compute an edge symmetry feature set for the object segment based upon the vertical symmetry values and the horizontal symmetry values;
• and,
• a background modeling unit comprising:
• a background modeling apparatus adapted to isolate from a current frame at least one segment of the current frame, each of the at least one segments containing a possible object of interest within the current frame, on a frame-wise basis, and to provide as an output each of the at least one segments;
• an object of interest classifying apparatus adapted to determine whether or not any segment of the at least one segments output from the background modeling apparatus includes a object of interest and to characterize any such segment as an object segment, utilizing at least in part the edge symmetry feature set;
• an object segment tracking apparatus adapted to track the location within the current frame of any object segment, and to determine a projected location of the object segment in a subsequent frame, and to provide the background modeling apparatus with the projected location in the subsequent frame of the object segment;
• wherein the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in lieu of a segment from the subsequent frame determined by the background modeling apparatus to contain a possible object of interest or in the event that the background modeling apparatus is unable in the subsequent frame to locate any segment containing a possible object of interest;
• and,
• the object segment tracking apparatus further comprising:
• a matching apparatus adapted to match the location of an object segment in the current frame to one of a plurality of projected locations of the object segment in the current frame, which projections are based upon the location of the respective object segment in at least one prior frame;
• a track provider adapted to receive and store the location of an object segment in the current frame and over a plurality of prior frames and adapted to provide the projections of the location of the object segment in a subsequent frame based upon the locations of the object segments in the current frame and the plurality of prior frames;
• and,
• the vertical symmetry detector is adapted to compute the vertical symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given column i as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each column i on opposing sides of the horizontal axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the horizontal axis;
• and,
• the horizontal symmetry detector is adapted to compute the horizontal symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given row j as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each row j on opposing sides of the vertical axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the vertical axis;
• the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in order to override any conflict in the labeling of pixels in the subsequent frame as part of a segment containing a possible object of interest;
• and,
• the feature set computation apparatus further comprising:
• a horizontal fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of horizontal fuzzy sets each having a selected weighting function applicable to a selected group of columns within the object segment, with the columns grouped according to position along the horizontal axis in the object segment;
• and,
• a vertical fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of vertical fuzzy sets each having a selected weighting function applicable to a selected group of rows within the object segment, with the rows grouped according to position along the vertical axis in the object segment.

A video detection apparatus adapted to detect and follow movement of a predefined object of interest within the video data of the output of a video camera encoded in frame-wise data, each frame occurring at sequential time intervals, and including the segmentation of at least one portion of the frame-wise data, an object segment, as potentially including the preselected object of interest, and including vertical edge pixels and horizontal edge pixels detected to be present within the object segment, comprising:

• an edge symmetry detection unit comprising:
• a vertical symmetry detector adapted to compute the vertical symmetry of the set of vertical edge pixels and the set of horizontal edge pixels in each column about a horizontal axis passing through the object segment, and to produce a vertical symmetry value for columns within the object segment;
• a horizontal symmetry detector adapted to compute the horizontal symmetry of the set of vertical edge pixels and set of horizontal edge pixels in each row about a vertical axis passing through the object segment, and to produce a horizontal symmetry value for rows within the object segment;
• and,
• an edge symmetry feature set computation apparatus adapted to compute an edge symmetry feature set for the object segment based upon the vertical symmetry values and the horizontal symmetry values;
• and,
• a background modeling unit comprising:
• a background modeling apparatus adapted to isolate from a current frame at least one segment of the current frame, each of the at least one segments containing a possible object of interest within the current frame, on a frame-wise basis, and to provide as an output each of the at least one segments;
• an object of interest classifying apparatus adapted to determine whether or not any segment of the at least one segments output from the background modeling apparatus includes a object of interest and to characterize any such segment as an object segment, utilizing at least in part the edge symmetry feature set;
• an object segment tracking apparatus adapted to track the location within the current frame of any object segment, and to determine a projected location of the object segment in a subsequent frame, and to provide the background modeling apparatus with the projected location in the subsequent frame of the object segment;
• wherein the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in lieu of a segment from the subsequent frame determined by the background modeling apparatus to contain a possible object of interest or in the event that the background modeling apparatus is unable in the subsequent frame to locate any segment containing a possible object of interest;
• and,
• the object segment tracking apparatus further comprising:
• a matching apparatus adapted to match the location of an object segment in the current frame to one of a plurality of projected locations of the object segment in the current frame, which projections are based upon the location of the respective object segment in at least one prior frame;
• a track provider adapted to receive and store the location of an object segment in the current frame and over a plurality of prior frames and adapted to provide the projections of the location of the object segment in a subsequent frame based upon the locations of the object segments in the current frame and the plurality of prior frames;
• and,
• the vertical symmetry detector is adapted to compute the vertical symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given column i as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each column i on opposing sides of the horizontal axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the horizontal axis;
• and,
• the horizontal symmetry detector is adapted to compute the horizontal symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given row j as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each row j on opposing sides of the vertical axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the vertical axis;
• the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in order to override any conflict in the labeling of pixels in the subsequent frame as part of a segment containing a possible object of interest;
• the feature set computation apparatus further comprising:
• a horizontal fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of horizontal fuzzy sets each having a selected weighting function applicable to a selected group of columns within the object segment, with the columns grouped according to position along the horizontal axis in the object segment;
• and,
• a vertical fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of vertical fuzzy sets each having a selected weighting function applicable to a selected group of rows within the object segment, with the rows grouped according to position along the vertical axis in the object segment;
• and,
• the background modeling apparatus is adapted to isolate from a current frame the at least one segment of the current frame utilizing the video object plane extracted from the current frame.

A video detection apparatus adapted to detect and follow movement of a predefined object of interest within the video data of the output of a video camera encoded in frame-wise data, each frame occurring at sequential time intervals, and including the segmentation of at least one portion of the frame-wise data, an object segment, as potentially including the preselected object of interest, and including vertical edge pixels and horizontal edge pixels detected to be present within the object segment, comprising:

• an edge symmetry detection unit comprising:
• a vertical symmetry detector adapted to compute the vertical symmetry of the set of vertical edge pixels and the set of horizontal edge pixels in each column about a horizontal axis passing through the object segment, and to produce a vertical symmetry value for columns within the object segment;
• a horizontal symmetry detector adapted to compute the horizontal symmetry of the set of vertical edge pixels and set of horizontal edge pixels in each row about a vertical axis passing through the object segment, and to produce a horizontal symmetry value for rows within the object segment;
• and,
• an edge symmetry feature set computation apparatus adapted to compute an edge symmetry feature set for the object segment based upon the vertical symmetry values and the horizontal symmetry values;
• and,
• a background modeling unit comprising:
• a background modeling apparatus adapted to isolate from a current frame at least one segment of the current frame, each of the at least one segments containing a possible object of interest within the current frame, on a frame-wise basis, and to provide as an output each of the at least one segments;
• an object of interest classifying apparatus adapted to determine whether or not any segment of the at least one segments output from the background modeling apparatus includes a object of interest and to characterize any such segment as an object segment, utilizing at least in part the edge symmetry feature set;
• an object segment tracking apparatus adapted to track the location within the current frame of any object segment, and to determine a projected location of the object segment in a subsequent frame, and to provide the background modeling apparatus with the projected location in the subsequent frame of the object segment;
• wherein the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in lieu of a segment from the subsequent frame determined by the background modeling apparatus to contain a possible object of interest or in the event that the background modeling apparatus is unable in the subsequent frame to locate any segment containing a possible object of interest;
• and,
• the object segment tracking apparatus further comprising:
• a matching apparatus adapted to match the location of an object segment in the current frame to one of a plurality of projected locations of the object segment in the current frame, which projections are based upon the location of the respective object segment in at least one prior frame;
• a track provider adapted to receive and store the location of an object segment in the current frame and over a plurality of prior frames and adapted to provide the projections of the location of the object segment in a subsequent frame based upon the locations of the object segments in the current frame and the plurality of prior frames;
• and,
• the vertical symmetry detector is adapted to compute the vertical symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a (given column i as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each column i on opposing sides of the horizontal axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the horizontal axis;
• and,
• the horizontal symmetry detector is adapted to compute the horizontal symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given row j as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each row j on opposing sides of the vertical axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the vertical axis;
• the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in order to override any conflict in the labeling of pixels in the subsequent frame as part of a segment containing a possible object of interest;
• the feature set computation apparatus further comprising:
• a horizontal fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of horizontal fuzzy sets each having a selected weighting function applicable to a selected group of columns within the object segment, with the columns grouped according to position along the horizontal axis in the object segment;
• and,
• a vertical fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of vertical fuzzy sets each having a selected weighting function applicable to a selected group of rows within the object segment, with the rows grouped according to position along the vertical axis in the object segment;
• and,
• the background modeling apparatus is adapted to isolate from a current frame the at least one segment of the current frame utilizing the video object plane extracted from the current frame;
• and
• the horizontal fuzzy sets further comprising:
• a left extreme fuzzy set having a weighting function that is a constant value for a first selected number of columns from the left border of the object segment toward the vertical axis of the object segment and decreases to zero over a second selected number of columns toward the vertical axis if the object segment;
• a right extreme fuzzy set having a weighting function that is a constant value for a third selected number of columns from the right border of the object segment toward the vertical axis of the object segment and decreases to zero over a fourth selected number of columns toward the vertical axis of the object segment;
• and
• at least one horizontal internal fuzzy set having a weighting function that increases from zero to a peak value and decreases to zero as the columns contained in each such internal fuzzy set go from left to right across the object image;
• the vertical fuzzy sets further comprising:
• a bottom extreme fuzzy set having a weighting function that is a constant value for a fifth selected number of rows from the bottom border of the object segment toward the horizontal axis of the object segment and decreases to zero over a sixth selected number of rows toward the horizontal axis if the object segment;
• a top extreme fuzzy set having a weighting function that is a constant value for a seventh selected number of columns from the top border of the object segment toward the horizontal axis of the object segment and decreases to zero over an eighth selected number of rows toward the horizontal axis of the object segment;
• and,
• at least one vertical internal fuzzy set having a weighting function that increases from zero to a peak value and decreases to zero as the rows contained in each such vertical internal fuzzy set go from bottom to top across the object image.

A video detection apparatus adapted to detect and follow movement of a predefined object of interest within the video data of the output of a video camera encoded in frame-wise data, each frame occurring at sequential time intervals, and including the segmentation of at least one portion of the frame-wise data, an object segment, as potentially including the preselected object of interest, and including vertical edge pixels and horizontal edge pixels detected to be present within the object segment, comprising:

• an edge symmetry detection unit comprising:
• a vertical symmetry detector adapted to compute the vertical symmetry of the set of vertical edge pixels and the set of horizontal edge pixels in each column about a horizontal axis passing through the object segment, and to produce a vertical symmetry value for columns within the object segment;
• a horizontal symmetry detector adapted to compute the horizontal symmetry of the set of vertical edge pixels and set of horizontal edge pixels in each row about a vertical axis passing through the object segment, and to produce a horizontal symmetry value for rows within the object segment;
• and,
• an edge symmetry feature set computation apparatus adapted to compute an edge symmetry feature set for the object segment based upon the vertical symmetry values and the horizontal symmetry values;
• and,
• a background modeling unit comprising:
• a background modeling apparatus adapted to isolate from a current frame at least one segment of the current frame, each of the at least one segments containing a possible object of interest within the current frame, on a frame-wise basis, and to provide as an output each of the at least one segments;
• an object of interest classifying apparatus adapted to determine whether or not any segment of the at least one segments output from the background modeling apparatus includes a object of interest and to characterize any such segment as an object segment, utilizing at least in part the edge symmetry feature set;
• an object segment tracking apparatus adapted to track the location within the current frame of any object segment, and to determine a projected location of the object segment in a subsequent frame, and to provide the background modeling apparatus with the projected location in the subsequent frame of the object segment;
• wherein the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in lieu of a segment from the subsequent frame determined by the background modeling apparatus to contain a possible object of interest or in the event that the background modeling apparatus is unable in the subsequent frame to locate any segment containing a possible object of interest;
• and,
• the object segment tracking apparatus further comprising:
• a matching apparatus adapted to match the location of an object segment in the current frame to one of a plurality of projected locations of the object segment in the current frame, which projections are based upon the location of the respective object segment in at least one prior frame;
• a track provider adapted to receive and store the location of an object segment in the current frame and over a plurality of prior frames and adapted to provide the projections of the location of the object segment in a subsequent frame based upon the locations of the object segments in the current frame and the plurality of prior frames;
• and,
• the vertical symmetry detector is adapted to compute the vertical symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given column i as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each column i on opposing sides of the horizontal axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the horizontal axis;
• and,
• the horizontal symmetry detector is adapted to compute the horizontal symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given row j as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each row j on opposing sides of the vertical axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the vertical axis;
• the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in order to override any conflict in the labeling of pixels in the subsequent frame as part of a segment containing a possible object of interest;
• the feature set computation apparatus further comprising:
• a horizontal fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of horizontal fuzzy sets each having a selected weighting function applicable to a selected group of columns within the object segment, with the columns grouped according to position along the horizontal axis in the object segment;
• and,
• a vertical fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of vertical fuzzy sets each having a selected weighting function applicable to a selected group of rows within the object segment, with the rows grouped according to position along the vertical axis in the object segment;
• and,
• the background modeling apparatus is adapted to isolate from a current frame the at least one segment of the current frame utilizing the video object plane extracted from the current frame;
• and
• the horizontal fuzzy sets further comprising:
• a left extreme fuzzy set having a weighting function that is a constant value for a first selected number of columns from the left border of the object segment toward the vertical axis of the object segment and decreases to zero over a second selected number of columns toward the vertical axis if the object segment;
• a right extreme fuzzy set having a weighting function that is a constant value for a third selected number of columns from the right border of the object segment toward the vertical axis of the object segment and decreases to zero over a fourth selected number of columns toward the vertical axis of the object segment;
• and
• at least one horizontal internal fuzzy set having a weighting function that increases from zero to a peak value and decreases to zero as the columns contained in each such internal fuzzy set go from left to right across the object image;
• the vertical fuzzy sets further comprising:
• a bottom extreme fuzzy set having a weighting function that is a constant value for a fifth selected number of rows from the bottom border of the object segment toward the horizontal axis of the object segment and decreases to zero over a sixth selected number of rows toward the horizontal axis if the object segment;
• a top extreme fuzzy set having a weighting function that is a constant value for a seventh selected number of columns from the top border of the object segment toward the horizontal axis of the object segment and decreases to zero over an eighth selected number of rows toward the horizontal axis of the object segment;
• and,
• at least one vertical internal fuzzy set having a weighting function that increases from zero to a peak value and decreases to zero as the rows contained in each such vertical internal fuzzy set go from bottom to top across the object image;
• and,
• the at least one segment of the current frame is a bounding box.

A video detection apparatus adapted to detect and follow movement of a predefined object of interest within the video data of the output of a video camera encoded in frame-wise data, each frame occurring at sequential time intervals, and including the segmentation of at least one portion of the frame-wise data, an object segment, as potentially including the preselected object of interest, and including vertical edge pixels and horizontal edge pixels detected to be present within the object segment, comprising:

• an edge symmetry detection unit comprising:
• a vertical symmetry detector adapted to compute the vertical symmetry of the set of vertical edge pixels and the set of horizontal edge pixels in each column about a horizontal axis passing through the object segment, and to produce a vertical symmetry value for columns within the object segment;
• a horizontal symmetry detector adapted to compute the horizontal symmetry of the set of vertical edge pixels and set of horizontal edge pixels in each row about a vertical axis passing through the object segment, and to produce a horizontal symmetry value for rows within the object segment;
• and,
• an edge symmetry feature set computation apparatus adapted to compute an edge symmetry feature set for the object segment based upon the vertical symmetry values and the horizontal symmetry values;
• and,
• a background modeling unit comprising:
• a background modeling apparatus adapted to isolate from a current frame at least one segment of the current frame, each of the at least one segments containing a possible object of interest within the current frame, on a frame-wise basis, and to provide as an output each of the at least one segments;
• an object of interest classifying apparatus adapted to determine whether or not any segment of the at least one segments output from the background modeling apparatus includes a object of interest and to characterize any such segment as an object segment, utilizing at least in part the edge symmetry feature set;
• an object segment tracking apparatus adapted to track the location within the current frame of any object segment, and to determine a projected location of the object segment in a subsequent frame, and to provide the background modeling apparatus with the projected location in the subsequent frame of the object segment;
• wherein the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in lieu of a segment from the subsequent frame determined by the background modeling apparatus to contain a possible object of interest or in the event that the background modeling apparatus is unable in the subsequent frame to locate any segment containing a possible object of interest;
• and,
• the object segment tracking apparatus further comprising:
• a matching apparatus adapted to match the location of an object segment in the current frame to one of a plurality of projected locations of the object segment in the current frame, which projections are based upon the location of the respective object segment in at least one prior frame;
• a track provider adapted to receive and store the location of an object segment in the current frame and over a plurality of prior frames and adapted to provide the projections of the location of the object segment in a subsequent frame based upon the locations of the object segments in the current frame and the plurality of prior frames;
• and,
• the vertical symmetry detector is adapted to compute the vertical symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given column i as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each column i on opposing sides of the horizontal axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the horizontal axis;
• and,
• the horizontal symmetry detector is adapted to compute the horizontal symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given row j as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each row j on opposing sides of the vertical axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the vertical axis;
• the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in order to override any conflict in the labeling of pixels in the subsequent frame as part of a segment containing a possible object of interest;
• the feature set computation apparatus further comprising:
• a horizontal fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of horizontal fuzzy sets each having a selected weighting function applicable to a selected group of columns within the object segment, with the columns grouped according to position along the horizontal axis in the object segment;
• and,
• a vertical fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of vertical fuzzy sets each having a selected weighting function applicable to a selected group of rows within the object segment, with the rows grouped according to position along the vertical axis in the object segment;
• and,
• the background modeling apparatus is adapted to isolate from a current frame the at least one segment of the current frame utilizing the video object plane extracted from the current frame;
• and
• the horizontal fuzzy sets further comprising:
• a left extreme fuzzy set having a weighting function that is a constant value for a first selected number of columns from the left border of the object segment toward the vertical axis of the object segment and decreases to zero over a second selected number of columns toward the vertical axis if the object segment;
• a right extreme fuzzy set having a weighting function that is a constant value for a third selected number of columns from the right border of the object segment toward the vertical axis of the object segment and decreases to zero over a fourth selected number of columns toward the vertical axis of the object segment;
• and
• at least one horizontal internal fuzzy set having a weighting function that increases from zero to a peak value and decreases to zero as the columns contained in each such internal fuzzy set go from left to right across the object image;
• the vertical fuzzy sets further comprising:
• a bottom extreme fuzzy set having a weighting function that is a constant value for a fifth selected number of rows from the bottom border of the object segment toward the horizontal axis of the object segment and decreases to zero over a sixth selected number of rows toward the horizontal axis if the object segment;
• a top extreme fuzzy set having a weighting function that is a constant value for a seventh selected number of columns from the top border of the object segment toward the horizontal axis of the object segment and decreases to zero over an eighth selected number of rows toward the horizontal axis of the object segment;
• and,
• at least one vertical internal fuzzy set having a weighting function that increases from zero to a peak value and decreases to zero as the rows contained in each such vertical internal fuzzy set go from bottom to top across the object image;
• the at least one segment of the current frame is a bounding box;
• and,
• the columns associated with each of the horizontal fuzzy sets are overlapping into respective adjacent horizontal fuzzy sets;
• and,
• the rows associated with each of the vertical fuzzy sets are overlapping into respective adjacent vertical fuzzy sets.

A video detection apparatus adapted to detect and follow movement of a predefined object of interest within the video data of the output of a video camera encoded in frame-wise data, each frame occurring at sequential time intervals, and including the segmentation of at least one portion of the frame-wise data, an object segment, as potentially including the preselected object of interest, and including vertical edge pixels and horizontal edge pixels detected to be present within the object segment, comprising:

• an edge symmetry detection unit comprising:
• a vertical symmetry detector adapted to compute the vertical symmetry of the set of vertical edge pixels and the set of horizontal edge pixels in each column about a horizontal axis passing through the object segment, and to produce a vertical symmetry value for columns within the object segment;
• a horizontal symmetry detector adapted to compute the horizontal symmetry of the set of vertical edge pixels and set of horizontal edge pixels in each row about a vertical axis passing through the object segment, and to produce a horizontal symmetry value for rows within the object segment;
• and,
• an edge symmetry feature set computation apparatus adapted to compute an edge symmetry feature set for the object segment based upon the vertical symmetry values and the horizontal symmetry values;
• and,
• a background modeling unit comprising:
• a background modeling apparatus adapted to isolate from a current frame at least one segment of the current frame, each of the at least one segments containing a possible object of interest within the current frame, on a frame-wise basis, and to provide as an output each of the at least one segments;
• an object of interest classifying apparatus adapted to determine whether or not any segment of the at least one segments output from the background modeling apparatus includes a object of interest and to characterize any such segment as an object segment, utilizing at least in part the edge symmetry feature set;
• an object segment tracking apparatus adapted to track the location within the current frame of any object segment, and to determine a projected location of the object segment in a subsequent frame, and to provide the background modeling apparatus with the projected location in the subsequent frame of the object segment;
• wherein the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in lieu of a segment from the subsequent frame determined by the background modeling apparatus to contain a possible object of interest or in the event that the background modeling apparatus is unable in the subsequent frame to locate any segment containing a possible object of interest;
• and,
• the object segment tracking apparatus further comprising:
• a matching apparatus adapted to match the location of an object segment in the current frame to one of a plurality of projected locations of the object segment in the current frame, which projections are based upon the location of the respective object segment in at least one prior frame;
• a track provider adapted to receive and store the location of an object segment in the current frame and over a plurality of prior frames and adapted to provide the projections of the location of the object segment in a subsequent frame based upon the locations of the object segments in the current frame and the plurality of prior frames;
• and,
• the vertical symmetry detector is adapted to compute the vertical symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given column i as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each column i on opposing sides of the horizontal axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the horizontal axis;
• and,
• the horizontal symmetry detector is adapted to compute the horizontal symmetry value of at least one of the set of horizontal edges or the set of vertical edges in a given row j as the ratio of the number of horizontal edge pixels or vertical edge pixels, respectively, in each row j on opposing sides of the vertical axis, independent of which side has more horizontal edge pixels or vertical edge pixels, respectively, expressed as a number between 0, if there are no horizontal edge pixels or no vertical edge pixels, respectively, on either side of the horizontal axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or vertical edge pixels, respectively on both sides of the vertical axis;
• the background modeling apparatus is adapted to provide as its output the projected location of the object segment in the subsequent frame in order to override any conflict in the labeling of pixels in the subsequent frame as part of a segment containing a possible object of interest;
• the feature set computation apparatus further comprising:
• a horizontal fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of horizontal fuzzy sets each having a selected weighting function applicable to a selected group of columns within the object segment, with the columns grouped according to position along the horizontal axis in the object segment;
• and,
• a vertical fuzzy set computation apparatus adapted to compute a fuzzy set value for each of a plurality of vertical fuzzy sets each having a selected weighting function applicable to a selected group of rows within the object segment, with the rows grouped according to position along the vertical axis in the object segment;
• and,
• the background modeling apparatus is adapted to isolate from a current frame the at least one segment of the current frame utilizing the video object plane extracted from the current frame;
• and
• the horizontal fuzzy sets further comprising:
• a left extreme fuzzy set having a weighting function that is a constant value for a first selected number of columns from the left border of the object segment toward the vertical axis of the object segment and decreases to zero over a second selected number of columns toward the vertical axis if the object segment;
• a right extreme fuzzy set having a weighting function that is a constant value for a third selected number of columns from the right border of the object segment toward the vertical axis of the object segment and decreases to zero over a fourth selected number of columns toward the vertical axis of the object segment;
• and
• at least one horizontal internal fuzzy set having a weighting function that increases from zero to a peak value and decreases to zero as the columns contained in each such internal fuzzy set go from left to right across the object image;
• the vertical fuzzy sets further comprising:
• a bottom extreme fuzzy set having a weighting function that is a constant value for a fifth selected number of rows from the bottom border of the object segment toward the horizontal axis of the object segment and decreases to zero over a sixth selected number of rows toward the horizontal axis if the object segment;
• a top extreme fuzzy set having a weighting function that is a constant value for a seventh selected number of columns from the top border of the object segment toward the horizontal axis of the object segment and decreases to zero over an eighth selected number of rows toward the horizontal axis of the object segment;
• and,
• at least one vertical internal fuzzy set having a weighting function that increases from zero to a peak value and decreases to zero as the rows contained in each such vertical internal fuzzy set go from bottom to top across the object image;
• the at least one segment of the current frame is a bounding box;
• the columns associated with each of the horizontal fuzzy sets are overlapping into respective adjacent horizontal fuzzy sets;
• and,
• the rows associated with each of the vertical fuzzy sets are overlapping into respective adjacent vertical fuzzy sets;
• and,
• the feature set computation apparatus further comprising a fuzzy set edge value score computing apparatus adapted to compute a fuzzy set edge value based upon the respective horizontal edge value or vertical edge value for the respective column or row and the respective horizontal fuzzy set weighting function value for the respective column or the respective vertical fuzzy set weighting function for the respective row.