Segmentation of image features using hierarchical analysis of multi-valued image data and weighted averaging of segmentation results

A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having motion relative to the second image feature between the selected video image frame and a preceding video image frame, comprising:

• obtaining a motion vector representation of corresponding pixels in the selected video image frame and the preceding video image frame;
• and
• segmenting the first and second video image features according to the motion vector representation, including
• determining a residue portion of the motion vector representation
• simplifying the residue portion to create a simplified motion vector representation, including filtering motion vectors of the residue portion to reduce variation in the motion vectors;
• distinguishing a region or regions of homogenous motion in the simplified motion vector representation;
• and
• repeating the determining, simplifying and distinguishing steps, so as to form first and second motion segmented video image features.

A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having a multidimensional characteristic that varies relative to the second image feature, comprising:

• obtaining a representation of the multidimensional characteristic for pixel locations in the selected video image frame;
• determining a residue portion of the multidimensional characteristic representation;
• simplifying the residue portion to create a simplified multidimensional characteristic representation including filtering multidimensional characteristics of the residue portion to reduce variation in the multidimensional characteristics;
• distinguishing a homogenous region or regions from a non-homogenous region in the simplified multidimensional characteristic representation;
• and
• repeating the determining, simplifying and distinguishing steps to form a first segmented video image feature.

A computer-readable medium storing computer-executable programming for segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having plural pixels of a first spatial image characteristic and motion relative to the second image feature between the selected video image frame and a preceding video image frame, and the second image feature having plural pixels of a second spatial image characteristic different from the first image characteristic, the medium comprising:

• programming for obtaining a motion representation of corresponding pixels in the selected video image frame and the preceding video image frame;
• programming for segmenting the first and second video image features according to the motion representation to form first and second motion segmented video image features;
• programming for segmenting the first and second video image features according to the respective first and second spatial image characteristics to form first and second spatial segmented video image features;
• and
• programming for forming jointly segmented first and second video image features as a weighted sum of the first and second motion segmented video image features and the first and second spatial segmented video image features.

A method for segmenting an arbitrary first image feature in a selected video image frame from a second image feature in a second video image frame, the arbitrary first image feature having a multidimensional characteristic that varies relative to the second image feature between the first and second video image frames, comprising:

• obtaining a representation of the multidimensional characteristic for pixel locations in the first video image frame
• determining a residue portion of the multidimensional characteristic representation;
• simplifying the residue portion to create a simplified multidimensional characteristic representation including filtering multidimensional characteristics of the residue portion to reduce variation in the multidimensional characteristics;
• distinguishing a homogenous region or regions from a non-homogenous region in the simplified multidimensional characteristic representation;
• and repeating the determining, simplifying and distinguishing steps to form first and second segmented video image features.

A computer-readable medium storing computer-executable programming for segmenting arbitrary image features in selected video image frames, the arbitrary image features having differing multidimensional characteristics that vary between the video image frames, the medium comprising:

• a) programming code for obtaining representations of the multidimensional characteristic for pixel locations in the video image frames;
• b) programming code for determining a residue portion of the multidimensional characteristic representation for a video image frame;
• c) programming code for simplifying the residue portion to create a simplified multidimensional characteristic representation including filtering multidimensional characteristics of the residue portion to reduce variation in the multidimensional characteristics;
• d) programming code for distinguishing a homogenous region or regions from a non-homogenous region in the simplified multidimensional characteristic representation;
• and
• e) programming code for iteratively executing the programming code listed in b), c), and d) above to form at least a first segmented video image feature.

A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having plural pixels of a first spatial image characteristic and motion relative to the second image feature between the selected video image frame and a preceding video image frame, and the second image feature having plural pixels and a second spatial image characteristic different from the first image characteristic, comprising:

• obtaining a motion representation of corresponding pixels in the selected video image frame and the preceding video image frame;
• segmenting the first and second video image features according to the motion representation to form first and second motion segmented video image features;
• segmenting the first and second video image features according to the respective first and second spatial image characteristics to form first and second spatially segmented video image features;
• and
• forming jointly segmented first and second video image features according to the first and second motion segmented video image features and the first and second spatial segmented video image features;
• wherein segmenting the first and second video image features according to the motion representation includes iterative application of a hierarchical motion segmentation engine;
• and
• wherein the hierarchical motion segmentation engine applies nonlinear motion simplification to the motion representation on each iteration according to a nonlinear filter that functions to maintain distinctions between the first and second video image features.

A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having motion relative to the second image feature between the selected video image frame and a preceding video image frame, comprising:

• obtaining a motion vector representation of corresponding pixels in the selected video image frame and the preceding video image frame;
• and
• segmenting the first and second video image features according to the motion vector representation to form first and second motion segmented video image features, including iteratively applying a hierarchical motion segmentation engine;
• wherein the hierarchical motion segmentation engine applies nonlinear motion simplification to the motion vectors on each iteration according to a nonlinear filter that functions to maintain distinctions between the first and second video image features.

A method of segmenting an arbitrary first image feature in a selected video image frame from an adjacent second image feature in the selected video image frame, the arbitrary first image feature having a multidimensional characteristic that varies relative to the second image feature, comprising;

• obtaining a representation of the multidimensional characteristic for pixel locations in the selected video image frame;
• modeling the multidimensional characteristics at pixel locations in the selected frame to compute a modeled image and
• determining a residue portion of the multidimensional characteristic representation, where the residue portion comprises a portion of the selected frame identified by computing a difference between the selected image frame and the modeled image;
• simplifying the residue portion to create a simplified multidimensional characteristic representation;
• distinguishing a homogenous region or regions from a non-homogenous region in the simplified multidimensional characteristic representation;
• and repeating the determining, simplifying and distinguishing steps to form a first segmented video image feature.

A computer-readable medium storing computer-executable instructions for segmenting arbitrary image features in a first video image frame, an arbitrary first image feature having motion relative to an adjacent arbitrary second image feature between the first video image frame and a preceding second video image frame, the computer-executable instructions for:

• obtaining a segmentation of the second video image frame, wherein the segmentation incorporates motion segmented video image features based upon comparison of the second video image frame with a preceding third video image frame;
• obtaining a motion vector representation of corresponding pixels in the first video image frame and the second video image frame;
• identifying one or more seed regions in the segmentation of the second video image frame;
• and
• segmenting the first and second video image features in the first video image frame by classifying each of plural motion vectors of the motion vector representation based upon similarity to the identified one or more seed regions.

An apparatus for segmenting image features of a selected video image frame, a first image feature having motion relative to an adjacent second image feature between the selected video image frame and a preceding video image frame, the apparatus comprising:

• means for obtaining a motion vector representation of corresponding pixels in a selected video image frame and a preceding video image frame;
• means for obtaining a video feature segmentation for the preceding video image frame;
• and
• means for segmenting a first video image feature in the selected video image frame from an adjacent second video image feature in the selected video image frame, said means for segmenting including a means for classifying each of plural motion vectors of the motion vector representation based upon the video image feature segmentation for the preceding video image frame.