Visual language for human computer interfaces

A non-transitory computer readable medium storing instructions for recognizing hand gestures, the instruction when executed by one or more processors cause the one or more processors to:

• present a region of interest (ROI) on a display;
• receive a digital color image of a user hand against a background, the digital color image captured using a digital image capturing device, the digital color image being represented by pixels in a first color space;
• select a set of pixels of the digital color image in the first color space from the pixels of the digital color image within the ROI, the selected set of pixels of the digital color image in the first color space describing a general parametric model associated with the digital color image in the first color space;
• obtain specific parametric templates in additional color spaces, each specific parametric template in a color space comprising a selected set of pixels within the ROI representing the user hand in a corresponding color space of the additional color spaces, the additional color spaces emphasizing chrominance over luminance information;
• combine the specific parametric templates in the additional color spaces to generate an updated specific parametric template;
• obtain a contour of the user hand by applying the updated specific parametric template to subsequent digital images of the user hand;
• and
• detect a hand gesture based on the contour of the user hand.

A computer implemented method for recognizing a visual gesture, the method comprising:

• receiving a first set of images including a part of a human body within a region of interest (ROI), the part of a human body oriented in a first configuration;
• registering a flesh tone of the part of the human body within the ROI of the first set of images in a first color space and a second color space;
• receiving a second set of images including the part of the human body, the part of the human body oriented in different configurations than the first configuration, the part of the human body oriented in the different configurations representing a visual gesture;
• identifying one or more objects in the second set of images corresponding to the part of the human body based on the registered flesh tone in the first color space and the second color space;
• obtaining motion vectors of the one or more objects in the second set of images by tracking the one or more objects in the second set of images;
• and
• determining the visual gesture represented by the part of the human body oriented in the different configurations according to the identified one or more objects and the motion vectors.

A computer-implemented method for recognizing hand gestures in video imagery, wherein the hand gesture has a static element, a motion element, or a mixture of the two, the method comprising:

• pre-registering a user hand, using one or multiple frames of the video;
• performing an adaptive hand-detection on subsequent video frames, performing an adaptive hand-detection comprising:
• applying at least one of skin color/tone analysis in color spaces and motion estimation, to segment regions or objects, especially a hand, within the video frames,
• wherein said skin color/tone analysis in color spaces comprises:
• using skin tone analysis in one or more color spaces, to obtain skin map(s),
• merging skin maps from the one or more color spaces, and
• using an adaptive threshold to segment a region or object by detecting skin pixels and grouping them together, and
• wherein said motion estimation comprises:
• obtaining motion vector fields between two successive frames, typically for blocks within video frames,
• tracking motion vectors on subsequent frames,
• applying a combination of cluster analysis of motion vectors, and
• tracking the evolution of motion vectors on spatial regions and their features;
• and
• recognizing a hand gesture in part by applying at least one of the following four tool groups:
• (i) skin color/tone analysis in one or more color spaces, (ii) motion estimation, (iii) morphological operations, and (iv) other image processing tools, on regions or objects, especially a segmented hand, to detect hands, hand contours and their features, as well as any motion of hand parts;
• wherein said skin color/tone analysis in one or more color spaces comprises:
• using skin tone analysis in one or more color spaces, to obtain skin map(s),
• merging skin maps from the one or more color spaces, and
• using an adaptive threshold to segment a region or object by detecting skin pixels and grouping them together,
• wherein said motion estimation comprises:
• obtaining motion vectors between two successive frames, typically for blocks within video frames,
• tracking motion vectors on subsequent frames, and
• applying any combination of cluster analysis of motion vectors, and tracking the evolution of motion vectors, defined spatial regions and their features, to detect regions or objects within video frames,
• wherein said morphological operations include dilations, erosions, opening, and closing operations, and
• wherein said other image processing tools include measuring distances, angles, extrema points, convexity, and shape on hand contours.

An apparatus for recognizing hand gestures, the apparatus comprising:

• a computer processor for performing steps comprising:
• presenting a region of interest (ROI) on a display;
• receiving a digital color image of a user hand against a background, the digital color image captured using a digital image capturing device, the digital color image being represented by pixels in a first color space;
• selecting a set of pixels of the digital color image in the first color space from the pixels of the digital color image within the ROI, the selected set of pixels of the digital color image in the first color space describing a general parametric model associated with the digital color image in the first color space;
• obtaining specific parametric templates in additional color spaces, each specific parametric template in a color space comprising a selected set of pixels within the ROI representing the user hand in a corresponding color space of the additional color spaces, the additional color spaces emphasizing chrominance over luminance information;
• combining the specific parametric templates in the additional color spaces to generate an updated specific parametric template;
• obtaining a contour of the user hand by applying the updated specific parametric template to subsequent digital images of the user hand;
• and
• detecting a hand gesture based on the contour of the user hand.

A non-transitory computer readable medium storing instructions for recognizing a visual gesture, the instruction when executed by one or more processors cause the one or more processors to:

• receive a first set of images including a part of a human body within a region of interest (ROI), the part of a human body oriented in a first configuration;
• register a flesh tone of the part of the human body within the ROI of the first set of images in a first color space and a second color space;
• receive a second set of images including the part of the human body, the part of the human body oriented in different configurations than the first configuration, the part of the human body oriented in the different configurations representing a visual gesture;
• identify one or more objects in the second set of images corresponding to the part of the human body based on the registered flesh tone in the first color space and the second color space;
• obtain motion vectors of the one or more objects in the second set of images by tracking the one or more objects in the second set of images;
• and
• determine the visual gesture represented by the part of the human body oriented in the different configurations according to the identified one or more objects and the motion vectors.

An apparatus for recognizing a visual gesture, the apparatus comprising:

• a computer processor for performing steps comprising:
• receive a first set of images including a part of a human body within a region of interest (ROI), the part of a human body oriented in a first configuration;
• register a flesh tone of the part of the human body within the ROI of the first set of images in a first color space and a second color space;
• receive a second set of images including the part of the human body, the part of the human body oriented in different configurations than the first configuration, the part of the human body oriented in the different configurations representing a visual gesture;
• identify one or more objects in the second set of images corresponding to the part of the human body based on the registered flesh tone in the first color space and the second color space;
• obtain motion vectors of the one or more objects in the second set of images by tracking the one or more objects in the second set of images;
• and
• determine the visual gesture represented by the part of the human body oriented in the different configurations according to the identified one or more objects and the motion vectors.

A non-transitory computer readable medium storing instructions for recognizing hand gestures in video imagery, wherein the hand gesture has a static element, a motion element, or a mixture of the two, the instruction when executed by one or more processors cause the one or more processors to:

• pre-register a user hand, using one or multiple frames of the video;
• perform an adaptive hand-detection on subsequent video frames, comprising:
• applying at least one of skin color/tone analysis in color spaces and motion estimation, to segment regions or objects, especially a hand, within the video frames,
• wherein said skin color/tone analysis in color spaces comprises:
• using skin tone analysis in one or more color spaces, to obtain skin map(s),
• merging skin maps from the one or more color spaces, and
• using an adaptive threshold to segment a region or object by detecting skin pixels and grouping them together, and
• wherein said motion estimation comprises:
• obtaining motion vector fields between two successive frames, typically for blocks within video frames,
• tracking motion vectors on subsequent frames,
• applying a combination of cluster analysis of motion vectors, and tracking the evolution of motion vectors on spatial regions and their features;
• and
• recognize a hand gesture in part by applying at least one of the following four tool groups:
• (i) skin color/tone analysis in one or more color spaces, (ii) motion estimation, (iii) morphological operations, and (iv) other image processing tools, on regions or objects, especially a segmented hand, to detect hands, hand contours and their features, as well as any motion of hand parts;
• wherein said skin color/tone analysis in one or more color spaces comprises:
• using skin tone analysis in one or more color spaces, to obtain skin map(s),
• merging skin maps from the one or more color spaces, and
• using an adaptive threshold to segment a region or object by detecting skin pixels and grouping them together,
• wherein said motion estimation comprises:
• obtaining motion vectors between two successive frames, typically for blocks within video frames,
• tracking motion vectors on subsequent frames, and
• applying any combination of cluster analysis of motion vectors, and tracking the evolution of motion vectors, defined spatial regions and their features, to detect regions or objects within video frames,
• wherein said morphological operations include dilations, erosions, opening, and closing operations, and
• wherein said other image processing tools include measuring distances, angles, extrema points, convexity, and shape on hand contours.

An apparatus for recognizing hand gestures in video imagery, wherein the hand gesture has a static element, a motion element, or a mixture of the two, the apparatus comprising:

• a computer processor for performing steps comprising:
• pre-registering a user hand, using one or multiple frames of the video;
• performing an adaptive hand-detection on subsequent video frames, performing an adaptive hand-detection comprising:
• applying at least one of skin color/tone analysis in color spaces and motion estimation, to segment regions or objects, especially a hand, within the video frames,
• wherein said skin color/tone analysis in color spaces comprises:
• using skin tone analysis in one or more color spaces, to obtain skin map(s),
• merging skin maps from the one or more color spaces, and
• using an adaptive threshold to segment a region or object by detecting skin pixels and grouping them together, and
• wherein said motion estimation comprises:
• obtaining motion vector fields between two successive frames, typically for blocks within video frames,
• tracking motion vectors on subsequent frames,
• applying a combination of cluster analysis of motion vectors, and
• tracking the evolution of motion vectors on spatial regions and their features;
• and
• recognizing a hand gesture in part by applying at least one of the following four tool groups:
• (i) skin color/tone analysis in one or more color spaces, (ii) motion estimation, (iii) morphological operations, and (iv) other image processing tools, on regions or objects, especially a segmented hand, to detect hands, hand contours and their features, as well as any motion of hand parts;
• wherein said skin color/tone analysis in one or more color spaces comprises:
• using skin tone analysis in one or more color spaces, to obtain skin map(s),
• merging skin maps from the one or more color spaces, and
• using an adaptive threshold to segment a region or object by detecting skin pixels and grouping them together,
• wherein said motion estimation comprises:
• obtaining motion vectors between two successive frames, typically for blocks within video frames,
• tracking motion vectors on subsequent frames, and
• applying any combination of cluster analysis of motion vectors, and tracking the evolution of motion vectors, defined spatial regions and their features, to detect regions or objects within video frames,
• wherein said morphological operations include dilations, erosions, opening, and closing operations, and
• wherein said other image processing tools include measuring distances, angles, extrema points, convexity, and shape on hand contours.