Image processing apparatus, method, and program capable of recognizing hand gestures

An image processing method being executed via at least one processor, and comprising:

• detecting a facial image from an input image;
• estimating a posture of a person in the input image from a position of the facial image detected in the detecting of the facial image;
• detecting positions of hands of the person based on the posture estimated in the estimating of the posture of the person;
• extracting a hand image of the person from the input image based on information regarding the positions of the hands of the person detected in the detecting of the hand positions of the hands;
• specifying shapes of the hands of the person based on the hand image extracted in the extracting of the hand image;
• storing, into a hand shape time-series storage unit, the shapes of the hands specified in the specifying of the shapes of the hands in the time-series by corresponding to the information regarding the positions of the hands;
• recognizing a hand gesture, which is a gesture using the hands of the person, based on the information regarding the shapes of the hands stored in the time-series in the storing of the shapes of the hands by corresponding to the information regarding the positions of the hands;
• storing, into a hand gesture storage unit, information regarding the shapes of the hands in a time-series by corresponding to information regarding the positions of the hands for each type of hand gesture;
• recognizing a hand shape and a motion as an integrated module;
• calculating a similarity between the information regarding the shapes of the hands stored in the time-series by corresponding to the information regarding the positions of the hands and the information regarding the shapes of the hands stored in the time-series by corresponding to the information regarding the positions of the hands for each hand gesture, and
• calculating, as the similarity, a ratio of a number of modules that are continuously identical among a plurality of modules continuously set as the hand gesture, to a total number of modules set for the hand gesture,
• wherein the information regarding the shapes of the hands stored in the time-series in the hand shape time-series storage unit by corresponding to the information regarding the positions of the hands is compared to the information regarding the shapes of the hands stored in the time-series by corresponding to the information regarding the positions of the hands for each hand gesture, and the hand gesture is then recognized based on a result of the comparison, and sets the hand gesture with the highest similarity as a recognition result.

A non-transitory computer-readable medium having embodied thereon a program, which when executed by a processor of a computer, causes the computer to execute a method, the method comprising:

• detecting a facial image from an input image,
• estimating a posture of a person in the input image from a position of the facial image detected in the detecting of the facial image,
• detecting positions of hands of the person based on the posture estimated in the estimating of the posture of the person,
• extracting a hand image of the person from the input image based on information regarding the positions of the hands of the person detected in the detecting of the hand positions of the hands,
• specifying shapes of the hands of the person based on the hand image extracted in the extracting of the hand image,
• storing, into a hand shape time-series storage unit, the shapes of the hands specified in the specifying of the shapes of the hands in a time-series by corresponding to the information regarding the positions of the hands,
• recognizing a hand gesture, which is a gesture using the hands of the person, based on the information regarding the shapes of the hands stored in the time-series in the storing of the shapes of the hands by corresponding to the information regarding the positions of the hands,
• storing, into a hand gesture storage unit, information regarding the shapes of the hands in a time-series by corresponding to information regarding the positions of the hands for each type of hand gesture,
• recognizing a hand shape and a motion as an integrated module,
• calculating a similarity between the information regarding the shapes of the hands stored in the time-series by corresponding to the information regarding the positions of the hands and the information regarding the shapes of the hands stored in the time-series by corresponding to the information regarding the positions of the hands for each hand gesture, and
• calculating, as the similarity, a ratio of a number of modules that are continuously identical among a plurality of modules continuously set as the hand gesture, to a total number of modules set for the hand gesture,
• wherein the information regarding the shapes of the hands stored in the time-series in the hand shape time-series storage unit by corresponding to the information regarding the positions of the hands is compared to the information regarding the shapes of the hands stored in the time-series in the hand gesture storage unit by corresponding to the information regarding the positions of the hands for each hand gesture, and the hand gesture is then recognized based on a result of the comparison, and
• wherein the hand gesture recognition unit sets the hand gesture with the highest similarity as a recognition result.