Information processing device, operation detection method, and storage medium that determine the position of an operation object in a three-dimensional space based on a histogram

An operation detection method, which is performed by an information processing device including a distance acquisition unit configured to acquire depth information on a distance from an image pickup unit to a surface to be imaged, the surface to be imaged containing an operation object for an operation on a surface to be operated, the operation detection method comprising the steps of:

• extracting a tip area of the operation object using the depth information;
• determining a tip position of the operation object in a height direction from the surface to be operated, based on a histogram, which indicates a relationship between a height of a point cloud in a three-dimensional space from the surface to be operated and a number of point clouds, generated from a group of pieces of depth information on the tip area;
• continuously acquiring, using the distance acquisition unit, the depth information with the histogram changing with lapse of time;
• determining whether a shape of the histogram has changed based on a ratio of a predetermined width of a window corresponding to the operation object and a width corresponding to a distribution of the point cloud;
• and
• increasing the predetermined width of the window based on continuity of the histogram when the shape has changed,
• wherein the tip position determining step determines the tip position using a lowest height value in the window.

A non-transitory computer readable storage medium storing a computer program executable by a computer of an information processing apparatus including a distance acquisition unit configured to acquire depth information on a distance from an image pickup unit to a surface to be imaged, the surface to be imaged containing an operation object for an operation on a surface to be operated, to execute a method comprising the steps of:

• extracting a tip area of the operation object using the depth information;
• determining a tip position of the operation object in a height direction from the surface to be operated, based on a histogram, which indicates a relationship between a height of a point cloud in a three-dimensional space from the surface to be operated and a number of point clouds, generated from a group of pieces of depth information on the tip area;
• continuously acquiring, using the distance acquisition unit, the depth information with the histogram changing with lapse of time;
• determining whether a shape of the histogram has changed based on a ratio of a predetermined width of a window corresponding to the operation object and a width corresponding to a distribution of the point cloud;
• and
• increasing the predetermined width of the window based on continuity of the histogram when the shape has changed,
• wherein the tip position determining step determines the tip position using a lowest height value in the window.