Back to results
US 10,242,255 B2Granted
Gesture recognition system using depth perceptive sensors
AssigneeMicrosoft Technology Licensing, LLC
InventorsAbbas Rafii, Salih Burak Gokturk, Carlo Tomasi, Fahri Sürücü
Life of this patent
Estimating the term…
Abstract
Acquired three-dimensional positional information is used to identify user created gesture(s), which gesture(s) are classified to determine appropriate input(s) to an associated electronic device or devices. Preferably at at least one instance of a time interval, the posture of a portion of a user is recognized, based at least one factor such as shape, position, orientation, velocity. Posture over each of the instance(s) is recognized as a combined gesture. Because acquired information is three-dimensional, two gestures may occur simultaneously.
Claims20
A method enabling user-interaction with an electronic device comprising a sensor using at least one gesture made by a body part of a user of the electronic device, the method comprising:
- acquiring depth values for a plurality of points in a scene;
- grouping at least some of the points by using the depth values to form a cluster that represents the body part, comprising points that have similar depth values, wherein said grouping comprises testing adjacent points in the scene for homogeneity using a gradient of the depth values for the cluster, and wherein using the gradient of the depth values comprises determining that a region is homogeneous when a greatest gradient magnitude in an interior of the region is below a predefined threshold;
- acquiring positional information for each of a plurality of discrete regions on the body part relative to the sensor over a particular time interval, the positional information includes three-dimensional depth information associated with each of the plurality of discrete regions on the body part;
- detecting the at least one gesture using the three-dimensional depth information, the at least one gesture includes a dynamic gesture corresponding with more than one consecutive posture of the body part within the particular time interval;
- wherein detecting the at least one gesture comprises:
- generating a matrix comprising coordinates of the plurality of discrete regions on the body part,
- applying singular value decomposition to the matrix, wherein the singular value decomposition generates singular vectors that define a shape of an ellipse that corresponds to the body part, and
- wherein a posture is defined at least in part by directions and lengths of major and minor axes of the ellipse;
- and
- determining the at least one gesture as at least one input to the electronic device.
7 more elements
An electronic device, comprising:
- a sensor, the sensor captures depth information associated with a body part of a user of the electronic device relative to the sensor, wherein the depth information comprises respective three-dimensional depth information for each of a plurality of discrete regions on the body part;
- a body posture module that identifies one or more poses associated with the user based on the depth information for each of the plurality of discrete regions on the body part, wherein the body posture module:
- determines coordinates of the plurality of discrete regions on the body part, and
- utilizes the coordinates to generate vectors that define a shape of an ellipse that corresponds to the body part,
- wherein a pose is defined by the directions and lengths of major and minor axes of the ellipse;
- a gesture representation module that detects a dynamic gesture corresponding with more than one consecutive pose of the one or more poses within a particular time interval, wherein the gesture representation module:
- constructs a posture representation vector for the body part for each of a plurality of instances that occur within the particular time interval, wherein each posture representation vector represents the location and the pose of the body part at a respective instance,
- determines a trajectory of posture representation vectors for a given duration that includes a duration of the gesture,
- divides the trajectory of the posture representation vectors between a beginning of the gesture and an end of the gesture into equal-length intervals, wherein each interval includes a plurality of the posture representation vectors,
- determines an interval representation vector for each interval, wherein each interval representation vector represents the plurality of the posture representation vectors in the interval;
- constructs a gesture representation vector from the interval representation vectors;
- and
- a classification module that, based on the gesture representation vector, classifies the dynamic gesture as an input to the electronic device.
10 more elements
A system comprising:
- a sensor, the sensor captures a plurality of depth images, each depth image of the plurality of depth images includes three-dimensional depth information associated with a body part of a user of the electronic device relative to the sensor, the plurality of depth images includes a first depth image captured at a first point in time and a second depth image captured at a second point in time subsequent to the first point in time;
- and
- one or more processors in communication with the sensor, wherein the one or more processors:
- (A) detect a first pose of the body part using the first depth image, wherein the first pose comprises a first three-dimensional position of the body part and a first three-dimensional orientation of the body part,
- (B) detect a second pose of the body part using the second depth image, wherein the second pose comprises a second three-dimensional position of the body part and a second three-dimensional orientation of the body part,
- (C) determine a velocity associated with a movement of the body part from the first pose to the second pose,
- (D) detect a dynamic gesture corresponding with the first pose and the second pose, wherein the one or more processors detect the dynamic gesture using the velocity to recognize that the body part is making a gesture to the exclusion of other body parts, and
- (E) classify the dynamic gesture as an input to the electronic device;
- wherein detect the first pose and detect the second pose comprises:
- determine coordinates for a plurality of discrete regions on the body part, and
- generate a matrix comprising the coordinates of the plurality of discrete regions on the body part;
- apply singular value decomposition to the matrix, wherein the singular value decomposition generates singular vectors that define a shape of an ellipse that corresponds to the body part,
- wherein a pose is defined by directions and lengths of major and minor axes of the ellipse.
10 more elements
Dependencies are inferred from the claim text.