US 9,311,560 B2 · Extraction of user behavior from depth images

Abstract

Embodiments described herein use depth images to extract user behavior, wherein each depth image specifies that a plurality of pixels correspond to a user. In certain embodiments, information indicative of an angle and/or curvature of a user's body is extracted from a depth image. This can be accomplished by fitting a curve to a portion of a plurality of pixels (of the depth image) that correspond to the user, and determining the information indicative of the angle and/or curvature of the user's body based on the fitted curve. An application is then updated based on the information indicative of the angle and/or curvature of the user's body. In certain embodiments, one or more average extremity positions of a user, which can also be referred to as average positions of extremity blobs, are extracted from a depth image. An application is then updated based on the average positions of extremity blobs.

Claims20

Dependent claims

A system that uses depth images to extract user behavior, comprising:

a communication interface that receives depth images;
one or more storage devices that store depth images;
a display interface;
and
one or more processors in communication with the one or more storage devices and the display interface, wherein for each of a plurality of depth images the one or more processors are configured to
determine which pixels of the depth image correspond to a user;
identify a subset of pixels of interest from the pixels that correspond to the user;
fit a curve to the subset of pixels of interest to thereby produce a fitted curve comprising a plurality of straight line segments;
identify a first straight line that extends between endpoints of the fitted curve;
identify a second straight line that extends orthogonally from the first straight line to a point of the fitted curve that is farthest away from the first straight line;
and
determine, based on the first and second straight lines, information indicative of a curvature of the user's body;
wherein the one or more processors are also configured to use the information indicative of the curvature of the user's body to update an application running on the system.

10 more elements

One or more processor readable storage devices having instructions encoded thereon which when executed cause one or more processors to perform a method for using depth images to extract user behavior, the method comprising:

receiving a depth image that that include a plurality of pixels that correspond to a user;
identifying a subset of pixels of interest from the pixels that correspond to the user;
fitting a curve to the subset of pixels of interest to thereby produce a fitted curve comprising a plurality of straight line segments;
identifying and determining a length of a first straight line that extends between endpoints of the fitted curve;
identifying and determining a length of a second straight line that extends orthogonally from the first straight line to a point of the fitted curve that is farthest away from the first straight line;
and
determine a ratio of the length of the first straight line to the length of the second straight line, wherein the ratio is indicative of a curvature of the user's body;
and
using the information indicative of the curvature of the user's body to update an application.

6 more elements

Dependencies are inferred from the claim text.

Extraction of user behavior from depth images

Life of this patent

Extraction of user behavior from depth images

Life of this patent

Abstract

Claims20