Human posture detection
AssigneeRealsense, Inc.
InventorsDavid Pearce, David Stanhill
Life of this patent
Estimating the term…
Abstract
A user computing device includes a camera sensor and a depth sensor. Image data generated by the camera captures an image of a user of the user computing device and is provided as an input to a first machine learning model trained to determine a feature set associated with posture of the user from the image data. Depth data generated by the depth sensor contemporaneously with generation of the image data is provided as input to a second machine learning model along with the first feature set to generate a second feature set as an output of the second machine learning model based on the depth data and the first feature set. The posture of the user is determined from the second feature set to provide feedback to the user.
Claims21
A non-transitory machine-readable storage medium with instructions stored thereon, the instructions executable by the machine to cause the machine to:
- receive image data generated by a camera with a first resolution, wherein the camera is provided on a user computing device to capture an image of a user of the user computing device;
- execute a first machine learning model trained to determine a feature set associated with posture of the user from the image data;
- receive depth data generated by a time of flight (ToF) sensor provided on the user computing device, wherein the depth data has a second resolution lower than the first resolution and is generated contemporaneously with generation of the image data;
- provide the first feature set as a first input and the depth data as a second input to a second machine learning model to generate a second feature set as an output of the second machine learning model;
- and
- determine a posture of the user from the second feature set.
A method comprising:
- receiving two-dimensional image data generated by a camera of a user computing device, wherein the image data comprises an image of a user using the user computing device;
- applying a first machine learning model to the image data to generate a first feature set, wherein the first feature set identifies features of a pose of the user from the image data;
- receiving depth data generated by a depth sensor of the user computing device, wherein the depth data comprises a grid of depth pixels and is generated contemporaneously with the image data;
- providing the first feature set as a first input and the depth data as a second input to a second machine learning model to generate a second feature set as an output of the second model;
- and
- determining a posture of the user from the second feature set.
An apparatus comprising:
- a processor;
- a memory;
- a display;
- a camera sensor oriented to face a human viewer of the display;
- a depth sensor oriented to face the human viewer of the display;
- a posture detection engine executable by the processor to:
- receive two-dimensional image data generated by the camera, wherein the image data comprises an image of the human viewer;
- provide the image data as an input to a first machine learning model to determine a first feature set, wherein the first machine learning model is trained to determine a post of a human from two-dimensional images;
- receive depth data generated by the depth sensor contemporaneously with generation of the image data, wherein the depth data comprises one or more depth measurements of the human viewer;
- provide the first feature set as a first input and the depth data as a second input to a second machine learning model to generate a second feature set as an output of the second machine learning model;
- determine a posture of the human viewer from the second feature set;
- and
- determine quality of the posture of the human viewer based on the second feature set.
Dependencies are inferred from the claim text.