US 11,961,601 B1Granted

Adaptive user interface for determining errors in performance of activities

AssigneeAmazon Technologies, Inc.

InventorsImry Kissos, Joel Wilson Brown, Ilia Vitsnudel, Omer Meir, Lior Fritz, Matan Goldman, Eduard Oks

Life of this patent

Estimating the term…

Loading patent…

Abstract

To assist a user in the correct performance of an activity, video data is acquired. A pose of the user is determined from the video data and an avatar is generated representing the user in the pose. The pose of the user is compared to one or more other poses representing correct performance of the activity to determine one or more differences that may represent errors by the user. Depending on the activity that is being performed, some errors may be presented to the user during performance of the activity, while other errors may be presented after performance of the activity has ceased. To present an indication of an error, a specific body part or other portion of the avatar that corresponds to a difference between the user's pose and a correct pose may be presented along with an instruction regarding correct performance of the activity.

Claims20

Dependent claims

A system comprising:

an output device;
one or more memories storing computer-executable instructions;
and
one or more hardware processors to execute the computer-executable instructions to:
acquire first video data representing a user performing an activity;
determine a first pose of the user based on the first video data;
generate an avatar representing the first pose of the user;
access activity data that indicates a second pose and a third pose, wherein the second pose and the third pose are indicative of correct performance of the activity;
determine a first difference between the first pose and the second pose;
determine, based on the activity data, that the first difference is associated with presentation of output during performance of the activity;
determine, based on the activity data, a first instruction associated with the first difference;
determine a first portion of the avatar associated with the first difference;
present, using the output device, first output that includes the first portion of the avatar at a first time during performance of the activity and excludes a second portion of the avatar, wherein the activity data associates the first difference with the first time;
present, using the output device, the first instruction at the first time;
acquire second video data representing the user performing the activity;
determine a fourth pose of the user based on the second video data;
determine a second difference between the third pose and the fourth pose;
determine, based on the activity data, that the second difference is associated with presentation of output after cessation of the activity;
determine, based on the activity data, a second instruction associated with the second difference;
determine a third portion of the avatar associated with the second difference;
determine cessation of the activity;
present, using the output device, second output that includes the third portion of the avatar at a second time subsequent to the cessation of the activity and excludes a fourth portion of the avatar, wherein the activity data associates the second difference with the second time;
and
present, using the output device, the second instruction at the second time.

21 more elements

A method comprising:

determining, based on first video data acquired at a first time, a first pose of a user performing an activity;
generating an avatar representing the first pose of the user;
determining, based on first data that indicates one or more second poses representing correct performance of the activity, a first difference between the first pose and the one or more second poses;
determining a first portion of the avatar associated with the first difference;
and
presenting, using an output device at a second time, first output that includes the first portion of the avatar and an indication of the first difference and that excludes a second portion of the avatar.

3 more elements

A system comprising:

one or more memories storing computer-executable instructions;
and
one or more hardware processors to execute the computer-executable instructions to:
determine, at a first time and based on first video data, a first pose of a user performing an activity;
generate a first avatar representing the first pose of the user;
determine one or more second poses representing correct performance of the activity;
determine a first difference between the first pose and the one or more second poses;
determine a first portion of the first avatar that is associated with the first difference;
and
at a second time after the first time, present, using one or more output devices, first output that includes the first portion of the first avatar and excludes a second portion of the first avatar.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/919,870Filed 2 Jul 2020
Granted: 16 Apr 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification21 codes

G16H 20/30
ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient…:…
A63B 24/0006
Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis: computerised…
A63B 71/0622
Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills: visual, audio or…
G06T 19/20
Manipulating three-dimensional [3D] models or images for computer graphics: editing of three-dimensional [3D] images, e.g. changing shapes or…