# US 2020/0402638 A1 · Exercise assistant device and exercise assistant method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402638 A1
- Document ID: US20200402638A1
- Status: Published application
- Kind: A1
- Application number: 16/905,133
- Filed: 2020-06-18
- Published: 2020-12-24
- Earliest claimed: 2019-06-20
- Assignees: Codevision Inc.
- Applicants: Codevision Inc.
- Inventors: Eung Yeol SONG; Sung Ju PARK
- Web page: https://patentgrep.com/patent/US20200402638A1

## CPC

- G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
- A61B 5/0022: {Monitoring a patient using a global network, e.g. telephone networks, internet}
- A61B 5/1114: {Tracking parts of the body}
- A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)}
- A61B 5/7435: {Displaying user selection data, e.g. icons in a graphical user interface}
- A61B 5/744: {Displaying an avatar, e.g. an animated cartoon character}
- A63B 24/0087: {Electric or electronic controls for exercising apparatus of groups A63B21/00 - A63B23/00, e.g. controlling load}
- G06N 3/02: Neural networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G16H 15/00: ICT specially adapted for medical reports, e.g. generation or transmission thereof
- G16H 40/67: for remote operation
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- A61B 5/7267: {involving training the classification device}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

There is provided a device for assisting exercise, comprising a video providing unit configured to provide a first video data including a first exercise movement, a data obtaining unit configured to obtain a second video data based on an input relating to the first video data, a joint information extracting unit configured to extract a first joint information obtained by detecting plural skeletons from the second video data, an analyzing unit configured to obtain an analysis information based on a similarity determined by comparing the first joint information with a second joint information of the first video data and a recommendation unit configured to obtain recommendation information for recommending an exercise movement to a user based on at least one of the first video data, the second video data and the analysis information from a database including plural exercise movements may be provided according to an embodiment.

## Claims

### Claim 1

A Device for assisting exercise, comprising
a video providing unit configured to provide a first video data including a first exercise movement;
a data obtaining unit configured to obtain a second video data based on an input relating to the first video data;
a joint information extracting unit configured to extract a first joint information obtained by detecting plural skeletons from the second video data;
an analyzing unit configured to obtain an analysis information based on a similarity determined by comparing the first joint information with a second joint information of the first video data; and
a recommendation unit configured to obtain recommendation information for recommending an exercise movement to a user based on at least one of the first video data, the second video data and the analysis information from a database including plural exercise movements.

### Claim 2 (depends on 1)

The device of the claim 1,
wherein the video providing unit configured to provide the first video data while the second exercise movement is performed.

### Claim 3 (depends on 1)

The device of the claim 1,
wherein the first joint information is obtained for each of frames included in the second video data,
wherein each of the frames is obtained based on a first interval,
wherein the first joint information is obtained by extracting a skeleton corresponding to a body part out of plural skeletons which are included in each of the frames included in the second video data.

### Claim 4 (depends on 3)

The device of the claim 3,
wherein the second joint information is obtained for each of frames included in the first video data,
wherein each of the frames is obtained based on a second interval,
wherein the first joint information is obtained by extracting a skeleton corresponding to a body part out of plural skeletons which are included in each of the frames included in the first video data.

### Claim 5 (depends on 4)

The device of the claim 4,
wherein the analyzing unit configured to compare the first joint information with the second joint information,
wherein the first joint information and the second joint information are normalized such that the first interval and the second interval correspond to each other.

### Claim 6 (depends on 1)

The device of the claim 1,
wherein the analyzing unit configured to obtain an analysis information calculated based on a following equation,
Similarity
=
(
1

0

0
-

θ
-
θ
t


θ
t

*
1

00
)


(
%
)
wherein
,
θ
=
cos
-
1

(
A
·
B

A



B

)
,
A is a vector value of the first joint information, B is a vector value of the second joint information, and θt is a predetermined threshold value

### Claim 7 (depends on 6)

The device of the claim 6,
wherein when a similarity included in the analysis information is greater than or equal to a predetermined value, the recommendation unit increases at least one of exercise intensity, number of performance, and performance velocity per for the first video data, or recommends an exercise movement which has a higher level of difficulty than the first video data.

### Claim 8 (depends on 6)

The device of claim 6,
Wherein when a similarity included in the analysis information is smaller than or equal to a predetermined value, the recommendation unit decreases at least one of exercise intensity, number of performance, and performance velocity for the first video data, or recommends an exercise movement which has a lower level of difficulty than the first video data.

### Claim 9 (depends on 1)

The device of claim 1,
wherein the second exercise movement is based on imitating movements of a user for the first video data

### Claim 10 (depends on 1)

The device of claim 1,
wherein the analysis information includes at least one of accuracy, number of performances, a movement time, a performance velocity of movement and an ability of each body part.

### Claim 11 (depends on 3)

The device of claim 3,
wherein the performance number of the second exercise movement is measured based on a change of location of the first joint information.

### Claim 12 (depends on 1)

The device of claim 1,
wherein the analysis information includes first analysis information which analyzes a performance ability for plural body parts,
wherein the recommendation information includes a required exercise movement to at least a part of the plural body parts based on the first analysis information.

### Claim 13 (depends on 12)

The device of claim 12,
wherein the plural body parts include at least one of an entire body, an upper body, a lower body, an abdominal region and arms.

### Claim 14 (depends on 1)

The device of claim 1,
wherein the recommendation unit recommends an exercise movement based on at least one of the first video data, the second video data, and the analysis information and pre-obtained a first information,
wherein the first information is an information which is obtained from a client device related to a user performing the first exercise movement.

### Claim 15 (depends on 14)

The device of claim 14,
wherein the first information includes at least one of gender, age, condition of health, information of BMI, cycle of exercise, amount of exercise and intensity of exercise.

### Claim 16

A method for assisting exercise, the method including,
providing a video providing a first video data including a first exercise movement;
obtaining a second video data including a second exercise movement obtained by responding to an input of the first video data;
extracting a fist joint information obtained by detecting plural skeleton from the second video data;
providing analysis information based on a similarity determined by comparing the first joint information with the second joint information of the first video data; and
providing a recommendation information recommends an exercise movement based on at least one of the first video data, the second video data and the analysis information from a data base including plural exercise movements, in order to recommend an appropriate exercise movement to a user.

### Claim 17 (depends on 16)

A non-transitory computer-readable recording medium having recorded thereon one or more program comprising commands for executing the method of the claim 16.
