# US 2024/0203557 A1 · Method and system for providing exercise therapy using artificial intelligence posture estimation model and motion analysis model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203557 A1
- Document ID: US20240203557A1
- Status: Published application
- Kind: A1
- Application number: 18/531,375
- Filed: 2023-12-06
- Published: 2024-06-20
- Earliest claimed: 2022-12-15
- Assignees: Everex
- Applicants: Everex
- Inventors: Chan YOON; Byung Hoon KIM
- Web page: https://patentgrep.com/patent/US20240203557A1

## CPC

- G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
- A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)}
- A63B 24/00: Electric or electronic controls for exercising apparatus of preceding groups; {Controlling or monitoring of exercises, sportive games, training or athletic performances}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44)
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G16H 30/40: for processing medical images, e.g. editing
- G16H 40/67: for remote operation
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 80/00: ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of providing exercise therapy using an artificial intelligence motion analysis model comprises: receiving, from a doctor terminal, prescription information related to exercise for a patient; allocating, to an account of the patient, based on the prescription information, an exercise plan including at least one prescribed exercise; receiving, from a patient terminal, an exercise image in which an exercise according to the prescribed exercise is photographed; extracting, from the exercise image including a subject of the patient, a keypoint corresponding to each of a plurality of preset joint points, using an artificial intelligence posture estimation model trained based on a training data set; and analyzing, using an artificial intelligence motion analysis model, a relative positional relationship between the keypoints, and analyzing, based on the analysis of the positional relationship, an exercise motion of the patient for the prescribed exercise.

## Claims

### Claim 1

A method of providing exercise therapy using an artificial intelligence motion analysis model, the method comprising:
receiving, from a doctor terminal, prescription information related to exercise for a patient;
allocating, to an account of the patient, based on the prescription information, an exercise plan including at least one prescribed exercise;
receiving, from a patient terminal, an exercise image in which an exercise according to the prescribed exercise is photographed;
extracting, from the exercise image including a subject of the patient, a keypoint corresponding to each of a plurality of preset joint points, using an artificial intelligence posture estimation model trained based on a training data set;
analyzing, using an artificial intelligence motion analysis model, a relative positional relationship between the keypoints, and analyzing, based on the analysis of the positional relationship, an exercise motion of the patient for the prescribed exercise; and
transmitting an analysis result of the exercise motion of the patient to the patient terminal,
wherein the posture estimation model, trained with the training data set, extracts each of visible joint points and invisible joint points of the subject from the exercise image as the keypoint, such that the motion analysis model analyzes the exercise motion of the patient based on all of the visible joint points that are visible and the invisible joint points that are invisible in the exercise image,
wherein the training data set from which the posture estimation model is trained is configured with a plurality of data groups, each corresponding to a different information attribute,
wherein, in a first data group of the plurality of data groups, position information on each of a predesignated plurality of training target joint points among the joint points of the subject included in the training target exercise image is sequentially arranged based on a predefined sequence corresponding to each of the training target joint points,
wherein, in a second data group of the plurality of data groups, a data value representing whether each of the training target joint points is visible in the training target exercise image is arranged in the same sequence as the position information included in the first data group,
wherein position information arranged in a specific sequence in the first data group is defined as position information of one of a first type and a second type different from the first type based on a data value arranged in the specific sequence in the second data group,
wherein the first type of position information is actual position information on an area where the training target joint point is actually positioned in the training target exercise image,
wherein the second type of position information is predicted position information on the training target joint point predicted from the training target exercise image, and
wherein the posture estimation model, based on the data value arranged in the specific sequence of the second data group, differently sets a training weight for the position information arranged in the specific sequence of the first data group.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
outputting the exercise image to the patient terminal in real time, in conjunction with the exercise image being photographed on the patient terminal; and
providing a graphic object corresponding to the extracted keypoint that overlaps an area where a subject corresponding to the patient is positioned in the exercise image, so as to allow the patient to recognize a joint point where an analysis is performed on an exercise posture of the patient.

### Claim 3 (depends on 2)

The method of claim 2, wherein in the extracting of the keypoint, the visible joint point that is visible in the exercise image is specified among the plurality of preset joint points, and the invisible joint point that is invisible in the exercise image is predicted among the plurality of preset joint points.

### Claim 4 (depends on 2)

The method of claim 2, wherein in the analyzing of the exercise motion of the patient, a relative positional relationship between the keypoints is analyzed based on rule information related to the prescribed exercise, and the exercise motion of the patient is analyzed by judging whether the relative positional relationship between the keypoints satisfies the rule information.

### Claim 5 (depends on 4)

The method of claim 4, wherein visual appearances of the graphic objects overlapping the exercise image are different depending on whether the relative positional relationship between the extracted keypoints satisfies the rule information.

### Claim 6 (depends on 5)

The method of claim 5, wherein the analysis result of the exercise motion of the patient includes:
a first analysis result providing the graphic object corresponding to the keypoint that overlaps the exercise image in real time with a different visual appearance based on the rule information, in a state in which the exercise image is being photographed on the patient terminal; and
a second analysis result including an evaluation score of the patient for the prescribed exercise based on a keypoint extracted from each of a plurality of frames constituting the exercise image,
wherein the first analysis result is generated by a motion analysis model of an application installed on the patient terminal,
wherein the second analysis result is generated on a cloud server in conjunction with the application, and
wherein both the first analysis result and the second analysis result are transmitted to the doctor terminal.

### Claim 7

A system for providing exercise therapy, the system comprising:
a communication unit configured to receive, from a doctor terminal, prescription information related to exercise for a patient; and
a control unit configured to allocate, to an account of the patient, based on the prescription information, an exercise plan including at least one prescribed exercise,
wherein the control unit is configured to:
receive, from a patient terminal, an exercise image in which an exercise according to the prescribed exercise is photographed;
extract, from the exercise image including a subject of the patient, a keypoint corresponding to each of a plurality of preset joint points, using an artificial intelligence posture estimation model trained based on a training data set;
analyze, using an artificial intelligence motion analysis model, an exercise motion of the patient for the prescribed exercise from the exercise image; and
transmit an analysis result of the exercise motion of the patient to the patient terminal,
wherein the posture estimation model, trained with the training data set, extracts each of visible joint points and invisible joint points of the subject from the exercise image as the keypoint, such that the motion analysis model analyzes the exercise motion of the patient based on all of the visible joint points that are visible and the invisible joint points that are invisible in the exercise image,
wherein the training data set from which the posture estimation model is trained is configured with a plurality of data groups, each corresponding to a different information attribute,
wherein, in a first data group of the plurality of data groups, position information on each of a predesignated plurality of training target joint points among the joint points of the subject included in the training target exercise image is sequentially arranged based on a predefined sequence corresponding to each of the training target joint points,
wherein, in a second data group of the plurality of data groups, a data value representing whether each of the training target joint points is visible in the training target exercise image is arranged in the same sequence as the position information included in the first data group,
wherein position information arranged in a specific sequence in the first data group is defined as position information of one of a first type and a second type different from the first type based on a data value arranged in the specific sequence in the second data group,
wherein the first type of position information is actual position information on an area where the training target joint point is actually positioned in the training target exercise image,
wherein the second type of position information is predicted position information on the training target joint point predicted from the training target exercise image, and
wherein the posture estimation model, based on the data value arranged in the specific sequence of the second data group, differently sets a training weight for the position information arranged in the specific sequence of the first data group.

### Claim 8

A program executable by one or more processes on an electronic device and stored on a computer-readable recording medium, the program comprising instructions for performing of:
receiving, from a doctor terminal, prescription information related to exercise for a patient;
allocating, to an account of the patient, based on the prescription information, an exercise plan including at least one prescribed exercise;
receiving, from a patient terminal, an exercise image in which an exercise according to the prescribed exercise is photographed;
extracting, from the exercise image including a subject of the patient, a keypoint corresponding to each of a plurality of preset joint points, using an artificial intelligence posture estimation model trained based on a training data set;
analyzing, using an artificial intelligence motion analysis model, a relative positional relationship between the keypoints, and analyzing, based on the analysis of the positional relationship, an exercise motion of the patient for the prescribed exercise; and
transmitting an analysis result of the exercise motion of the patient to the patient terminal,
wherein the posture estimation model, trained with the training data set, extracts each of visible joint points and invisible joint points of the subject from the exercise image as the keypoint, such that the motion analysis model analyzes the exercise motion of the patient based on all of the visible joint points that are visible and the invisible joint points that are invisible in the exercise image,
wherein the training data set from which the posture estimation model is trained is configured with a plurality of data groups, each corresponding to a different information attribute,
wherein, in a first data group of the plurality of data groups, position information on each of a predesignated plurality of training target joint points among the joint points of the subject included in the training target exercise image is sequentially arranged based on a predefined sequence corresponding to each of the training target joint points,
wherein, in a second data group of the plurality of data groups, a data value representing whether each of the training target joint points is visible in the training target exercise image is arranged in the same sequence as the position information included in the first data group,
wherein position information arranged in a specific sequence in the first data group is defined as position information of one of a first type and a second type different from the first type based on a data value arranged in the specific sequence in the second data group,
wherein the first type of position information is actual position information on an area where the training target joint point is actually positioned in the training target exercise image,
wherein the second type of position information is predicted position information on the training target joint point predicted from the training target exercise image, and
wherein the posture estimation model, based on the data value arranged in the specific sequence of the second data group, differently sets a training weight for the position information arranged in the specific sequence of the first data group.
