# US 11,992,731 B2 · AI motion based smart hometraining platform

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,992,731 B2
- Document ID: US11992731B2
- Status: Granted
- Kind: B2
- Application number: 17/967,965
- Filed: 2022-10-18
- Published: 2024-04-18
- Granted: 2024-05-28
- Earliest claimed: 2022-10-18
- Assignees: Multics Co., Ltd.
- Applicants: Multics Co., Ltd.
- Inventors: Seung-Su Yu
- Web page: https://patentgrep.com/patent/US11992731B2

## CPC

- A63B 24/0075: {Means for generating exercise programmes or schemes, e.g. computerized virtual trainer, e.g. using expert databases}
- A63B 71/0622: {Visual, audio or audio-visual systems for entertaining, instructing or motivating the user}
- G06T 7/0016: {involving temporal comparison}
- G06T 7/251: {involving models}
- G06T 11/00: Two-dimensional [2D] image generation
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- G16H 15/00: ICT specially adapted for medical reports, e.g. generation or transmission thereof
- G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 80/00: ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
- A63B 2024/0068: {Comparison to target or threshold, previous performance or not real time comparison to other individuals}
- A63B 2071/0625: {Emitting sound, noise or music}
- A63B 2071/0647: {Visualisation of executed movements}
- A63B 2071/068: {Input by voice recognition}
- A63B 2220/05: Image processing for measuring physical parameters
- A63B 2220/807: Photo cameras
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30196: Human being; Person
- G06T 2207/30221: Sports video; Sports image
- G16H 40/67: for remote operation
- A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

An AI motion recognition based smart hometraining platform is configured to include: a photographing unit; an motion image generation unit; a joint information generation unit; a deep learning determination unit determining whether the user motion is suitable by inputting user joint information; a correction information generation unit extracting correction information, and generating an image for a correction image or correction information based on the extracted correction information; a user image generation unit generating a user image acquired by overlaying a joint information image of the joint information generation unit to the user image; and a display displaying both the image for the correction message or the correction information and the user image of the user image generation unit in the display.

## Claims

### Claim 1

An AI motion recognition based smart hometraining platform for enhancing mild cognitive impairment, comprising:
a photographing unit photographing a motion of a user and generating an motion image;
an motion image generation unit generating a frame-specific image (image, Ik, k=0, −1, −2, . . . , k represents a discrete time) from the motion image at a predetermine time interval (e.g., ΔT1=1/1˜10 sec, for each frame);
a joint information generation unit extracting a joint point for each frame based on the user image, and generating positional information (2D coordinates or 3D coordinates (Xki, Yki, Zki)) of each joint point;
a deep learning determination unit determining whether the user motion is suitable by inputting user joint information generated by the joint information generation unit by using a prelearned deep learning determination model M with joint information of multiple users for a corresponding training motion as learning data;
a correction information generation unit extracting correction information by comparing joint information of a trainer motion (a model motion for each type) and the joint information of the user motion, and generating an image for a correction image or correction information based on the extracted correction information;
a user image generation unit generating a user image acquired by overlaying a joint information image of the joint information generation unit to the user image; and
a display displaying both the image for the correction message or the correction information generated by the correction information generation unit and the user image of the user image generation unit in the display.

### Claim 2 (depends on 1)

The AI motion recognition based smart hometraining platform for enhancing mild cognitive impairment of claim 1, further comprising:
a service providing server receiving a user exercise history and a user exercise performing result from a server communication unit provided in a local device and transmitting the user exercise history and performing result to an expert terminal and a guardian terminal;
an expert terminal presenting an exercise prescription by viewing the user exercise history and performing result from the service providing server; and
a guardian terminal receiving a report for the user exercise history and performing result.

### Claim 3 (depends on 2)

The AI motion recognition based smart hometraining platform for enhancing mild cognitive impairment of claim 2, wherein the correction information generation unit is constituted by a correction information extraction unit extracting correction information for an error part and a correction message generation unit visualizing the extracted correct information, and
when the deep learning determination unit determines that the user motion is unsuitable, the correction information extraction unit selects the error part and extracts the correction information by comparing joint information of a training motion of a training motion information storage unit and the joint information for the user motion provided by the joint information generation unit.

### Claim 4 (depends on 3)

The AI motion recognition based smart hometraining platform for enhancing mild cognitive impairment of claim 3, wherein the correction message generation unit performs displaying a user error part in a training motion skeleton image, displaying the user error part in a training motion image, displaying the user error part in a user skeleton image, displaying the user error part in the user motion image, color expression or blink marking for the error part, or literal expression of correction information such as an error part correction angle by a voice transfer method of the correction information through a sound output unit.

### Claim 5 (depends on 3)

The AI motion recognition based smart hometraining platform for enhancing mild cognitive impairment of claim 3, wherein the training motion as a motion for improving mild cognitive impairment or a dementia prevention motion includes:
a palm applause exercise using palms, such as vertical applause and horizontal applause for peripheral nerve and blood circulation promotion cognitive function enhancement,
a both-arm exercise such as pushing both arms upward, forward, and sideward for brain balance and exercise ability cognitive function enhancement, and
a full-body touch exercise that touches a head, a shoulder, and a hip with hands, and
the joint information generation unit generates positional information (2D coordinates or 3D coordinates) of a body, a shoulder joint, an elbow joint, a wrinkle joint, the head, etc., and
the correction information extraction unit uses adjacent joint angles (e.g., an angle between an upper arm and a lower arm) of the training motion and the user motion as parameters and compares a difference of the adjacent joint angles, and recognizes a corresponding part as the motion error part when a difference value of the angle is equal to or more than a threshold.

### Claim 6 (depends on 2)

The AI operation recognition based smart hometraining platform for enhancing mild cognitive impairment of claim 2, further comprising:
a sound input unit generating a voice file by sensing a voice of a user;
a voice recognition unit recognizing a pronunciation of the recorded voice file and
a voice determination unit (not illustrated) determining whether the voice of the user is suitable,
wherein the training motion further includes brain nerve gymnastics constituted by making a sound, pupil exercise, cheek tongue writing, patting a face and the head with the hands for simulating triple nerves, facial serves, glossopharyngeal nerves, and hypoglossal nerves through vocalization, and
the joint information generation unit further generates positional information for at least facial major spot points (non-resin major points) such as philtrum, lip, eye, noise, etc.

### Claim 7 (depends on 2)

The AI operation recognition based smart hometraining platform for enhancing mild cognitive impairment of claim 2, wherein the service providing server is configured to include a user database, an exercise evaluation unit, a level management unit, an expert exercise prescription unit, and a deep learning determination model learning unit,
the user database stores an exercise-specific performing history of the user, a cognitive impairment degree of the user, and an evaluation results of a motion for each type daily, weekly, and monthly,
the exercise evaluation unit determines whether the training motion is successfully performed by counting the number of errors determined by the deep learning determination unit,
the level management unit classifies and stores levels into a basic level, a middle level, and a high level for the evaluation result of the exercise evaluation unit,
the expert exercise prescription unit expresses training motion information recommended by the expert terminal through the display or an sound output unit, and
the deep learning determination model learning unit trains a model by using motions and determination results of various users as learning data.