# US 10,864,423 B2 · Augmented learning system for tai-chi chuan with head-mounted display

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,864,423 B2
- Document ID: US10864423B2
- Status: Granted
- Kind: B2
- Application number: 15/805,885
- Filed: 2017-11-07
- Published: 2018-05-10
- Granted: 2020-12-15
- Earliest claimed: 2016-11-10
- Assignees: National Taiwan University
- Applicants: National Taiwan University
- Inventors: Yi-Ping Hung; Ping-Hsuan Han
- Web page: https://patentgrep.com/patent/US10864423B2

## CPC

- G09B 5/02: with visual presentation of the material to be studied, e.g. using film strip
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/16: Sound input; Sound output (speech processing G10L)
- G09B 19/0038: {Sports}
- A63B 71/0622: {Visual, audio or audio-visual systems for entertaining, instructing or motivating the user}
- A63B 2071/0638: {Displaying moving images of recorded environment, e.g. virtual environment}
- A63B 2071/0647: {Visualisation of executed movements}
- A63B 2071/0666: {worn on the head or face, e.g. combined with goggles or glasses}
- A63B 2220/803: Motion sensors
- A63B 2220/836: Sensors arranged on the body of the user
- A63B 2225/50: Wireless data transmission, e.g. by radio transmitters or telemetry
- A63B 2244/10: Combat sports
- B64U 2101/30: for imaging, photography or videography
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H)
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR

## Abstract

Disclosed is an augmented learning system for tai-chi chuan, including a head-mounted display put on a head of a learner, and a plurality of action sensors worn on the head or a body of the learner. The action of the learner is sensed by the action sensors, which generate and wireless transfer corresponding sensing signals to the head-mounted display. Based on the sensing signals, the head-mounted display constructs and shows images of the learner as well as a virtual coach demonstrating tai-chi chuan in front of the learner. Also, virtual assistants at different positions demonstrate the same tai-chi chuan such that the learner clearly watches each detailed move of tai-chi chuan at various views. Therefore, the present invention helps the learner effectively self-examining and self-correcting every move by means of video and audio so as to overcome learning obstacles and speed up learning progress for tai-chi chuan.

## Claims

### Claim 1

An augmented learning system for tai-chi chuan for assisting a tai-chi chuan learner, comprising:
a head-mounted display configured to be put on a head of the learner, and configured to provide a function of augmented reality (AR)/mixed reality (MR) and display a virtual reality image, and having a function of wirelessly receiving and transferring a signal; and
a movable photographing device having a function of moving and wirelessly receiving and transferring a signal, and provided with a camera for photographing the learner to obtain a learner image, the learner image being wirelessly transferred to the head-mounted display,
wherein the head-mounted display is provided with a location and direction sensor and a wireless transmitter, the location and direction sensor senses a location and direction of the head-mounted display, the wireless transmitter transfers the location and direction of the head-mounted display to the movable photographing device, the movable photographing device receives the location and direction of the head-mounted display, and performs self-adjustment to face the learner, the camera keeps photographing a front image of the learner, the movable photographing device transfers the front image of the learner to the head-mounted display, the virtual reality image comprises a virtual mirror, a plurality of virtual coaches, and a plurality of virtual assistants, the virtual coaches and the virtual assistants are around the virtual mirror, the head-mounted display takes the front image of the learner as a front image of a virtual learner, the front image of the virtual learner is shown on the virtual mirror, and each of the virtual coaches and the virtual assistants stands on a virtual support plate to demonstrate tai-chi chuan for the learner to view and learn.

### Claim 2 (depends on 1)

The augmented learning system for tai-chi chuan as claimed in claim 1, wherein the movable photographing device is implemented by a drone or a robot provided with a function of moving, photographing, and remote control.

### Claim 3 (depends on 1)

The augmented learning system for tai-chi chuan as claimed in claim 1, wherein the head-mounted display shows the virtual reality image based on a voice command or a body gesture command by the learner.

### Claim 4 (depends on 3)

The augmented learning system for tai-chi chuan as claimed in claim 3, wherein the voice command comprises:
“continuous” for starting or switching to the continuous play mode;
“segment” for starting or switching to the segmental play mode;
“go” or “play” for displaying the tai-chi chuan learning image;
“pause” for stopping displaying the tai-chi chuan learning image;
“freeze” for freezing the location of the virtual coach, moving toward the virtual coach to closely watch the action of the virtual coach, and displaying the resultant image;
“reset” for rotating a facing direction of the virtual coach to the same facing direction of the learner after the voice command of freeze, and displaying the resultant image;
“restart” for returning a first move of the tai-chi chuan learning image, and displaying the resultant image;
“last” for returning to a previous, and displaying the resultant image;
“next” for switching to a next move, and displaying the resultant image;
“last move” for returning to a first frame of the previous move, and pausing;
“next move” for switching a first frame of the next move, and pausing;
“last action” for switching to a first frame of a previous detailed action move under the segmental play mode, and pausing;
“next action” for switching to a first frame of a next detailed action move under the segmental play mode, and pausing; and
“close” for terminating the play mode, and returning to a preset state after power on.