# US 2012/0251995 A1 · Apparatus and method for tutoring in convergence space of real and virtual environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0251995 A1
- Document ID: US20120251995A1
- Status: Published application
- Kind: A1
- Application number: 13/439,809
- Filed: 2012-04-04
- Published: 2012-10-04
- Earliest claimed: 2011-04-04
- Assignees: Electronics and Telecommunications Research Institute
- Inventors: Sang Won GHYME; Sanghyun JOO; Jaehwan KIM; Hyo Taeg JUNG; Il Kwon JEONG
- Web page: https://patentgrep.com/patent/US20120251995A1

## CPC

- G09B 5/06: with both visual and audible presentation of the material to be studied
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

An apparatus for tutoring in a convergence space of real and virtual environment, includes a character data processing unit for generating character shape and operation data of a participant or performing rendering on the character shape and operation data; a desk data processing unit for generating desk shape data or perform the rendering on the desk shape data; a voice and sound data processing unit for generating voice or sound data or outputs the voice or sound data; and a virtual space processing unit for generating real space data and providing the generated data to another participant, generating the convergence space data, or generating the character shape data, the operation data, the desk shape data, and the voice or sound data, then providing the generated data to the character data processing unit, the desk data processing unit, and the voice and sound data processing unit.

## Claims

### Claim 1

An apparatus for tutoring in a convergence space of real and virtual environment, comprising:
a character data processing unit configured to generate character shape and operation data of a participant by capturing a character image of the participant or perform rendering on the character shape and operation data within the convergence space provided from the outside;
a desk data processing unit configured to generate desk shape data by capturing a desk of the participant or perform the rendering on the desk shape data within the convergence space provided from the outside;
a voice and sound data processing unit configured to generate voice or sound data based on voice or sound generated under the environment that the participant receives a tutoring service or outputs the voice or sound data within the convergence space provided from the outside; and
a virtual space processing unit configured to generate real space data virtualized by using the character shape data, the operation data, the desk shape data, and the voice or sound data and provide the generated data to another participant through a communication network, generate the convergence space data based on the virtualized real space data and the generated virtualized real space data received from another participant, or generate the character shape data, the operation data, the desk shape data, and the voice or sound data within the convergence space from the convergence space data and then, provide the generated data to the character data processing unit, the desk data processing unit, and the voice and sound data processing unit.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the character data processing unit generates a three-dimensional character shape by applying a structured from motion method to the character image and then, generates three-dimensional operation data by analyzing object deformation configured from the changing three-dimensional shape.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the character data processing unit generates a three-dimensional shape by applying a stereo-matching method to the character image and then, generates the three-dimensional operation data by analyzing the object deformation.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the character data processing unit transforms the character shape and operation data by referring to a character selected or preferred by the participant and then, provides the transformed data to the virtual space processing unit or transforms the character shape and operation data provided from the virtual space processing unit.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the desk data processing unit generates the desk shape data augmented or diminished by applying an augmented or diminished reality method to the captured desk image.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the desk data processing unit uses a diminished reality method based on luminance correction.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the virtual space processing unit transforms the operation data of another participant in the convergence space according to body conditions of the participant into voice or transforms the voice data of another participant into operation data.

### Claim 8

A method for tutoring in a convergence space, comprising:
generating character shape and operation data of a participant by capturing a character image of the participant;
generating desk shape data based on an image obtained by capturing a desk of the participant;
generating voice or sound data based on voice or sound generated under the environment that the participant receives a tutoring service;
generating virtualized real space data by using the character shape data, the operation data, the desk shape data, and the voice or sound data of the participant and providing the generated virtualized real space data to another participant through a communication network;
generating convergence space data based on the virtualized real space data and the generated virtualized real space data that are received from another participant; and
generating the character shape data, the operation data, the desk shape data, and the voice or sound data within the convergence space from the convergence space data and then, providing the generated data through each of the output devices.

### Claim 9 (depends on 8)

The method of claim 8, wherein said generating the operation data includes:
generating a three-dimensional character shape by applying a structured from motion method to the character image; and
generating three-dimensional operation data by analyzing object deformation configured from the changing three-dimensional shape.

### Claim 10 (depends on 8)

The method of claim 8, wherein said generating the operation data generates a three-dimensional shape by applying a stereo-matching method to the character image and then, generates the three-dimensional operation data by analyzing the object deformation.

### Claim 11 (depends on 8)

The method of claim 8, wherein said generating the operation data transforms the character shape and operation data into the selected or preferred character by referring to a character selected or preferred by the participant to generate the character shape and operation data.

### Claim 12 (depends on 8)

The method of claim 8, wherein said generating the desk shape data generates the desk shape data augmented or diminished by applying an augmented or diminished reality method to the captured desk image.

### Claim 13 (depends on 12)

The method of claim 12, wherein said generating the desk shape data uses a diminished reality method based on luminance correction to generate the desk shape data.

### Claim 14 (depends on 8)

The method of claim 8, wherein said outputting to each of the output devices further includes transforming the operation data of another participant in the convergence space according to body conditions of the participant into voice or transforming the voice data of another participant into operation data.

### Claim 15 (depends on 8)

The method of claim 8, wherein said outputting to each of the output device deforms the desk shape data of the participant or another participant by comparing the desk shape data of the virtualized real space data of the participant with the desk shape data of the virtualized real space data of another participant or generates new desk shape data to generate the desk shape data within the convergence space.
