# US 8,465,108 B2 · Directed performance in motion capture system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,465,108 B2
- Document ID: US8465108B2
- Status: Granted
- Kind: B2
- Application number: 13/603,912
- Filed: 2012-09-05
- Published: 2012-12-27
- Granted: 2013-06-18
- Earliest claimed: 2010-01-15
- Assignees: Microsoft Corporation
- Applicants: Relja Markovic; Stephen G Latta; Kevin A Geisner; Christopher Vuchetich; Darren A Bennett; Brian S Murphy; Shawn C Wright
- Inventors: Relja Markovic; Stephen G Latta; Kevin A Geisner; Christopher Vuchetich; Darren A Bennett; Brian S Murphy; Shawn C Wright
- Web page: https://patentgrep.com/patent/US8465108B2

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/42: by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
- A63F 13/50: Controlling the output signals based on the game progress
- A63F 2300/1081: Input via voice recognition
- A63F 2300/1087: comprising photodetecting means, e.g. a camera
- A63F 2300/305: for providing a graphical or textual hint to the player
- A63F 2300/61: Score computation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR

## Abstract

Techniques for enhancing the use of a motion capture system are provided. A motion capture system tracks movement and audio inputs from a person in a physical space, and provides the inputs to an application, which displays a virtual space on a display. Bodily movements can be used to define traits of an avatar in the virtual space. The person can be directed to perform the movements by a coaching avatar, or visual or audio cues in the virtual space. The application can respond to the detected movements and voice commands or voice volume of the person to define avatar traits and initiate pre-scripted audio-visual events in the virtual space to provide an entertaining experience. A performance in the virtual space can be captured and played back with automatic modifications, such as alterations to the avatar's voice or appearance, or modifications made by another person.

## Claims

### Claim 1

A motion capture system, comprising:
a depth camera system, the depth camera system obtains images of a field of view;
a display; and
a processor in communication with the depth camera system and the display, the processor executes instructions to:
display a virtual space comprising an avatar on the display, provide directions to a person, the person performs movements in the field of view in a first time period in response to the directions, process the images to detect the movements of the person, update the virtual space so that the avatar provides a performance, the avatar exhibits a trait and moves correspondingly to the movements of the person in real time as the person performs the movements in the performance, and provide a play back of the performance in a second time period, the avatar exhibits a modification to the trait and moves correspondingly to the movements of the person in the play back of the performance.

### Claim 2 (depends on 1)

The motion capture system of claim 1, wherein:
the trait comprises a costume of the avatar.

### Claim 3 (depends on 2)

The motion capture system of claim 2, wherein:
the costume of the avatar is exchanged with another avatar in the virtual space.

### Claim 4 (depends on 1)

The motion capture system of claim 1, wherein:
the trait comprises at least one of a hair color or a hair style of the avatar.

### Claim 5 (depends on 1)

The motion capture system of claim 1, wherein:
the modification to the trait comprises a modulation of a voice.

### Claim 6 (depends on 1)

The motion capture system of claim 1, wherein:
the trait comprises a facial expression.

### Claim 7 (depends on 1)

The motion capture system of claim 1, wherein:
the trait comprises a manner of walking.

### Claim 8 (depends on 1)

The motion capture system of claim 1, wherein:
the trait comprises a vocal trait.

### Claim 9 (depends on 8)

The motion capture system of claim 8, wherein:
the vocal trait comprises at least one of tone, accent, cadence, rhythm, intonation, degree of articulation or loudness.

### Claim 10 (depends on 1)

The motion capture system of claim 1, wherein:
the trait comprises a language, such that the avatar speaks in a different language in the play back of the performance in the second time period than in the performance in the first time period.

### Claim 11

A processor-implemented method in a motion capture system, comprising the processor-implemented steps of:
receiving images of a person in a field of view of the motion capture system, the person performs movements in the field of view;
displaying a virtual space on a display, the virtual space comprises an avatar which represents the person;
based on the images, detecting the movements of the person;
in a first time period, creating a scene in a performance in the virtual space by translating the movements of the person to movements of an avatar in the virtual space as the person explores different features of the virtual space; and
in a second time period, playing back the performance by depicting the movements of the avatar with a modification of the scene.

### Claim 12 (depends on 11)

The processor-implemented method of claim 11, wherein:
the modification of the scene comprise a different viewpoint of the virtual space in the second time period than in the first time period.

### Claim 13 (depends on 11)

The processor-implemented method of claim 11, wherein:
the modification of the scene comprise a different lighting in the second time period than in the first time period.

### Claim 14 (depends on 11)

The processor-implemented method of claim 11, wherein:
the modification of the scene comprise an enhanced soundtrack of the scene.

### Claim 15

A computer readable storage device comprising computer readable software embodied thereon for programming a processor to perform a method in a motion capture system, the method comprising:
receiving images of a person in a field of view of the motion capture system;
displaying a virtual space on a display, the virtual space comprises an avatar which represents the person;
based on the images, detecting the person;
displaying the avatar in the virtual space;
providing a plurality of directed moments in a performance of the avatar, each directed moment of the plurality of directed moments requires the person to complete a directed movement and each directed movement must be successfully completed before a next directed moment of the plurality of directed moments, until a final directed moment of the plurality of directed moments is reached, the avatar moves correspondingly to the directed movements of the person to interact with objects in the virtual space;
providing a record of the performance of the avatar; and
once the person has successfully completed the final directed moment, automatically modifying the record of the performance to provide a modified record, and playing back the modified record.

### Claim 16 (depends on 15)

The computer readable storage device of claim 15, wherein:
the directed movements are indicated by a plurality of visual cues in the virtual space.

### Claim 17 (depends on 16)

The computer readable storage device of claim 16, wherein:
one visual cue of the plurality of visual cues is on a ground in the virtual space; and
the person performs one of the directed movements by causing the avatar to stand on the one of the visual cues.

### Claim 18 (depends on 15)

The computer readable storage device of claim 15, wherein:
providing an optional moment in the performance of the avatar, the optional moment provides a visual cue in the virtual space for an action to perform; and
providing a visual reward in response to completion of the action by the person.

### Claim 19 (depends on 15)

The computer readable storage device of claim 15, wherein:
providing a discoverable moment in the performance of the avatar, the discoverable moment comprises an action that is discoverable by the person but is not identified by a visual cue in the virtual space; and
providing a result in the virtual space in response to discovery of the action by the person.

### Claim 20 (depends on 15)

The computer readable storage device of claim 15, wherein:
the modified record provides a surprise final moment of the performance.