# US 2012/0075462 A1 · Blow tracking user interface system and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0075462 A1
- Document ID: US20120075462A1
- Status: Published application
- Kind: A1
- Application number: 12/889,347
- Filed: 2010-09-23
- Published: 2012-03-29
- Earliest claimed: 2010-09-23
- Assignees: Sony Computer Entertainment Inc.
- Inventors: Ruxin Chen; Steven Osman
- Web page: https://patentgrep.com/patent/US20120075462A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A blow tracking user interface method and apparatus may detect an orientation of blowing of a user's breath and a magnitude of blowing of the user's breath. A blow vector may be generated from the orientation and magnitude of the blowing of the user's breath. The blow vector may be used as a control input in a computer program.

## Claims

### Claim 1

A blow tracking interface method for control of a computer program, comprising:
detecting orientation of blowing of a breath;
detecting a magnitude of blowing of the breath;
generating a blow vector from the orientation and magnitude; and
using the blow vector as a control input in a computer program.

### Claim 2 (depends on 1)

The method of claim 1, further comprising determining a timing of the blowing of the breath, wherein the timing of the blowing of the breath determines timing of an action in the computer program.

### Claim 3 (depends on 2)

The method of claim 2, further comprising synchronizing timing of actions in the computer program to the timing of the blowing of breath.

### Claim 4 (depends on 1)

The method of claim 1, wherein detecting the orientation and/or magnitude of the blowing of the breath includes obtaining one or more thermal infrared images of a user with one or more thermographic cameras, analyzing the one or more infrared images, and identifying one or more patterns in the one or more images corresponding to the blowing of breath from the one or more thermal infrared image.

### Claim 5 (depends on 4)

The method of claim 4 wherein detecting the orientation and/or magnitude of the breath includes analyzing thermal contours in the one or more thermal infrared images, identifying one or more patterns characteristic of the blowing of the breath from the thermal infrared image, and determining the orientation from the one or more patterns.

### Claim 6 (depends on 4)

The method of claim 4, wherein the thermographic camera is mounted to a pair of glasses worn by the user.

### Claim 7 (depends on 4)

The method of claim 4, wherein generating the blow vector includes augmenting information obtained from the one or more thermal infrared images with information relating to the breath from one or more auxiliary sensors.

### Claim 8 (depends on 6)

The method of claim 6, wherein the one or more auxiliary sensors includes a microphone or camera.

### Claim 9 (depends on 6)

The method of claim 6, wherein the auxiliary detector is an additional thermographic camera mounted to a pair of glasses worn by the user.

### Claim 10 (depends on 1)

The method of claim 1 wherein generating a blow vector includes generating a blow vector for breath through the user's nose and determining an orientation of the user's face from an orientation of the blow vector for breath through the user's nose.

### Claim 11 (depends on 1)

The method of claim 1, wherein detecting the orientation and/or magnitude of the blowing of the breath includes detecting sounds of the blowing of the breath with a microphone.

### Claim 12 (depends on 1)

The method of claim 1 wherein detecting the orientation of the breath includes determining a source of the blowing of the breath.

### Claim 13 (depends on 6)

The method of claim 6 wherein determining the source of the breath includes determining which user from among a plurality of users is the source of the blowing of the breath.

### Claim 14 (depends on 6)

The method of claim 6 wherein determining the source of the breath includes determining whether the source of the blowing of the breath is a user's mouth or nose.

### Claim 15 (depends on 1)

The method of claim 1, further comprising mapping the timing and intensity of a user's breathing to simulated breathing of an avatar controlled by a user so that the avatar appears to breathe in synchronization with user.

### Claim 16 (depends on 1)

The method of claim 1 wherein detecting orientation of a breath includes determining whether a user is blowing breath through the user's mouth or nose.

### Claim 17 (depends on 1)

The method of claim 1 wherein detecting orientation of a breath includes determining whether a user is breathing in or out.

### Claim 18 (depends on 1)

The method of claim 1 wherein detecting orientation or magnitude of a breath includes determining an intended temperature of the breath.

### Claim 19 (depends on 1)

The method of claim 1 wherein using the blow vector as a control input includes using the orientation of the breath to determine a direction of an action taken by the computer program.

### Claim 20 (depends on 1)

The method of claim 1 wherein using the blow vector as a control input includes using the magnitude of the breath to determine the magnitude of an action taken by the computer program.

### Claim 21 (depends on 1)

The method of claim 1 wherein the orientation of the breath is determined relative to a video display.

### Claim 22 (depends on 1)

The method of claim 1 wherein generating the blow vector includes using a past breath history to improve an estimation of the blow vector.

### Claim 23

A blow tracking interface apparatus for control of a computer program, comprising:
a blow detector configured to detect one or more signals corresponding to blowing of a breath; and
a processor coupled to the blow detector, wherein the processor is configured to determine an orientation and magnitude of the blowing of the breath from the one or more signals, and generate a blow vector from the orientation and magnitude, wherein the processor is configured to use the blow vector as a control input in a computer program.

### Claim 24 (depends on 23)

The apparatus of claim 23 wherein the breath detector includes one or more thermographic cameras.

### Claim 25 (depends on 24)

The apparatus of claim 24, wherein the processor is configured to detect the orientation and/or magnitude of the breath by analyzing thermal contours in the image to identify one or more patterns characteristic the breath from the thermal infrared image.

### Claim 26 (depends on 24)

The apparatus of claim 24 wherein the one or more thermographic cameras include a thermographic camera mounted to a video display that is coupled to the processor.

### Claim 27 (depends on 24)

The apparatus of claim 24 wherein the one or more thermographic cameras include a thermographic camera mounted to a pair of glasses.

### Claim 28 (depends on 23)

The apparatus of claim 23 wherein the blow detector includes one or more microphones.

### Claim 29

A non-transitory computer readable medium having computer executable instructions embodied therein, wherein the instructions are configured to implement a blow tracking interface method for control of a computer program when executed by a computer processor, wherein the method implemented by the execution of the instructions comprises:
detecting orientation of blowing of a breath;
detecting a magnitude of blowing of the breath;
generating a blow vector from the orientation and magnitude; and
using the blow vector as a control input in a computer program.
