# US 2012/0075501 A1 · Storage medium having stored information processing program therein, information processing apparatus, information processing method, and information processing system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0075501 A1
- Document ID: US20120075501A1
- Status: Published application
- Kind: A1
- Application number: 12/949,298
- Filed: 2010-11-18
- Published: 2012-03-29
- Earliest claimed: 2010-09-28
- Assignees: Nintendo Co., Ltd.
- Inventors: Yasuyuki OYAGI; Tatsuya Takadera
- Web page: https://patentgrep.com/patent/US20120075501A1

## CPC

- A63F 13/42: by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/44: involving timing of operations, e.g. performing an action within a time slot
- A63F 13/655: by importing photos, e.g. of the player
- A63F 13/79: involving player-related data, e.g. identities, accounts, preferences or play histories
- H04N 5/772: {the recording apparatus and the television camera being placed in the same enclosure}
- A63F 13/2145: the surface being also a display device, e.g. touch screens
- A63F 13/215: comprising means for detecting acoustic signals, e.g. using a microphone
- A63F 13/26: having at least one additional display device, e.g. on the game controller or outside a game booth
- A63F 13/424: involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
- A63F 13/825: Fostering virtual characters
- A63F 13/92: Video game devices specially adapted to be hand-held while playing
- A63F 2300/1075: using a touch screen
- A63F 2300/204: the platform being a handheld device
- H04N 9/8205: {involving the multiplexing of an additional signal and the colour video signal}
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A storage medium having stored an information processing program therein, an information processing apparatus, an information processing method, and an information processing system, which enable highly entertaining display in accordance with a user which performs an operation input, are provided. A characteristic of the face of a user currently using a game apparatus 10 is detected, and data indicating the detected characteristic is compared to already stored data indicating characteristics of the faces of users. If there is data indicating a closest characteristic, among the already stored data, the data is selected. When the data is selected, a motion of an object is set and controlled in accordance with a number of times which the data is selected.

## Claims

### Claim 1

A computer-readable storage medium having stored therein an information processing program which is executed by a computer of an information processing apparatus which includes an imaging device and renders one or more objects appearing in a virtual space, the information processing program causing the computer to operate as:
face characteristic data storage means for storing face characteristic data indicating a characteristic of a face;
face characteristic detection means for detecting a characteristic of a face included in a taken image which is obtained from the imaging device;
face characteristic data selection means for, when the characteristic of the face is detected by the face characteristic detection means, selecting a piece of the face characteristic data which corresponds to the characteristic of the face detected by the face characteristic detection means, from the face characteristic data storage means; and
motion control means for controlling a motion of each object on the basis of the piece of the face characteristic data selected by the face characteristic data selection means.

### Claim 2 (depends on 1)

The computer-readable storage medium according to claim 1, wherein
the information processing program further causes the computer to operate as:
selection times storage means for storing, as a number of selection times, a number of times which the face characteristic data stored in the face characteristic data storage means is selected by the face characteristic data selection means, such that the number of times corresponds to each piece of the face characteristic data; and
selection times update means for updating the number of selection times which is stored in the selection times storage means and corresponds to the piece of the face characteristic data selected by the face characteristic data selection means, and
the motion control means performs motion control of the object on the basis of the number of selection times corresponding to the piece of the face characteristic data selected by the face characteristic data selection means.

### Claim 3 (depends on 2)

The computer-readable storage medium according to claim 2, wherein the motion control means: causes the object to perform a predetermined first motion when the number of selection times corresponding to the piece of the face characteristic data selected by the face characteristic data selection means is equal to or more than a predetermined first threshold; and causes the object to perform a predetermined second motion different from the first motion, when the number of selection times corresponding to the piece of the face characteristic data selected by the face characteristic data selection means is less than the first threshold.

### Claim 4 (depends on 3)

The computer-readable storage medium according to claim 3, wherein the motion control means: causes the object to perform the second motion when the number of selection times corresponding to the piece of the face characteristic data selected by the face characteristic data selection means is equal to or more than a predetermined second threshold which is lower than the first threshold; and causes the object to perform a predetermined third motion different from the first motion and the second motion, when the number of selection times corresponding to the piece of the face characteristic data selected by the face characteristic data selection means is less than the second threshold.

### Claim 5 (depends on 4)

The computer-readable storage medium according to claim 4, wherein
the information processing program further causes the computer to operate as:
face collation means for determining whether or not the piece of the face characteristic data which corresponds to the characteristic of the face detected by the face characteristic detection means is stored in the face characteristic data storage means, and
when a determination result of the face collation means is negative, the motion control means causes the object to perform the second motion.

### Claim 6 (depends on 4)

The computer-readable storage medium according to claim 4, wherein the motion control means causes the object to perform the second motion, when the characteristic of the face is not detected by the face characteristic detection means.

### Claim 7 (depends on 5)

The computer-readable storage medium according to claim 5, wherein
the information processing program further causes the computer to operate as:
identification information storage means for storing identification information which identifies the piece of the face characteristic data selected by the face characteristic data selection means, and
the motion control means controls the motion of the object on the basis of the piece of the face characteristic data which is identified by the identification information stored in the identification information storage means.

### Claim 8 (depends on 1)

The computer-readable storage medium according to claim 1, wherein
the information processing program further causes the computer to operate as:
detection means for detecting at least one of a relative position and a relative orientation of the face, which is included in the taken image obtained from the imaging device, relative to the imaging device, and
the motion control means causes the object to perform a motion on the basis of the face characteristic data and the at least one of the relative position and the relative orientation which is detected by the detection means.

### Claim 9 (depends on 8)

The computer-readable storage medium according to claim 8, wherein
the information processing program further causes the computer to operate as:
storage means for storing the relative position and the relative orientation which are detected by the detection means, and
the motion control means controls the motion of the object on the basis of the face characteristic data and the at least one of the relative position and the relative orientation which is stored in the storage means.

### Claim 10 (depends on 2)

The computer-readable storage medium according to claim 2, wherein
the information processing program further causes the computer to operate as:
elapsed time calculation means for calculating an elapsed time from a time when the number of selection times is updated by the selection times update means last time, and
when a piece of the face characteristic data is selected when the elapsed time is equal to or more than a predetermined time, the selection times update means updates the number of selection times corresponding to the selected piece of the face characteristic data.

### Claim 11 (depends on 2)

The computer-readable storage medium according to claim 2, wherein
the selection times storage means stores the number of selection times corresponding to the face characteristic data such that the number of selection times corresponds to each object,
the selection times update means updates, for each object, the number of selection times corresponding to the piece of the face characteristic data selected by the face characteristic data selection means, and
the motion control means controls the motion of each object on the basis of the number of selection times corresponding to the piece of the face characteristic data selected by the face characteristic data selection means.

### Claim 12 (depends on 11)

The computer-readable storage medium according to claim 11, wherein the selection times update means updates the number of selection times corresponding to each object rendered so as to appear in the virtual space, among the numbers of selection times corresponding to the piece of the face characteristic data selected by the face characteristic data selection means.

### Claim 13 (depends on 2)

The computer-readable storage medium according to claim 2, wherein
the selection times storage means further stores date and time when the number of selection times is updated, as last update date and time, such that the date and time corresponds to each piece of the face characteristic data, and
when a piece of the face characteristic data is selected when an elapsed time from the last update date and time is equal to or more than a predetermined time, the selection times update means updates the number of selection times corresponding to the selected piece of the face characteristic data.

### Claim 14 (depends on 1)

The computer-readable storage medium according to claim 1, wherein
the information processing program further causes the computer to operate as:
face collation means for determining whether or not the piece of the face characteristic data which corresponds to the characteristic of the face detected by the face characteristic detection means is stored in the face characteristic data storage means; and
face characteristic data registration means for, when a determination result of the face collation means is negative, newly storing, in the face characteristic data storage means, the piece of the face characteristic data which corresponds to the characteristic of the face detected by the face characteristic detection means.

### Claim 15 (depends on 14)

The computer-readable storage medium according to claim 14, wherein
the information processing program further causes the computer to operate as:
face characteristic data deletion means for deleting any one piece of the face characteristic data if the number of pieces of the face characteristic data stored in the face characteristic data storage means is a predetermined registerable number when the face characteristic data registration means stores a new piece of the face characteristic data, and
the face characteristic data registration means stores the new piece of the face characteristic data after the one piece of the face characteristic data is deleted by the face characteristic data deletion means.

### Claim 16 (depends on 1)

The computer-readable storage medium according to claim 1, wherein
the information processing program further causes the computer to operate as:
face characteristic data update means for updating the characteristic of the face detected by the face characteristic detection means with a characteristic of a face which is indicated by the piece of the face characteristic data selected by the face characteristic data selection means.

### Claim 17 (depends on 16)

The computer-readable storage medium according to claim 16, wherein
the information processing program further causes the computer to operate as:
similarity degree calculation means for calculating a similarity degree between the characteristic of the face detected by the face characteristic detection means and the characteristic of the face which is indicated by the piece of the face characteristic data selected by the face characteristic data selection means, and
when the similarity degree is equal to or more than a predetermined similarity degree threshold, the face characteristic data update means updates the characteristic of the face which is indicated by the piece of the face characteristic data selected by the face characteristic data selection means.

### Claim 18 (depends on 1)

The computer-readable storage medium according to claim 1, wherein the information processing program is executed by the computer of the information processing apparatus which is held by a user and in which the imaging device is located at a position where the imaging device is capable of taking an image of a face of the user from a front of the user when the user looks straight at a display screen on which each object is rendered.

### Claim 19

An information processing apparatus which includes an imaging device and renders one or more objects appearing in a virtual space, the information processing apparatus comprising:
face characteristic data storage means for storing face characteristic data indicating a characteristic of a face;
face characteristic detection means for detecting a characteristic of a face included in a taken image which is obtained from the imaging device;
face characteristic data selection means for, when the characteristic of the face is detected by the face characteristic detection means, selecting a piece of the face characteristic data which corresponds to the characteristic of the face detected by the face characteristic detection means, from the face characteristic data storage means; and
motion control means for controlling a motion of each object on the basis of the piece of the face characteristic data selected by the face characteristic data selection means.

### Claim 20

An information processing method which is performed in an information processing apparatus which includes an imaging device and renders one or more objects appearing in a virtual space, the information processing method comprising:
a face characteristic data storage step of storing face characteristic data indicating a characteristic of a face;
a face characteristic detection step of detecting a characteristic of a face included in a taken image which is obtained from the imaging device;
a face characteristic data selection step of, when the characteristic of the face is detected at the face characteristic detection step, selecting a piece of the face characteristic data corresponding to the characteristic of the face detected at the face characteristic detection step, from among the face characteristic data stored at the face characteristic data storage step; and
a motion control step of controlling a motion of each object on the basis of the piece of the face characteristic data selected at the face characteristic data selection step.

### Claim 21

An information processing system which includes an imaging device and renders one or more objects appearing in a virtual space, the information processing system comprising:
face characteristic data storage means for storing face characteristic data indicating a characteristic of a face;
face characteristic detection means for detecting a characteristic of a face included in a taken image which is obtained from the imaging device;
face characteristic data selection means for, when the characteristic of the face is detected by the face characteristic detection means, selecting a piece of the face characteristic data corresponding to the characteristic of the face detected by the face characteristic detection means, from the face characteristic data storage means; and
motion control means for controlling a motion of each object on the basis of the piece of the face characteristic data selected by the face characteristic data selection means.
