# US 12,179,100 B2 · Game program, method, information processing device, and system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,179,100 B2
- Document ID: US12179100B2
- Status: Granted
- Kind: B2
- Application number: 17/954,359
- Filed: 2022-09-28
- Published: 2023-01-19
- Granted: 2024-12-31
- Earliest claimed: 2020-05-29
- Assignees: The Pokémon Company
- Applicants: The Pokémon Company
- Inventors: Yosuke Oshino
- Web page: https://patentgrep.com/patent/US12179100B2

## CPC

- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/428: involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
- A63F 13/245: specially adapted to a particular type of game, e.g. steering wheels
- A63F 13/35: Details of game servers
- A63F 13/44: involving timing of operations, e.g. performing an action within a time slot
- A63F 13/48: Starting a game, e.g. activating a game device or waiting for other players to join a multiplayer session
- A63F 13/49: Saving the game status; Pausing or ending the game
- A63F 13/533: for prompting the player, e.g. by displaying a game menu
- A63F 13/655: by importing photos, e.g. of the player
- A63F 13/69: by enabling or updating specific game elements, e.g. unlocking hidden features, items, levels or versions
- A63F 13/80: Special adaptations for executing a specific game genre or game mode
- G06T 7/248: {involving reference images or patches}
- G06V 40/165: {using facial parts and geometric relationships}
- G09B 19/24: Use of tools
- A63F 2300/308: Details of the user interface
- A63F 2300/609: for unlocking hidden game elements, e.g. features, items, levels
- A63F 2300/636: involving process of starting or resuming a game
- A63F 2300/695: Imported photos, e.g. of the player
- A63F 2300/8094: Unusual game types, e.g. virtual cooking
- G06T 2207/30201: Face
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

A tooth brushing game includes starting counting a time that defines one play of a toothbrush game in response to starting the toothbrush game; capturing an image of a user; setting an area around a mouth by detecting a face area of the user from the captured image and detecting a facial part from the detected face area; detecting movement of an object including a toothbrush and a hand in the area around the mouth; controlling the toothbrush game based on the detected movement of the object; and stopping the counting of the time when the area around the mouth is not set but not stopping the counting when the movement of the object is not detected.

## Claims

### Claim 1

A non-transitory computer-readable medium storing computer-readable instructions thereon which, when executed by a computer, cause the computer to perform a method, the method comprising:
starting counting a time that defines one play of a toothbrush game in response to starting the toothbrush game;
capturing an image of a user;
setting an area around a mouth by detecting a face area of the user from the captured image and detecting a facial part from the detected face area;
detecting movement of an object including a toothbrush and a hand in the area around the mouth;
controlling the toothbrush game based on the detected movement of the object; and
stopping the counting of the time in response to the area around the mouth not being set and not stopping the counting in response to the movement of the object not being detected,
wherein controlling the toothbrush game includes changing a rate of granting an in-game advantage based on a result of detecting the movement of the object.

### Claim 2 (depends on 1)

The non-transitory computer-readable medium according to claim 1, wherein setting the area around the mouth includes
detecting the face area from the captured image and detecting a facial part different from the mouth of the user from the detected face area to set the area around the mouth, and
wherein the area around the mouth being not set corresponds to the face area not being detected.

### Claim 3 (depends on 1)

The non-transitory computer-readable medium according to claim 1, further comprising:
in response to a state where the face area has not been detected, restarting the counting of the time.

### Claim 4 (depends on 1)

The non-transitory computer-readable medium according to claim 1, wherein setting the area around the mouth includes displaying a notification to encourage the user to show a face in response to failing to detect the face area.

### Claim 5 (depends on 1)

The non-transitory computer-readable medium according to claim 1, wherein detecting the movement of the object includes displaying a notification to encourage the user to improve tooth brushing in response to failing to detect the movement of the object or in response to the detected movement not satisfying a predetermined condition.

### Claim 6 (depends on 1)

The non-transitory computer-readable medium according to claim 1, further comprising:
comparing a past image of the object with a current image; and
displaying a notification to encourage the user to replace the toothbrush in response to the images not matching.

### Claim 7 (depends on 1)

The non-transitory computer-readable medium according to claim 1, wherein controlling the toothbrush game includes increasing the rate of granting in response to the movement of the object being continuously detected, and reducing the rate of granting in response to the movement of the object not being continuously detected.

### Claim 8 (depends on 1)

The non-transitory computer-readable medium according to claim 1, further comprising launching the toothbrush game at a preset time.

### Claim 9 (depends on 2)

The non-transitory computer-readable medium according to claim 2, further comprising:
superimposing, in response to a head area being detected as a facial part, a headgear designed with a face of a game character on the head area; and
changing facial expression of the headgear based on a result of detecting the movement of the object.

### Claim 10 (depends on 1)

The non-transitory computer-readable medium according to claim 1, further comprising:
determining whether one of a plurality of predetermined conditions for the toothbrush game is satisfied;
displaying a notification associated with the satisfied condition on a game screen;
capturing a game screen with the notification; and
displaying at least one screenshot created by capturing the game screen in response to one play of the toothbrush game ending.

### Claim 11 (depends on 1)

The non-transitory computer-readable medium according to claim 1, further comprising:
determining whether one play of the toothbrush game has ended;
making an evaluation of tooth brushing for one play; and
notifying the user of a comment about the tooth brushing based on the evaluation.

### Claim 12 (depends on 1)

The non-transitory computer-readable medium according to claim 1, further comprising:
transmitting information on the toothbrush game to a server.

### Claim 13 (depends on 12)

The non-transitory computer-readable medium according to claim 12, wherein the information on the toothbrush game includes an age of the user, a time limit for one play, a time for notifying start of the toothbrush game, a toothbrush replacement date, a toothbrush game execution date, a start time of the toothbrush game, and an evaluation of the toothbrush game.

### Claim 14

A method, comprising:
starting counting a time that defines one play of a toothbrush game in response to starting the toothbrush game;
capturing an image of a user;
setting an area around a mouth by detecting a face area of the user from the captured image and detecting a facial part from the detected face area;
detecting movement of an object including a toothbrush and a hand in the area around the mouth;
controlling the toothbrush game based on the detected movement of the object; and
stopping the counting of the time in response to the area around the mouth not being set and not stopping the counting in response to the movement of the object not being detected,
wherein controlling the toothbrush game includes changing a rate of granting an in-game advantage based on a result of detecting the movement of the object.

### Claim 15

An information processing device, comprising:
processing circuitry, wherein the processing circuitry is configured to
start counting a time that defines one play of a toothbrush game in response to starting the toothbrush game;
capture an image of a user;
set an area around a mouth, wherein the processing circuitry for setting the area around the mouth is further configured to detect a face area of the user from the captured image and detect a facial part from the detected face area;
detect movement of an object including a toothbrush and a hand in the area around the mouth;
control the toothbrush game based on the detected movement of the object;
stop the counting of the time in response to the area around the mouth not being set and not stopping the counting in response to the movement of the object not being detected; and
change a rate of granting an in-game advantage based on a result of detecting the movement of the object.

### Claim 16

A system comprising a plurality of information processing devices, wherein each of the plurality of information processing device includes processing circuitry, the processing circuitry being configured to:
start counting a time that defines one play of a toothbrush game in response to starting the toothbrush game;
capture an image of a user;
set an area around a mouth, wherein the processing circuitry for setting the area around the mouth is further configured to detect a face area of the user from the captured image and detect a facial part from the detected face area;
detect movement of an object including a toothbrush and a hand in the area around the mouth;
control the toothbrush game based on the detected movement of the object;
stop the counting of the time in response to the area around the mouth not being set and not stopping the counting in response to the movement of the object not being detected; and
change a rate of granting an in-game advantage based on a result of detecting the movement of the object.