# US 12,181,670 B2 · Image generation device, image display system, and information presentation method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,181,670 B2
- Document ID: US12181670B2
- Status: Granted
- Kind: B2
- Application number: 18/210,518
- Filed: 2023-06-15
- Published: 2023-10-12
- Granted: 2024-12-31
- Earliest claimed: 2019-04-23
- Assignees: Sony Interactive Entertainment Inc.
- Applicants: Sony Interactive Entertainment Inc.
- Inventors: Junichi Muramoto
- Web page: https://patentgrep.com/patent/US12181670B2

## CPC

- G02B 27/017: {Head mounted}
- A63F 13/211: using inertial sensors, e.g. accelerometers or gyroscopes
- A63F 13/212: using sensors worn by the player, e.g. for measuring heart beat or leg activity
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/25: Output arrangements for video game devices
- A63F 13/40: Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
- A63F 13/52: involving aspects of the displayed game scene
- A63F 13/537: using indicators, e.g. showing the condition of a game character on screen
- A63F 13/56: Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
- A63F 13/65: automatically by game devices or servers from real world data, e.g. measurement in live racing competition
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- A63F 2300/8082: Virtual reality
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Disclosed herein is an image generation device including: a space construction section that constructs a virtual world to be displayed on a head-mounted display; an image generation section that generates, based on a position and a posture of a head of a player wearing the head-mounted display, a display image representing the virtual world in a field of view corresponding to a point of view of the player, and causes the head-mounted display to display the generated display image; and a visitor information acquisition section that acquires information regarding presence of a visitor without a head-mounted display in a space where the player is able to move. While the visitor is present, the space construction section displays an object indicating the presence of the visitor at a corresponding position in the virtual world.

## Claims

### Claim 1

An image generation device comprising:
circuitry configured to
construct a virtual world to be displayed on a head-mounted display;
generate, based on a position and a posture of a head of a player wearing the head-mounted display, a display image representing the virtual world in a field of view corresponding to a point of view of the player, and cause the head-mounted display to display the generated display image;
acquire information regarding presence of an obstacle which is not related to the player in a space where the player is located; and
cause the head-mounted display to display a first object in the virtual world corresponding to the obstacle while the obstacle is present within the space.

### Claim 2 (depends on 1)

The image generation device according to claim 1,
wherein the obstacle is able to move in the space,
wherein the object is displayed at a position in the virtual world corresponding to a location of the obstacle in the space, and
wherein, when the object is not within the space, the first object is not displayed.

### Claim 3 (depends on 1)

The image generation device according to claim 1, wherein
the circuitry is configured to acquire information regarding a position of the obstacle in accordance with a result of detection by a sensor configured to detect an ultrasonic wave.

### Claim 4 (depends on 1)

The image generation device according to claim 1, wherein
the circuitry is configured to acquire information regarding a position of the obstacle in accordance with a result of detection by a sensor configured to detect light having a predetermined color.

### Claim 5 (depends on 1)

The image generation device according to claim 1, wherein the circuitry is configured to:
predict a direction of travel of the obstacle;
cause the head-mounted display to display a second object moving in addition to the first object in accordance with a movement of the obstacle as the object indicating the presence of the obstacle; and
label the second object with the direction of travel.

### Claim 6 (depends on 1)

The image generation device according to claim 1, wherein the circuitry is configured to
detect the obstacle's entry into and exit from a stay area preset in the space; and
while the obstacle is present in the stay area, cause the head-mounted display to display an object surrounding a region corresponding to the stay area as the object indicating the presence of the obstacle, wherein
the stay area is a sub-area of the space represented in the virtual space at a corresponding location.

### Claim 7 (depends on 6)

The image generation device according to claim 6, wherein
the circuitry is configured to detect the obstacle's entry into and exit from the stay area through an input device operated by the obstacle or a person outside the space.

### Claim 8 (depends on 6)

The image generation device according to claim 6, wherein, when an input operation is performed by the player to indicate that the player wants to take shelter, the circuitry is configured to
cause the head-mounted display to display a pictorial figure for guiding the player to the stay area in the display image on a head-mounted display worn by the player; and
cause the head-mounted display to display the object surrounding the region corresponding to the stay area even while the player is present in the stay area.

### Claim 9 (depends on 1)

The image generation device according to claim 1, wherein the circuitry is configured to:
acquire information regarding the position and the posture of the player;
detect a state where the posture indicated by the acquired information is outside a predefined normal range;
cause the head-mounted display to display a second object representing the posture of the player at a corresponding position in the virtual world; and
when the posture is outside the normal range, lock the second object and change a mode of the locked object.

### Claim 10

An image display system comprising:
an image generation device including circuitry configured to
construct a virtual world to be displayed on a head-mounted display;
generate, based on a position and a posture of a head of a player wearing the head-mounted display, a display image representing the virtual world in a field of view corresponding to a point of view of the player, and cause the head-mounted display to display the generated display image;
acquire information regarding presence of an obstacle which is not related to the player in a space where the player is able to move;
cause the head-mounted display to display an object in the virtual world corresponding to the obstacle while the obstacle is present in the space;
the head-mounted display that displays an image output by the circuitry; and
a sensor located at a fixed position that detects the obstacle and transmits data on the obstacle to the circuitry.

### Claim 11

An image display system comprising:
circuitry configured to
construct a virtual world to be displayed on a head-mounted display;
generate, based on a position and a posture of a head of a player wearing the head-mounted display, a display image representing the virtual world in a field of view corresponding to a point of view of the player, and cause the head-mounted display to display the generated display image;
acquire information regarding presence of an obstacle which is not related to the player in a space where the player is able to move;
cause the head-mounted display to display an object in the virtual world corresponding to the obstacle while the obstacle is present;
detect the obstacle's entry into and exit from a stay area preset in the space, wherein the stay area is a sub-area of the space represented in the virtual space at a corresponding location; and
cause the head-mounted display to display an object surrounding a region corresponding to the stay area as the object indicating the presence of the obstacle while the obstacle is present in the stay area;
the head-mounted display configured to display an image outputted from the image generation device; and
an electronic device configured to input an operation indicative of the obstacle's entry into and exit from the stay area, and transmit information regarding the obstacle's entry into and exit from the stay area to the circuitry.

### Claim 12

An information presentation method of an image generation device, comprising:
constructing a virtual world to be displayed on a head-mounted display;
generating, based on a position and a posture of a head of a player wearing the head-mounted display, a display image representing the virtual world in a field of view corresponding to a point of view of the player, and causing the head-mounted display to display the generated display image;
acquiring information regarding presence of an obstacle which is not related to the player in a space where the player is able to move; and
cause the head-mounted display to display an object in the virtual world corresponding indicating the presence of the obstacle while the obstacle is present within the area.

### Claim 13

A non-transitory computer readable storage medium having stored thereon a program for causing an image generation device execute processing comprising:
constructing a virtual world to be displayed on a head-mounted display;
generating, based on a position and a posture of a head of a player wearing the head-mounted display, a display image representing the virtual world in a field of view corresponding to a point of view of the player, and causing the head-mounted display to display the generated display image;
acquiring information regarding presence of an obstacle which is not related to the player in a space where the player is located; and
causing the head-mounted display to display a first object in the virtual world corresponding to the obstacle while the obstacle is present within the space.