# US 9,873,049 B2 · Camera based safety mechanisms for users of head mounted displays

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,873,049 B2
- Document ID: US9873049B2
- Status: Granted
- Kind: B2
- Application number: 15/473,469
- Filed: 2017-03-29
- Published: 2017-07-20
- Granted: 2018-01-23
- Earliest claimed: 2013-09-30
- Assignees: Sony Interactive Entertainment Inc.
- Applicants: Sony Interactive Entertainment Inc.
- Inventors: Jeffrey Roger Stafford
- Web page: https://patentgrep.com/patent/US9873049B2

## CPC

- A63F 13/537: using indicators, e.g. showing the condition of a game character on screen
- A63F 13/211: using inertial sensors, e.g. accelerometers or gyroscopes
- A63F 13/212: using sensors worn by the player, e.g. for measuring heart beat or leg activity
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/25: Output arrangements for video game devices
- A63F 13/26: having at least one additional display device, e.g. on the game controller or outside a game booth
- A63F 13/49: Saving the game status; Pausing or ending the game
- A63F 13/5255: according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
- A63F 13/533: for prompting the player, e.g. by displaying a game menu
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/16: Sound input; Sound output (speech processing G10L)
- A63F 2300/306: for displaying a marker associated to an object or location in the game field
- A63F 2300/538: for performing operations on behalf of the game client, e.g. rendering
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Methods and systems for warning a user of a head mounted display (HMD) during game play of a video game. A game is executed causing interactive scenes of the game to be transmitted for rendering on a display portion of the HMD. Coordinates of the HMD are determined in a three-dimensional space of a real-world environment in which the user wearing the HMD is present, to identify a current position of the user. A movement of the HMD is determined during execution of the game by identifying a change in one or more coordinates of the HMD in the three-dimensional space. When it is determined that the user is approaching a boundary of an interaction space in a real-world environment, a warning signal is conveyed to the HMD to indicate proximity of the user to the boundary of the interaction space.

## Claims

### Claim 1

A method, comprising:
executing a game on a client device, the execution causing interactive scenes of the game to be transmitted to a head mounted display (HMD) for rendering on a display portion, the HMD when worn by a user is configured to block a view to a real-world environment, the HMD being in wireless communication with the client device;
determining coordinates of the HMD in a three-dimensional space of a real-world environment in which the user is present, the coordinates used to identify a current position of the user wearing the HMD;
monitoring a movement of the HMD during execution of the game, the movement of the HMD determined by identifying a change in one or more coordinates of the HMD in the three-dimensional space;
evaluating the change in the one or more coordinates of the HMD worn by the user to determine when the user is approaching a boundary of an interaction space defined in the real-world environment, wherein the interaction space is an area in the real-world environment in which the user is present, the boundary is defined as an edge of the interaction space; and
conveying a warning signal for rendering on the HMD, wherein the warning signal is indicative of a proximity of the user to the boundary of the interaction space, the warning signal includes an image of an outline representing the boundary of the interaction space for presenting with at least a part of the interactive scenes of the game,
wherein the method operations are performed by a processor.

### Claim 2 (depends on 1)

The method of claim 1, further includes adjusting an intensity of the warning signal based on the evaluating, wherein the intensity is adjusted to increase as the user continues to approach the boundary of the interaction space and to decrease as the user moves away from the boundary of the interaction space toward a center of the interaction space.

### Claim 3 (depends on 1)

The method of claim 1, wherein the warning signal is further provided in an audio format or a textual format or a haptic format, or any combinations thereof.

### Claim 4 (depends on 1)

The method of claim 1, wherein the warning signal further includes instructions to direct the user away from the boundary of the interaction space in the real-world environment and toward a center of the interaction space.

### Claim 5 (depends on 1)

The method of claim 1, further includes detecting one or more real-world objects disposed along the boundary, the warning signal providing an image of the one or more real-world objects that the user is approaching.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining coordinates further includes,
receiving images of the HMD in the three-dimensional space of the real-world environment captured by one or more cameras that are external to the HMD; and
computing the coordinates of the HMD in the three-dimensional space.

### Claim 7 (depends on 6)

The method of claim 6, further includes receiving secondary images of the HMD in the three-dimensional space of the real-world environment captured by one or more forward facing cameras coupled to the HMD, while the game is being executed, the secondary images analyzed to verify the coordinates of the HMD in the three-dimensional space.

### Claim 8 (depends on 1)

The method of claim 1, wherein the image of an outline representing the boundary is presented in a form of a grid.

### Claim 9

A method, comprising:
receiving interactive scenes of a game at a head mounted display (HMD) for rendering on a display screen of the HMD, the game being executed on a client device, the HMD when worn by a user is configured to block a view to a real-world environment, data for the interactive scenes being communicated by the client device to the HMD using wireless communication;
capturing images of an interaction space of a real-world environment in a vicinity of the user using a forward facing camera coupled to the HMD, the images being captured while the user is interacting with the game that is providing the interactive scenes;
forwarding the images to the client device for processing to determine coordinates of the HMD in the interaction space, wherein the coordinates identify a current position of the HMD and of the user wearing the HMD;
capturing additional images of the interaction space of the real-world environment, the additional images forwarded to the client device for evaluating a change in one or more coordinates of the HMD during execution of the game, wherein the change in the coordinates identifies movement of the user in the interaction space of the real-world environment; and
receiving a warning signal from the client device for rendering on the display portion of the HMD, when it is determined that the user is approaching a boundary of the interaction space, the warning signal is indicative of proximity of the user to the boundary of the interaction space, wherein the warning signal received from the client device includes an image of an outline representing the boundary of the interaction space for rendering with at least a part of the interactive scenes of the game,
wherein operations of the method are performed by a processor of the HMD.

### Claim 10 (depends on 9)

The method of claim 9, further includes,
receiving secondary images of the interaction space captured by one or more cameras that are external to the HMD; and
forwarding the secondary images to the client device for verifying the movement of the user.

### Claim 11 (depends on 9)

The method of claim 9, wherein rendering the warning signal includes dynamically adjusting an intensity of the image of the outline representing the boundary based on the current position of the user, the warning signal includes information for dynamically adjusting the intensity of the image of the outline, such that the intensity of the image of the outline is set to increase as the user continues to approach the boundary of the interaction space and decrease as the user moves away from the boundary of the interaction space.

### Claim 12 (depends on 9)

The method of claim 9, wherein the warning signal is further provided in audio format or haptic format or textual format, or any combinations thereof, and wherein information provided with the warning signal causes a dynamic adjustment to an intensity of the warning signal, such that the intensity of the warning signal rendered at the HMD is set to increase when the user continues to approach the boundary of the interaction space and decrease when the user move away from the boundary toward a center of the interaction space.