# US 2018/0025531 A1 · Method of providing virtual space, method of providing virtual experience, program and recording medium therefor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0025531 A1
- Document ID: US20180025531A1
- Status: Published application
- Kind: A1
- Application number: 15/654,652
- Filed: 2017-07-19
- Published: 2018-01-25
- Earliest claimed: 2016-07-20
- Assignees: Colopl, Inc.
- Applicants: COLOPL, Inc.
- Inventors: Shuhei TERAHATA
- Web page: https://patentgrep.com/patent/US20180025531A1

## CPC

- G06T 15/20: Perspective computation
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G06T 7/12: Edge-based segmentation
- G06T 7/13: Edge detection
- G06T 2219/2016: Rotation, translation, scaling
- H04N 13/344: with head-mounted left-right displays
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method of providing a virtual experience to a user includes identifying a plurality of virtual objects. The method further includes detecting a position of a part of the user's body other than the user's head. The method further includes detecting a reference line of sight of the user. The method further includes setting an extension direction for a first virtual object of the plurality of virtual objects based on a direction of the reference line of sight. The method further includes setting a region for a first virtual object of the plurality of virtual objects, wherein the region comprises a part extending in the extension direction. The method further includes determining whether the first virtual object and a virtual representation of the part of the body have touched based on a positional relationship between the region and a position of the virtual representation of the part of the body.

## Claims

### Claim 1

A method comprising:
defining a virtual space including a virtual camera, a plurality of virtual objects and an operation target object;
detecting movement of a part of a user's body other than the user's head;
moving the operation target object in synchronization with the detected movement of a part of a user's body other than the user's head;
determining a reference line of sight of the user in the virtual space;
defining a field-of-view region based on the reference line of sight;
setting an extension direction for each virtual object of the plurality of virtual objects based on a position of the virtual camera or a direction of the reference line of sight;
setting a region for determining a touch between the operation target object and a first virtual object of the plurality of virtual objects based on a corresponding extension direction for the first virtual object; and
determining whether the first virtual object touches the operation target object based on a positional relationship between the region and the operation target object.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the extension direction for each virtual object of the plurality of virtual objects is substantially parallel to the reference line of sight.

### Claim 3 (depends on 1)

The method according to claim 1, further comprising:
defining a first reference vector for the extension direction for the first virtual object,
wherein the setting the region comprises:
rotating the first reference vector based on the position of the virtual camera or the direction of the reference line of sight, and
setting the region based on the rotated first reference vector.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the setting the region comprises rotating the first reference vector such that the first reference vector points parallel to and in a direction opposite to the direction of the reference line of sight.

### Claim 5 (depends on 3)

The method according to claim 3, wherein the setting the region further comprises rotating the first reference vector such that the reference vector points:
parallel to and in a direction opposite to a direction from the position of the virtual camera toward a position of the first virtual object, or
toward a position identified based on the position of the first virtual object.

### Claim 6 (depends on 1)

The method according to claim 1, further comprising extending the region in response to a change of the position of the virtual camera or a change of the direction of the reference line of sight.

### Claim 7

A method of providing a virtual experience to a user comprising:
identifying a plurality of virtual objects for display in a field of view overlapping with real objects;
detecting a position of a part of the user's body other than the user's head;
detecting a reference line of sight of the user;
setting an extension direction for a first virtual object of the plurality of virtual objects based on a direction of the reference line of sight;
setting a region for a first virtual object of the plurality of virtual objects, wherein the region a part extending in the extension direction; and
determining whether the first virtual object and the part of the body have touched based on a positional relationship between the region and a position of the part of the body.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the extension direction extends opposite to the direction of the reference line of sight.

### Claim 9 (depends on 7)

The method according to claim 7, further comprising defining a reference vector for the extension direction,
wherein the setting the region comprises:
rotating the reference vector based on the direction of the reference line of sight, and
setting the region based on the rotated reference vector.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the setting the region comprises rotating the reference vector such that the reference vector points parallel to and in a direction opposite to the direction of the reference line of sight.

### Claim 11 (depends on 7)

The method according to claim 7, further comprising extending the part along the extension direction in response to a change in the direction of the reference line of sight.

### Claim 12 (depends on 1)

The method according to claim 1, further comprising:
determining whether the first virtual object overlaps with a second virtual object of the plurality of virtual objects in the virtual space;
setting a second region for the second virtual object in response to a determination that the first virtual object overlaps the second virtual object; and
determining whether the second virtual object and the operation target object have touched based on a positional relationship between the second region and the operation target object.

### Claim 13 (depends on 12)

The method according to claim 12, wherein the setting the second region comprises:
defining a first line along a first edge of the first virtual object; and
setting the second region to be located on a single side of the first line.

### Claim 14 (depends on 12)

The method according to claim 12, wherein the setting the region comprises:
defining a first line along a first edge of the first virtual object;
defining a second line along a second edge of the first virtual object, wherein the second edge is angled with respect to the first edge; and
setting the region to be bounded by the first line and the second line.

### Claim 15 (depends on 12)

The method of claim 12, further comprising obscuring a portion of the second virtual object in response to a determination that the first virtual object overlaps with the second virtual object.

### Claim 16

A system for providing a virtual experience to a user comprising:
a head mounted display (HMD);
a control device connected to the HMD, wherein the control device comprises at least one processor, and the control device is configured to:
identify a plurality of virtual objects in a virtual space;
detect a position of a part of the user's body other than the user's head;
detect a reference line of sight of the user;
set an extension direction for a first virtual object of the plurality of virtual objects based on a direction of the reference line of sight;
set a region for a first virtual object of the plurality of virtual objects, wherein the region comprises a part extending in the extension direction; and
determine whether the first virtual object and a virtual representation of the part of the body have touched based on a positional relationship between the region and a position of the virtual representation of the part of the body.

### Claim 17 (depends on 16)

The system according to claim 16, further comprising a controller, wherein the control device is configured to detect the position of the part of the user's body based on a detected position of the controller.

### Claim 18 (depends on 16)

The system according to claim 16, wherein the control device is configured to:
define a reference vector for the extension direction,
rotate the reference vector based on the direction of the reference line of sight, and
set the region based on the rotated reference vector.

### Claim 19 (depends on 18)

The system according to claim 18, wherein the control device is configured to rotate the reference vector such that the reference vector points parallel to and in a direction opposite to the direction of the reference line of sight.

### Claim 20 (depends on 16)

The system according to claim 16, wherein the control device is configured to extend the part along the extension direction in response to a change in the direction of the reference line of sight.