# US 12,001,646 B2 · Computer-implemented method, computer, and program for rendering a three-dimensional object in a virtual reality space

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,001,646 B2
- Document ID: US12001646B2
- Status: Granted
- Kind: B2
- Application number: 18/153,906
- Filed: 2023-01-12
- Published: 2023-05-11
- Granted: 2024-06-04
- Earliest claimed: 2020-07-27
- Assignees: Wacom Co., Ltd.
- Applicants: Wacom Co., Ltd.
- Inventors: Hiroshi Fujioka; Naoya Nishizawa; Kenton J. Loftus; Milen Dimitrov Metodiev; Markus Weber; Anthony Ashton
- Web page: https://patentgrep.com/patent/US12001646B2

## CPC

- G06F 3/046: by electromagnetic means
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves}
- G06F 3/03545: {Pens or stylus}
- G06F 3/0421: {by interrupting or reflecting a light beam, e.g. optical touch-screen}
- G06F 3/044: by capacitive means
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06F 2203/04101: 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
- G06F 2203/04106: Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
- G06T 2219/2004: Aligning objects, relative positioning of parts
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

According to one aspect, it becomes possible to easily modify a 3D object which is displayed in a virtual reality space. A method is performed by a computer configured to be communicable with a position detection device that includes a drawing surface and that, in operation, detects a position of an electronic pen on the drawing surface. The method includes rendering, in a virtual reality space, a first object that is a 3D object, rendering, near the first object, a display surface that is a 3D object, rendering, on the display surface, a 3D line that is a 3D object generated based on the position of the electronic pen on the drawing surface, wherein the position of the electronic pen is detected by the position detection device, and outputting the first object, the display surface, and the 3D line, which are the 3D objects, to a display.

## Claims

### Claim 1

A method performed by a computer configured to be communicable with a position detection device that includes a drawing surface and that, in operation, detects a position of an electronic pen on the drawing surface, the method comprising:
rendering, in a virtual reality space, a first object that is a three-dimensional object;
rendering, near the first object, a display surface that is a three-dimensional object;
rendering, on the display surface, a three-dimensional line that is a three-dimensional object generated based on the position of the electronic pen on the drawing surface, wherein the position of the electronic pen is detected by the position detection device;
outputting the first object, the display surface, and the three-dimensional line, which are the three-dimensional objects, to a display;
obtaining first correspondence information indicating a correspondence between a first coordinate system defined on the drawing surface and a second coordinate system defined on the display surface;
obtaining second correspondence information indicating a correspondence between the second coordinate system and a third coordinate system indicating the virtual reality space; and
converting the position of the electronic pen into a position in the third coordinate system based on the first correspondence information and the second correspondence information.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising:
causing the three-dimensional line to move according to an operation of a three-dimensional controller.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the three-dimensional controller is the electronic pen.

### Claim 4 (depends on 3)

The method according to claim 3, further comprising:
further detecting a position and an orientation of the electronic pen based on light reception level information from a position sensor attached to the electronic pen; and
causing the three-dimensional line to move based on the detected position and orientation of the electronic pen.

### Claim 5 (depends on 3)

The method according to claim 3, further comprising:
detecting an operation state of an operation unit included in the electronic pen; and
causing the three-dimensional line to move based on the detected operation state of the operation unit.

### Claim 6 (depends on 1)

The method according to claim 1, further comprising:
updating viewpoint information indicating a viewpoint of a user in the virtual reality space, based on a position and an orientation of a virtual reality display in the third coordinate system indicating the virtual reality space; and
performing rendering based on the viewpoint information.

### Claim 7

A computer configured to be communicable with a position detection device that includes a drawing surface and that, in operation, detects a position of an electronic pen on the drawing surface, the computer comprising:
a control unit,
wherein the control unit, in operation,
renders, in a virtual reality space, a first object that is a three-dimensional object,
renders, near the first object, a display surface that is a three-dimensional object,
renders, on the display surface, a three-dimensional line that is a three-dimensional object generated based on the position of the electronic pen on the drawing surface, wherein the position of the electronic pen is detected by the position detection device,
outputs the first object, the display surface, and the three-dimensional line which are the three-dimensional objects, to a display;
obtains first correspondence information indicating a correspondence between a first coordinate system defined on the drawing surface and a second coordinate system defined on the display surface;
obtains second correspondence information indicating a correspondence between the second coordinate system and a third coordinate system indicating the virtual reality space; and
converts the position of the electronic pen into a position in the third coordinate system based on the first correspondence information and the second correspondence information.

### Claim 8 (depends on 7)

The computer according to claim 7, wherein the three-dimensional line is caused to move according to an operation of a three-dimensional controller.

### Claim 9 (depends on 8)

The computer according to claim 8, wherein the three-dimensional controller is the electronic pen.

### Claim 10

A non-transitory computer-readable medium including a computer-executable program for a computer configured to be communicable with a position detection device that includes a drawing surface and that, in operation, detects a position of an electronic pen on the drawing surface, the program causing the computer to perform processing of:
rendering, in a virtual reality space, a first object that is a three-dimensional object;
rendering, near the first object, a display surface that is a three-dimensional object;
rendering, on the display surface, a three-dimensional line that is a three-dimensional object generated based on the position of the electronic pen on the drawing surface, wherein the position of the electronic pen is detected by the position detection device;
outputting the first object, the display surface, and the three-dimensional line which are the three-dimensional objects, to a display;
obtaining first correspondence information indicating a correspondence between a first coordinate system defined on the drawing surface and a second coordinate system defined on the display surface;
obtaining second correspondence information indicating a correspondence between the second coordinate system and a third coordinate system indicating the virtual reality space; and
converting the position of the electronic pen into a position in the third coordinate system based on the first correspondence information and the second correspondence information.

### Claim 11 (depends on 10)

The medium according to claim 10, wherein the program causes the computer to perform processing of causing the three-dimensional line to move according to an operation of a three-dimensional controller.

### Claim 12 (depends on 11)

The medium according to claim 11, wherein the three-dimensional controller is the electronic pen.