# US 2019/0094534 A1 · Information processing apparatus, information processing system, and non-transitory computer readable medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0094534 A1
- Document ID: US20190094534A1
- Status: Published application
- Kind: A1
- Application number: 16/130,195
- Filed: 2018-09-13
- Published: 2019-03-28
- Earliest claimed: 2017-09-28
- Assignees: Fuji Xerox Co., Ltd.
- Applicants: Fuji Xerox Co., Ltd.
- Inventors: Yusuke YAMAURA; Seiya INAGI; Kazunari HASHIMOTO; Hidetaka IZUMO; Tadaaki SATO; Teppei AOKI; Daisuke YASUOKA; Hiroshi UMEMOTO
- Web page: https://patentgrep.com/patent/US20190094534A1

## CPC

- G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)}
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0237: {using prediction or retrieval techniques}
- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 16/951: Indexing; Web crawling techniques
- G06F 40/166: Editing, e.g. inserting or deleting
- G06F 40/169: Annotation, e.g. comment data or footnotes
- G06F 40/171: by use of digital ink
- G06V 20/20: in augmented reality scenes
- G06V 40/107: {Static hand or arm}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An information processing apparatus includes a detector and a command unit. The detector detects movement of a user using a display device based on an image photographed by the display device that displays a virtual-space image in such a manner as to be overlapped with a real space and that has a photographing function. The command unit commands the display device to display, as the virtual-space image, relevant information related to input information input to an input target at a position near the input target based on the detected movement of the user.

## Claims

### Claim 1

An information processing apparatus comprising:
a detector that detects movement of a user using a display device based on an image photographed by the display device that displays a virtual-space image in such a manner as to be overlapped with a real space and that has a photographing function; and
a command unit that commands the display device to display, as the virtual-space image, relevant information related to input information input to an input target at a position near the input target based on the detected movement of the user.

### Claim 2 (depends on 1)

The information processing apparatus according to claim 1,
wherein the relevant information is a ruled line.

### Claim 3 (depends on 2)

The information processing apparatus according to claim 2,
wherein the command unit makes a command to change the ruled line in accordance with a size of the input information.

### Claim 4 (depends on 1)

The information processing apparatus according to claim 1,
wherein the detector detects movement of a finger or hand of the user holding a writing unit that is used for inputting the input information to the input target, and
wherein the command unit makes a command to display the relevant information in accordance with the movement of the finger or hand of the user.

### Claim 5 (depends on 1)

The information processing apparatus according to claim 1,
wherein the command unit makes a command to display the relevant information on a surface, at a front side, or at an adjacent position of the input target.

### Claim 6 (depends on 1)

The information processing apparatus according to claim 1,
wherein the relevant information is support information that supports input of the input information to the input target.

### Claim 7 (depends on 6)

The information processing apparatus according to claim 6,
wherein the support information is a word candidate in response to a detected handwriting operation performed in midair by the user.

### Claim 8 (depends on 6)

The information processing apparatus according to claim 6,
wherein the support information is information indicating a location of a typographical error in a case where the input information includes the typographical error.

### Claim 9 (depends on 1)

The information processing apparatus according to claim 1,
wherein the relevant information is support information that is a file list or a web search result related to the input information.

### Claim 10 (depends on 1)

The information processing apparatus according to claim 1,
wherein the command unit makes a command to display the relevant information including the input information at a position located in front of the input target, and
wherein the information processing apparatus further comprises a processor that executes processing based on an operation performed by the user with respect to the input information.

### Claim 11 (depends on 1)

The information processing apparatus according to claim 1,
wherein the display device includes a sound input unit,
wherein the information processing apparatus further comprises a converter that extracts a portion from a voice acquired from the sound input unit and converts the portion into text, and
wherein the command unit makes a command to display the converted text as the relevant information.

### Claim 12

An information processing system comprising:
a display device that displays a virtual-space image in such a manner as to be overlapped with a real space and that has a photographing function;
a detector that detects movement of a user using the display device based on an image photographed by the display device; and
a command unit that commands the display device to display, as the virtual-space image, relevant information related to input information input to an input target at a position near the input target based on the detected movement of the user.

### Claim 13

A non-transitory computer readable medium storing a program causing a computer to execute a process for processing information, the process comprising:
detecting movement of a user using a display device based on an image photographed by the display device that displays a virtual-space image in such a manner as to be overlapped with a real space and that has a photographing function; and
commanding the display device to display, as the virtual-space image, relevant information related to input information input to an input target at a position near the input target based on the detected movement of the user.
