# US 2026/0267405 A1 · Portable information terminal and method for displaying virtual reality object

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0267405 A1
- Document ID: US20260267405A1
- Status: Published application
- Kind: A1
- Application number: 19/164,878
- Filed: 2023-03-17
- Published: 2026-09-10
- Earliest claimed: 2023-03-17
- Assignees: Maxell, Ltd.
- Applicants: Maxell, Ltd.
- Inventors: Mayumi NAKADE; Yasunobu HASHIMOTO; Hitoshi AKIYAMA; Kazuyuki TAKIZAWA; Tatsuro NAKAMURA
- Web page: https://patentgrep.com/patent/US20260267405A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 1/00: General purpose image data processing
- G06T 7/74: {involving reference images or patches}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 20/20: in augmented reality scenes
- G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
- G09G 5/00: Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators (image data processing or generation, in general G06T)
- H04N 5/66: Transforming electric information into light information (scanning details H04N3/00)
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- G06V 10/772: Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 30/1914: {Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries, e.g. user dictionaries}
- G06V 40/174: {Facial expression recognition}
- G06V 40/18: Eye characteristics, e.g. of the iris
- G06V 2201/131: Book
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A portable information terminal is configured to: carry out an image analysis on a captured image captured by a camera; based on description information included in a document captured in the captured image, extract appearing object information including attribution information about an appearing object which appears in the document; store the appearing object information in a memory; acquire line-of-sight information obtained and output by a line-of-sight detection sensor that has detected a line of sight of a user; identify a portion on which a viewpoint of the user is present; generate a virtual reality object including a description about the appearing object corresponding to the portion on which the viewpoint of the user is present based on the appearing object information; and display the virtual reality object in a display area of a display, which corresponds to a periphery of the document.

## Claims

### Claim 1

A portable information terminal, comprising:
a camera that captures an image of a field of view of a user and generates a captured image;
a display;
a memory;
a line-of-sight detection sensor that detects a line of sight of the user and outputs line-of-sight information; and
a processor,
the processor being configured to:
carry out an image analysis on the captured image to extract appearing object information including attribution information about an appearing object which appears in a document included in the captured image;
store the appearing object information in the memory;
acquire the line-of-sight information to identify a portion on which a viewpoint of the user is present;
generate a virtual reality object including a description about the appearing object corresponding to the portion on which the viewpoint of the user is present based on the appearing object information; and
display the virtual reality object in a display area of the display, which corresponds to a periphery of the document.

### Claim 2 (depends on 1)

The portable information terminal according to claim 1, wherein
in a case where the captured image includes a content that comes after a portion where the viewpoint of the user is present, the processor does not reflect the appearing object information extracted from the content in display of the virtual reality object.

### Claim 3 (depends on 1)

The portable information terminal according to claim 1, wherein
the virtual reality object is a character image corresponding to the appearing object.

### Claim 4 (depends on 3)

The portable information terminal according to claim 3, wherein
the processor is configured to:
store a reference model serving as a basis of the character image in advance; and
process a shape of the reference model by reflecting the appearing object information corresponding to the portion on which the viewpoint of the user is present to generate the character image.

### Claim 5 (depends on 4)

The portable information terminal according to claim 4, wherein
the appearing object information includes age information about the appearing object, and
the processor processes the shape of the reference model based on the age information to generate the character image.

### Claim 6 (depends on 4)

The portable information terminal according to claim 4, wherein
the appearing object information includes physical constitution information about the appearing object, and
the processor processes the shape of the reference model based on the physical constitution information to generate the character image.

### Claim 7 (depends on 1)

The portable information terminal according to claim 1, wherein
based on the line-of-sight information, the processor displays the virtual reality object upon determining that the user is watching the display area of the display, which corresponds to the periphery of the document, and hides the virtual reality object being displayed upon determining that the user is not watching the display area of the display, which corresponds to the periphery of the document.

### Claim 8 (depends on 1)

The portable information terminal according to claim 1, wherein
the virtual reality object is a correlation chart showing a correlation among a plurality of appearing objects.

### Claim 9 (depends on 1)

The portable information terminal according to claim 1, wherein
upon determining, based on the line-of-sight information, that the user is unsure in understanding the appearing object, the processor displays the virtual reality object.

### Claim 10 (depends on 1)

The portable information terminal according to claim 1, further comprising a communication unit for communicating with the information apparatus, wherein
when the communication unit receives reading support information from the information apparatus, the processor displays the virtual reality object.

### Claim 11 (depends on 1)

The portable information terminal according to claim 1, wherein
the processor is configured to:
store book information indicative of a content of the document in the memory;
upon recognizing a title of a book based on the captured image, determine which portion the user is reading based on the line-of-sight information;
generate a plot summary of portions that have been read by the user based on the stored book information; and
generate the virtual reality object indicative of the plot summary.

### Claim 12

A portable information terminal, comprising:
a camera that captures an image of a field of view of a user and generates a captured image;
a display;
a memory; and
a processor,
the processor being configured to:
carry out an image analysis on the captured image to extract appearing object information including attribution information about an appearing object which appears in a video included in the captured image;
store the appearing object information in the memory;
identify a scene in which the video is being displayed based on the captured image;
generate a virtual reality object including a description about the appearing object corresponding to the scene in which the video is being displayed based on the appearing object information; and
display the virtual reality object in a display area of the display, which corresponds to a periphery of the video.

### Claim 13

A method for displaying a virtual reality object, comprising the steps of:
by a processor,
carrying out an image analysis on a captured image in which a field of view of a user is captured, and based on description information included in a document captured in the captured image, extracting appearing object information including attribution information about an appearing object which appears in the document;
acquiring line-of-sight information to identify a portion on which a viewpoint of the user is present, the line-of-sight information being obtained and output by a line-of-sight detection sensor that has detected a line of sight of the user;
generating a virtual reality object including a description about the appearing object corresponding to the portion on which the viewpoint of the user is present based on the appearing object information; and
displaying the virtual reality object in a display area of a display, which corresponds to a periphery of the document.

### Claim 14

A method for displaying a virtual reality object, comprising the steps of:
by a processor,
carrying out an image analysis on a captured image in which a field of view of a user is captured to extract appearing object information including attribution information about an appearing object which appears in a video included in the captured image;
identifying a scene in which the video is being displayed based on the captured image;
generating a virtual reality object including a description about the appearing object corresponding to the scene in which the video is being displayed based on the appearing object information; and
displaying the virtual reality object in a display area of a display, which corresponds to a periphery of the video.
