# US 2011/0254861 A1 · Information displaying apparatus and information displaying method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2011/0254861 A1
- Document ID: US20110254861A1
- Status: Published application
- Kind: A1
- Application number: 13/140,889
- Filed: 2009-12-22
- Published: 2011-10-20
- Earliest claimed: 2008-12-25
- Assignees: Panasonic Corporation
- Inventors: Koichi Emura; Yasuaki Inatomi; Junko Ueda; Toshiki Kanehara
- Web page: https://patentgrep.com/patent/US20110254861A1

## CPC

- H04N 23/633: {for displaying additional information relating to control or operation of the camera}
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04N 5/445: for displaying additional information (H04N5/50 takes precedence)
- H04N 21/42202: {environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes}
- H04N 21/4223: Cameras (H04N23/00 takes precedence)
- H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window}
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/4622: {Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859)}
- H04N 21/4725: using interactive regions of the image, e.g. hot spots {(details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94)}
- H04N 23/62: Control of parameters via user interfaces
- H04N 23/632: {for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters}
- H04N 23/67: Focus control based on electronic image sensor signals
- H04N 21/41407: {embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57)}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An information displaying apparatus capable of providing an easy-to-see display of information that the user wants to know out of information related to objects seen in a captured real-world image. The information displaying apparatus (100) comprises: an image input unit (300) for inputting a captured real-world image; a distance acquiring unit (650) for acquiring the distance between an object seen in the input captured real-world image and a view point of the captured real-world image; an object superimposing unit (710) for outputting an image in which information related to the object is superimposed on the captured real-world image; and an information output control unit (690) for controlling the information outputted by the object superimposing unit (710). The information output control unit (690) decides, in accordance with the acquired distance, how to present the information outputted by the object superimposing unit (710).

## Claims

### Claim 1

An information displaying apparatus comprising:
an image input section that receives as input a captured real-world image;
a distance acquisition section that acquires a distance between an object shown in the input captured real-world image and a viewpoint of the captured real-world image;
an object superimposition section that outputs an image in which information relating to the object is superimposed on the captured real-world image; and
an information output control section that controls the information output from the object superimposition section,
wherein the information output control section decides a way of presenting the information output from the object superimposition section, according to the acquired distance.

### Claim 2 (depends on 1)

The information displaying apparatus according to claim 1, wherein, the shorter the distance, the information output control section decides upon at least one of a greater amount of information and a more emphasized presentation mode for the information output from the object superimposition section.

### Claim 3 (depends on 1)

The information displaying apparatus according to claim 1, further comprising:
a subject position and orientation acquisition section that acquires a position of a viewpoint of the captured real-world image; and
an object positioning section that acquires a position in the real world of an object shown in the input captured real-world image,
wherein the object positioning section acquires the object position based on the acquired viewpoint position, and a distance, direction, and angle of elevation of an object with reference to the viewpoint.

### Claim 4 (depends on 3)

The information displaying apparatus according to claim 3, further comprising an image recognition section that recognizes from the input captured real-world image an object shown in the captured real-world image,
wherein the information output control section makes the object superimposition section output information corresponding to the decided way of presenting information among information relating to the recognized object.

### Claim 5 (depends on 3)

The information displaying apparatus according to claim 3, wherein the object positioning section acquires the object position based on an area in the captured real-world image of the recognized object and a viewpoint and field of view of the captured real-world image.

### Claim 6 (depends on 2)

The information displaying apparatus according to claim 2, further comprising a lens control section that changes a zoom factor of the captured real-world image input by the image input section, wherein, the larger a zoom factor of the captured real-world image is, the information output control section decides upon a greater amount of information.

### Claim 7 (depends on 2)

The information displaying apparatus according to claim 2, wherein, the smaller a number of objects that are information display objects is, the information output control section decides upon a greater amount of information for the information per object.

### Claim 8 (depends on 1)

The information displaying apparatus according to claim 1, wherein the information output control section decides upon an object that is an information display object from among objects shown in the input captured real-world image.

### Claim 9 (depends on 8)

The information displaying apparatus according to claim 8, wherein the information output control section decides upon an object that is an information display object within a range of a predetermined plurality of objects.

### Claim 10 (depends on 8)

The information displaying apparatus according to claim 8, further comprising a lens control section that changes a focus position of the captured real-world image input by the image input section, wherein the information output control section decides upon an object near the focus position as an information display object.

### Claim 11 (depends on 10)

The information displaying apparatus according to claim 10, wherein:
the lens control section fixes receives a predetermined operation and fixes the focus position; and
the object superimposition section changes a way of presenting information that is presented when the focus position has been fixed.

### Claim 12 (depends on 8)

The information displaying apparatus according to claim 8, further comprising a lens control section that changes a depth of field of the captured real-world image input by the image input section, wherein the information output control section decides upon an object positioned within a range of the depth of field as an information display object.

### Claim 13 (depends on 3)

The information displaying apparatus according to claim 3, wherein the object superimposition section displays information relating to the recognized object, associated with an area in the captured real-world image of that object.

### Claim 14 (depends on 4)

The information displaying apparatus according to claim 4, further comprising an information position decision section that, when information of a plurality of objects is displayed, decides display positions of information of the plurality of objects as positions corresponding to real-world positional relationships of the plurality of objects.

### Claim 15 (depends on 14)

The information displaying apparatus according to claim 14, wherein the information position decision section detects a depth direction of input the captured real-world image, and decides display positions of information of a plurality of objects so as to be aligned in the depth direction when the information of a plurality of objects is displayed.

### Claim 16 (depends on 1)

The information displaying apparatus according to claim 1, further comprising:
an operation input section that receives a user operation; and
an operation control section that switches among and sets a mode in which the user operation is reflected in the input captured real-world image, a mode in which the user operation is reflected in information of the presented captured real-world image, and a mode in which the user operation is reflected in the input captured real-world image and the object information is changed in conjunction with the input captured real-world image.

### Claim 17 (depends on 1)

The information displaying apparatus according to claim 1, further comprising an operation input section that receives a user operation by means of a touch panel operation for the captured real-world image displayed by the object superimposition section.

### Claim 18

An information displaying method comprising the steps of:
receiving as input a captured real-world image;
acquiring a distance between an object shown in the input captured real-world image and a viewpoint of the captured real-world image;
deciding a way of presenting information relating to the object according to the acquired distance; and
outputting the information superimposed on the captured real-world image so as to be presented in a decided way of presenting.
