# US 2006/0215027 A1 · Picked-up image display method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2006/0215027 A1
- Document ID: US20060215027A1
- Status: Published application
- Kind: A1
- Application number: 10/550,550
- Filed: 2003-06-20
- Published: 2006-09-28
- Earliest claimed: 2003-06-20
- Assignees: Mitsubishi Denki Kabushiki Kaisha
- Inventors: Yasumasa Nonoyama; Yoshiko Maeda
- Web page: https://patentgrep.com/patent/US20060215027A1

## CPC

- G01C 11/00: Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G09B 29/106: {using electronic means}
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

An image display method in which a position of a shot image of the ground surface having been taken from the air is specified three-dimensionally, a photographic area on the ground surface having been shot is obtained by computation, and a shot image is transformed in conformity with the photographic area thereof and thereafter displayed being superposed on a map of geographic information system, and in which landmarks are extracted from a map of a geographic information system and a shot image respectively, and the corresponding landmarks are compared, whereby a parameter for use in computing a photographic area having been shot is compensated.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

An image display apparatus of image processing and displaying a shot image of the ground surface having been taken with photographic equipment that is mounted on an airframe in the air,
the image display apparatus comprising:
an image frame computing means in which a shooting position in the air is specified three-dimensionally based on posture of said airframe and said photographic equipment with respect to said ground surface and a photographic area on the ground surface having been shot is obtained by computation;
an image transformation means in which a shot image is transformed in conformity with said photographic area;
a superposing means in which said transformed image is superposed on a map of a geographic information system; and
a monitor display means for displaying said superposed map.

### Claim 22

An image display apparatus of image processing and displaying a shot image of the ground surface having been taken with photographic equipment that is mounted on an airframe in the air,
the image display apparatus comprising:
an image frame computing means in which a shooting position in the air is specified three-dimensionally based on posture of said airframe and said photographic equipment with respect to said ground surface, and each of a plurality of photographic areas on the ground surface having been continuously shot is obtained by computation;
an image transformation means in which each of shot images is transformed in conformity with said each of the photographic areas;
a superposing means in which said plurality of transformed shot images are superposed on a map of a geographic information system; and
a monitor display means for displaying said superposed map.

### Claim 23 (depends on 22)

The image display apparatus according to claim 22, further comprising an image joining and compensating means in which a plurality of shot images to be superposed are partially overlapped with each other, and the shot images are moved and compensated so that an overlapping at the overlap part may be of the largest extent and thereafter are joined.

### Claim 24 (depends on 22)

The image display apparatus according to claim 22, wherein said plurality of shot images to be superposed are obtained by sampling the images having been continuously shot in cycles of a predetermined time period.

### Claim 25 (depends on 21)

The image display apparatus according to claim 21, wherein said image frame computing means obtains a photographic area on the ground surface having been shot by computation based on an inclination and a rotation angle of said photographic equipment with respect to said airframe.

### Claim 26 (depends on 21)

The image display apparatus according to claim 21, wherein said image frame computing means obtains a photographic area on the ground surface having been shot by computation based on an inclination and a roll angle of said airframe with respect to the ground surface.

### Claim 27 (depends on 21)

The image display apparatus according to claim 21, wherein said image frame computing means obtains a photographic area on the ground surface having been shot by computation based on an inclination and rotation angle of said photographic equipment with respect to said airframe, and an inclination and roll angle of said airframe with respect to the ground surface.

### Claim 28 (depends on 27)

The image display apparatus according to claim 27, wherein said image frame computing means obtains a photographic area on the ground surface by computation, thereafter obtains an altitude of the ground surface of said photographic area by utilizing a three-dimensional topographic data including altitude information as to undulation of the ground surface which data has been preliminarily prepared, computes an altitude of shooting point as a relative altitude obtained by subtracting an altitude of the ground surface from an absolute altitude of the airframe;
the image transformation means transforms a shot image in conformity with said photographic area; and
the superposing means displays the transformed image so as to be superposed on a map of the geographic information system.

### Claim 29

An image display apparatus for taking a shot of the ground surface with photographic equipment that is mounted on an airframe in the air, and to identify situations existing on said ground surface by comparison between a shot image and a map;
wherein a shooting position in the air is specified three-dimensionally based on posture of said airframe and said photographic equipment with respect to said ground surface, and signals of said airframe positional information, camera information, and airframe information are transmitted in synchronization with signals of an image having been shot; and
a photographic area on the ground surface having been shot is obtained by computation on the receiving side, and a shot image is transformed in conformity with said photographic area and thereafter displayed being superposed on a map of a geographic information system.

### Claim 30

An image display method of image processing and displaying a shot image of the ground surface having been taken with photographic equipment that is mounted on an airframe in the air,
wherein a shooting position in the air is specified three-dimensionally based on posture of said airframe and said photographic equipment with respect to said ground surface and, a photographic area of at least one image of the ground surface having been shot is obtained by computation;
a shot image is transformed in conformity with said photographic area;
thereafter the transformed shot image is displayed being superposed on a map of a geographic information system, and the shot image having been superposed on the map can be erased leaving only a photographic area frame.

### Claim 31 (depends on 30)

The image display method according to claim 30, wherein landmarks are extracted from a map of said geographic information system and said shot image respectively, and the corresponding landmarks are compared, whereby a parameter for use in computing a photographic area of the ground surface having been shot is compensated, and a shot image is displayed being superposed with high precision on a map of the geographic information system.

### Claim 32 (depends on 31)

The image display method according to claim 31, wherein a parameter to be compensated is changed in accordance with the number of landmarks having been extracted.

### Claim 33 (depends on 31)

The image display method according to claim 31, wherein an inclination and a rotation angle of said photographic equipment with respect to said airframe are compensated based on the landmark having been extracted, and a photographic area on the ground surface having been shot is computed.

### Claim 34 (depends on 31)

The image display method according to claim 31, wherein an inclination and a roll angle of said airframe with respect to the ground surface are compensated based on the landmark having been extracted, and a photographic area on the ground surface having been shot is computed.

### Claim 35 (depends on 31)

The image display method according to claim 31, wherein 2 points of landmarks having been extracted are used, an altitude of said airframe is compensated based on a distance between two points, and a photographic area of the ground surface having been shot is computed.

### Claim 36 (depends on 31)

The image display method according to claim 31, wherein an average value of parameter compensation values between 2 points of each landmark is used in the case of not less than 3 landmarks having been extracted, and a photographic area of the ground surface having been shot is computed.

### Claim 37 (depends on 31)

The image display method according to claim 31, wherein in the case of absence of the corresponding landmarks at the time of extracting landmarks from a map of said geographic information system and each of said plural pieces of shot images respectively, a parameter for use in computing a photographic area on the ground surface having been shot is compensated based on a compensation value at the time of having extracted a landmark last, and shot images to be joined partially overlapped with each other and displayed being superposed on the map are moved such that an overlapped state at said overlap part is of the largest extent, and thereafter are joined.

### Claim 38 (depends on 31)

The image display method according to claim 31, wherein landmarks are extracted from a map of the geographic information system and each shot image respectively, a parameter for use in computing each photographic area of the ground surface having been continuously shot is compensated based on a current compensation value, getting back to a halfway point between the shot image of when a landmark has been extracted last time and the current shot image, and said plural pieces of shot images are displayed being superposed with high precision on a map of the geographic information system.

### Claim 39 (depends on 35)

The image display method according to claim 35, wherein an altitude compensation value is registered at a point of land where altitude compensation processing of a shot image is executed due to coincidence of the landmarks, and said registered altitude compensation value can be utilized again as a reference value of altitude compensation in the case of flying at a point of land close to said point from the next time on.
