# US 2014/0192055 A1 · Method and apparatus for displaying video on 3d map

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0192055 A1
- Document ID: US20140192055A1
- Status: Published application
- Kind: A1
- Application number: 13/935,689
- Filed: 2013-07-05
- Published: 2014-07-10
- Earliest claimed: 2013-01-08
- Assignees: Samsung Techwin Co., Ltd.
- Applicants: Samsung Techwin Co., Ltd.
- Inventors: Sung-Duck KIM
- Web page: https://patentgrep.com/patent/US20140192055A1

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Exemplary embodiments disclose a method of projecting an image onto a surface of a three-dimensional (3D) electronic map. The method includes: extracting nearest intersecting points for each of a plurality of virtual view angle vectors with respect to a position of a virtual photographing apparatus and a plurality of polygons that constitute the 3D electronic map; comparing 3D coordinates of the extracted nearest intersecting points and 3D coordinates of a plurality of pixels constituting the plurality of polygons to select pixels that are within a range of the 3D coordinates of the extracted nearest intersecting points; converting 3D coordinates of the selected pixels to two-dimensional (2D) coordinates to display the selected pixels on a 2D display; and superimposing an input image on top of the selected pixels to output the superimposed image in real-time.

## Claims

### Claim 1

A method of displaying a captured image onto a three-dimensional (3D) electronic map, the method comprising:
determining a position of a virtual photographing apparatus within the 3D electronic map corresponding to a position on which a real photographing apparatus is installed in a real world environment;
extracting nearest intersecting points, which are positioned on a plurality of virtual view angle vectors, and each of which intersects with one of a plurality of polygons constituting the 3D electronic map in a region around the real photographing apparatus or the virtual photographing apparatus, wherein the plurality of virtual view angle vectors are generated based on view angle information of the virtual photographing apparatus and the position of the virtual photographing apparatus;
selecting pixels that are within a range of coordinates of the extracted nearest intersecting points by comparing 3D coordinates of the extracted nearest intersecting points and 3D coordinates of a plurality of pixels constituting the plurality of polygons;
converting 3D coordinates of the selected pixels to two-dimensional (2D) coordinates to display the selected pixels on a 2D display;
superimposing an image captured by the real photographing apparatus on top of the selected pixels, using the converted 2D coordinates of the selected pixels; and
outputting, in real-time, the 3D electronic map, including the superimposed image, on the 2D display.

### Claim 2 (depends on 1)

The method of claim 1, wherein the nearest intersecting points are positioned on the virtual view angle vectors and intersect selected polygons which are nearest to the real photographing apparatus or the virtual photographing apparatus among the polygons constituting the 3D electronic map.

### Claim 3 (depends on 1)

The method of claim 1, wherein the plurality of virtual view angle vectors are used within the 3D electronic map and are formed of four vectors, and the each of the plurality of virtual view angle vectors has an angle corresponding to a view angle of the real photographing apparatus.

### Claim 4 (depends on 1)

The method of claim 1, wherein 2D coordinates information and 3D coordinates information, which refer to coordinates information obtained by converting the 3D coordinates information used in the 3D electronic map to the 2D coordinates information to display the 3D coordinate information on the 2D display, are stored in advance in a storage unit, and the stored 2D coordinates information is used in the converting.

### Claim 5 (depends on 1)

The method of claim 1, wherein a portion of the superimposed image may be designated by a user.

### Claim 6 (depends on 1)

The method of claim 1, wherein in the outputting, the 3D electronic map is output on the 2D display in real-time, such that only the superimposed image is highlighted in the 3D electronic map.

### Claim 7 (depends on 5)

The method of claim 5, wherein areas not corresponding to the superimposed image within the 3D electronic map are made dark or semi-transparent to highlight the superimposed image.

### Claim 8

An apparatus for displaying a captured image onto a three-dimensional (3D) electronic map, comprising:
a reference setting device configured to determine a position of a virtual photographing apparatus within the 3D electronic map corresponding to a position on which a real photographing apparatus is installed in a real world environment;
an intersecting point extracting device configured to extract a nearest intersecting point which is positioned on each of a plurality of virtual view angle vectors each of which intersects one of a plurality polygons constituting the 3D electronic map in a region around the real photographing apparatus or the virtual photographing apparatus, wherein the plurality of virtual view angle vectors are generated based on view angle information of the virtual photographing apparatus and the position of the virtual photographing apparatus;
a selecting device configured to select pixels that are within a range of coordinates of the extracted nearest intersecting points by comparing 3D coordinates of the extracted nearest intersecting points and 3D coordinates of a plurality of pixels constituting the plurality of polygons;
a converting device configured to convert 3D coordinates of the selected pixels to two-dimensional (2D) coordinates of the selected pixels to display the selected pixels on a 2D display;
an image superimposing device configured to superimpose an image captured by the real photographing apparatus on top of the selected pixels, wherein the superimposing device uses the converted 2D coordinates of the selected pixels; and
an output device configured to output, in real-time, the 3D electronic map, including the superimposed image, on the 2D display.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the intersecting points are positioned on the virtual view angle vectors and intersect selected polygons which are nearest to the real photographing apparatus or the virtual photographing apparatus among the polygons constituting the 3D electronic map.

### Claim 10 (depends on 8)

The apparatus of claim 8, further comprising a storage unit that stores 3D coordinates information used in the 3D electronic map and 2D coordinates information, wherein the 2D coordinates information is obtained by converting the 3D coordinates information to display the 3D coordinates information on the 2D display.

### Claim 11 (depends on 8)

The apparatus of claim 8, wherein the position of the virtual photographing apparatus is a position within the 3D electronic map corresponding to the position of the real photographing apparatus, wherein the position of the real photographing apparatus is determined by a global positioning system (GPS).

### Claim 12 (depends on 8)

The apparatus of claim 8, wherein the plurality of virtual view angle vectors are used in the 3D electronic map and are formed of four vectors, and the each of the plurality of virtual view angle vectors have an angle corresponding to a view angle of the real photographing apparatus.

### Claim 13

A system for displaying a captured image onto a three-dimensional (3D) electronic map, comprising:
a map server configured to store data of the 3D electronic map, wherein the 3D electronic map consists of a plurality of polygons;
a real photographing apparatus configured to photograph a monitor area; and
an image superimposing apparatus configured to superimpose an image captured by the real photographing apparatus with a position on the 3D electronic map,
wherein the image superimposing apparatus comprises:
a reference setting device configured to determine a position of a virtual photographing apparatus within the 3D electronic map corresponding to a position on which a real photographing apparatus is installed in a real world environment;
an intersecting point extracting device configured to extract nearest intersecting points which are positioned on a plurality of virtual view angle vectors and each of which intersect the respective plurality of polygons, wherein the plurality of virtual view angle vectors are generated based on view angle information of the virtual photographing apparatus and the position of the virtual photographing apparatus;
a selecting device configured to select pixels that are within a range of coordinates of the extracted nearest intersecting points by comparing 3D coordinates of the extracted nearest intersecting points and 3D coordinates of a plurality of pixels constituting the plurality of polygons;
a converting device configured to convert 3D coordinates of the selected pixels to two-dimensional (2D) coordinates of the selected pixels to display the selected pixels on a 2D display;
an image superimposing device configured to superimpose an image captured by the real photographing apparatus on top of the selected pixels, wherein the superimposing device uses the selected pixels and the converted 2D coordinates of the selected pixels; and
an output device configured to output, in real-time, the 3D electronic map, including the superimposed image, on the 2D display.

### Claim 14 (depends on 13)

The system of claim 13, wherein the intersecting points are positioned on the virtual view angle vectors and intersect selected polygons which are nearest to the real photographing apparatus or the virtual photographing apparatus among the polygons constituting the 3D electronic map.

### Claim 15 (depends on 13)

The system of claim 13, wherein the position of the virtual photographing apparatus is a position within the 3D electronic map corresponding to the position of the real photographing apparatus, wherein the position of the real photographing apparatus is determined by global positioning system (GPS).

### Claim 16 (depends on 13)

The apparatus of claim 13, wherein the plurality of virtual view angle vectors are used in the 3D electronic map and are formed of four vectors, and the each of the plurality of virtual view angle vectors have an angle corresponding to a view angle of the real photographing apparatus.