# US 2011/0254915 A1 · Three-Dimensional Overlays Within Navigable Panoramic Images, and Applications Thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2011/0254915 A1
- Document ID: US20110254915A1
- Status: Published application
- Kind: A1
- Application number: 13/171,143
- Filed: 2011-06-28
- Published: 2011-10-20
- Earliest claimed: 2007-05-25
- Assignees: Google Inc.
- Inventors: Luc Vincent; Daniel Filip; Stephen Chau; Stéphane Lafon; Zhonghao Yang; Andrew Timothy Szybalski
- Web page: https://patentgrep.com/patent/US20110254915A1

## CPC

- G06T 19/003: {Navigation within 3D models or images}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06T 17/05: Geographic models
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2219/028: Multiple view windows (top-side-front-sagittal-orthogonal)
- G06T 2219/2004: Aligning objects, relative positioning of parts
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A panorama viewer is disclosed which facilitates navigation from within the panorama of a larger, structured system such as a map. The panorama viewer presents a viewport on a portion of a panoramic image, the viewport including a three-dimensional overlay rendered with the panoramic image. As the orientation of the viewport within the panoramic image changes, the three-dimensional overlay's orientation in three-dimensional space also changes as it is rendered with the panoramic image in a manner that matches the change in orientation of the viewport.

## Claims

### Claim 1

A method for viewing panoramic images, comprising:
(a) receiving at least a portion of a first panoramic image geocoded to a first location;
(b) receiving configuration information specifying an overlay for the first panoramic image;
(c) rendering, in a three-dimensional space, the at least a portion of the first panoramic image for display;
(d) rendering the overlay in the three-dimensional space of the first panoramic image for display with the first panoramic image in a viewport;
(e) in response to receipt of a first input to change orientation of the viewport, changing the orientation of the overlay in three-dimensional space as it is rendered with the first panoramic image so as to match a change in orientation of the first panoramic image within the viewport; and
(f) in response to receipt of a second input specifying a position on the first panoramic image, presenting a second panoramic image geocoded to a second location different from the first location, the second location determined according to the position.

### Claim 2 (depends on 1)

The method of claim 1, wherein the rendering (d) comprises modeling the overlay using a pinhole camera model, wherein a focal length of the pinhole camera model corresponds to a height of the viewport and a depth of the three-dimensional space.

### Claim 3 (depends on 1)

The method of claim 1, wherein the rendering (c) comprises texture mapping the at least a portion of the first panoramic image to a surface having canonical homomorphism.

### Claim 4 (depends on 1)

The method of claim 1, wherein the overlay comprises an annotation describing a feature of the first panoramic image, the annotation positioned to correspond to a position of the feature in the first panoramic image.

### Claim 5 (depends on 4)

The method of claim 4, wherein the annotation is a street line positioned to correspond to the direction of a street in the first panoramic image, and wherein the overlay further comprises a street name positioned to identify the street in the first panoramic image.

### Claim 6 (depends on 1)

The method of claim 1, wherein the overlay includes a navigation icon associated with the second panoramic image and wherein the second input in (f) is a selection of the navigation icon.

### Claim 7 (depends on 1)

The method of claim 1, wherein the receiving (a) comprises receiving street-level imagery.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
(g) displaying a user orientation character at a first position on a map, the first position corresponding to the first location; and
(h) in response to the second input, displaying the user orientation character at a second position on the map, the second position corresponding to the second location.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
(j) presenting a first line on the map representing a first street where panoramic image data is unavailable; and
(k) presenting a second line on the map differently from the presenting in (j), the second line representing a second street where panoramic image data is available,
whereby the difference in the presenting (j) and the presenting (k) indicates to a user where panoramic image data is available.

### Claim 10

A system for viewing panoramic images, comprising:
a panorama viewer that
(i) receives at least a portion of a first panoramic image and configuration information, the first panoramic image geocoded to a first location, and the configuration information specifying an overlay for the first panoramic image,
(ii) renders, in a three-dimensional space, the at least a portion of the first panoramic image and the overlay of the first panoramic image for display in a viewport,
(iii) in response to receipt of a first input to change orientation of the viewport, changes the orientation of the overlay in three-dimensional space as it is rendered with the first panoramic image so as to match a change in orientation of the first panoramic image within the viewport, and
(iv) in response to receipt of a second input specifying a position on the first panoramic image, presents a second panoramic image geocoded to a second location different from the first location, the second location determined according to the position.

### Claim 11 (depends on 10)

The system of claim 10, wherein the panorama viewer models the overlay using a pinhole camera model to render the overlay in the three-dimensional space, wherein a focal length of the pinhole camera model corresponds to a height of the viewport and a depth of the three-dimensional space.

### Claim 12 (depends on 10)

The system of claim 10, wherein the panorama viewer models texture maps the at least a portion of the first panoramic image to a surface having canonical homomorphism to render the first panoramic image.

### Claim 13 (depends on 10)

The system of claim 10, wherein the overlay comprises an annotation describing a feature of the first panoramic image, the annotation positioned in the first panoramic image to correspond to a position of the feature.

### Claim 14 (depends on 13)

The system of claim 13, wherein the annotation is a street line positioned to correspond to the direction of a street in the first panoramic image, and wherein the overlay further comprises a street name positioned to identify the street in the first panoramic image.

### Claim 15 (depends on 10)

The system of claim 10, wherein the overlay includes a navigation icon associated with the second panoramic image and wherein the second input is a selection of the navigation icon.

### Claim 16 (depends on 10)

The system of claim 10, wherein the first and second panoramic images comprise street-level imagery.

### Claim 17 (depends on 10)

The system of claim 10, further comprising:
a mapping service that displays a user orientation character at a first position on a map, the first position corresponding to the first location and, in response to the second input, displays the user orientation character at a second position on the map, the second position corresponding to the second location.

### Claim 18 (depends on 17)

The system of claim 17, wherein the mapping service presents a first line on the map representing a first street where panoramic image data is unavailable and presents a second line on the map differently from the presentation of the first line, the second line representing a second street where panoramic image data is available,
whereby the difference in the presentation of the first line and the presentation of the second line indicates to a user where panoramic image data is available.

### Claim 19

A system for serving panoramic images and metadata, comprising:
a server that sends data to a client, the data including at least a portion of a panoramic image geocoded to a location, a configuration information specifying an overlay for the panoramic image, and a file,
wherein the file instructs the client to render, in a three-dimensional space, the at least a portion of the panoramic image for display and to render the overlay in the three-dimensional space of the panoramic image for display with the panoramic image in a viewport,
wherein the file instructs the client, in response to receipt of a first input to change orientation of the viewport, to change the orientation of the overlay in three-dimensional space as it is rendered with the panoramic image so as to match a change in orientation of the panoramic image within the viewport, and
wherein the file instructs the client, in response to receipt of a second input specifying a position on the panoramic image, to present a second panoramic image geocoded to a second location different from the first location, the second location determined according to the position.

### Claim 20 (depends on 19)

The system of claim 19, wherein the file is a Flash file.
