# US 2013/0278755 A1 · Apparatus and Method for Spatially Referencing Images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0278755 A1
- Document ID: US20130278755A1
- Status: Published application
- Kind: A1
- Application number: 13/783,878
- Filed: 2013-03-04
- Published: 2013-10-24
- Earliest claimed: 2012-03-19
- Assignees: Google Inc.
- Applicants: Google, Inc
- Inventors: Alexander T. Starns; Arjun Raman; Gadi Royz
- Web page: https://patentgrep.com/patent/US20130278755A1

## CPC

- H04N 13/20: Image signal generators
- G01C 11/02: Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
- G01C 11/025: {by scanning the object}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06T 7/593: from stereo images
- G06T 7/68: of symmetry
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G06T 2207/10012: Stereo images
- G06T 2207/30244: Camera pose
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Provided is a method of spatially referencing a plurality of images captured from a plurality of different locations within an indoor space by determining the location from which the plurality of images were captured. The method may include obtaining a plurality of distance-referenced panoramas of an indoor space. The distance-referenced panoramas may each include a plurality of distance-referenced images each captured from one position in the indoor space and at a different azimuth from the other distance-referenced images, a plurality of distance measurements, and orientation indicators each indicative of the azimuth of the corresponding one of the distance-referenced images. The method may further include determining the location of each of the distance-referenced panoramas based on the plurality of distance measurements and the orientation indicators and associating in memory the determined locations with the plurality of distance-referenced images captured from the determined location.

## Claims

### Claim 1

A method of spatially referencing a plurality of images captured from a plurality of different locations within an indoor space by determining the locations from which the plurality of images were captured, comprising:
obtaining a plurality of distance-referenced panoramas of an indoor space captured from a plurality of different locations in the indoor space, the distance-referenced panoramas each comprising:
a plurality of distance-referenced images each captured from one location among the plurality of different locations in the indoor space and at a different azimuth from the other distance-referenced images captured from the one location;
a plurality of distance measurements, each distance measurement being indicative of a distance between the one location and a feature depicted in a corresponding one of the distance-referenced images;
a plurality of orientation indicators each indicative of the azimuth of the corresponding one of the distance-referenced images;
wherein each of the plurality of distance measurements is associated with the corresponding one of the distance-referenced images and one of the plurality of orientation indicators;
determining the location of each of the distance-referenced panoramas based on the plurality of distance measurements and the plurality of orientation indicators by:
determining whether the indoor space is rectangular based on the plurality of distance measurements and the plurality of orientation indicators by comparing the sum of opposing distance measurements at different locations; and
upon determining that the indoor space is rectangular, determining the location of each of the distance-referenced panoramas by calculating coordinates relative to an origin; and
associating in memory the determined locations with the plurality of distance-referenced images captured from the determined location.

### Claim 2 (depends on 1)

The method of claim 1, wherein the number of distance measurements is equal to the number of distance referenced images.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining whether the indoor space is rectangular based on the plurality of distance measurements and the plurality of orientation indicators by comparing the sum of opposing distance measurements at different locations comprises:
identifying, based on the plurality of orientation indicators, two pairs of distance measurements in approximately opposite directions, one pair taken at a first location and the other pair taken at a second location;
calculating four cross-space dimensions, two at each location among the first and second locations, by summing each the opposing distance measurements within each pair;
calculating two delta-cross-space dimensions based on a difference between each cross-space dimension from one location among the first and second locations and the corresponding cross-space dimension from the other location among the first and second locations; and
determining whether the indoor space is rectangular based on whether the two delta-cross-space dimensions are less than a threshold distance.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining the location of each of the distance-referenced panoramas based on the plurality of distance measurements and the plurality of orientation indicators comprises:
matching features depicted in images captured at a first and at a second location; and
based on the matching features, selecting a distance measured at the second location; and
determining the second location based on the selected distance.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining the location of each of the distance-referenced panoramas based on the plurality of distance measurements and the plurality of orientation indicators comprises:
identifying a cross-space dimension shared between a first and a second distance-referenced panorama of the distance-referenced panoramas, the first and the second distance-referenced panoramas being captured at a first location and at a second location, respectively; and
determining the location of the second distance-referenced panorama based on both the location of the first distance-referenced panorama and a difference between segments of the shared cross-space dimension measured at the first location and at the second location.

### Claim 6 (depends on 1)

The method of claim 1, comprising:
obtaining a dimensioned layout of the indoor space, the dimensioned layout comprising a plurality of layout cross-space dimensions of the indoor space;
wherein determining the location of each of the distance-referenced panoramas based on the plurality of distance measurements and the plurality of orientation indicators comprises:
obtaining a plurality of measured cross-space dimensions calculated by summing opposing distance measurements measured at a location of one of the distance-referenced panoramas;
matching the measured cross-space dimensions to the layout cross-space dimensions of the indoor space;
determining the location of one of the distance-referenced panoramas based on the matched cross-space dimensions and the dimensioned layout of the indoor space.

### Claim 7 (depends on 1)

The method of claim 1, comprising:
obtaining a new-feature indicator indicative of whether a distance measurement among the plurality of distance measurements measures a distance to a feature that was not measured in a previous distance-referenced panorama; and
wherein determining the location of each of the distance-referenced panoramas based on the plurality of distance measurements and the plurality of orientation indicators comprises:
determining the location of each of the distance-referenced panoramas based on the plurality of distance measurements, the plurality of orientation indicators, and the new-feature indicator.

### Claim 8

A camera for acquiring spatially-referenced images of indoor spaces, the camera comprising:
a camera having an optical axis;
a camera mount coupled to the camera and configured to align the optical axis with a plurality of generally radially symmetric directions, each of the radially symmetric directions approximately corresponding to an angular field of view of the camera;
a laser range finders coupled to the camera and oriented approximately parallel to the optical axis of the camera;
an orientation sensor coupled to the camera and configured to indicate which of the radially symmetric directions the camera is facing;
one or more processors communicatively connected to the laser range finders and the camera; and
tangible machine-readable memory communicatively connected to the processors, the memory storing instructions that when executed by one or more of the one or more processors cause the following to occur:
storing an image captured by the camera in the memory;
storing a distance measured by the laser range finder between the laser range finder and a feature depicted in the image;
storing an orientation indicator measured by the orientation sensor, the orientation indicators indicating an orientation of the optical axis at the time the image is captured; and
associating in the memory the image stored in the memory with the distance stored in memory and the orientation indicator.

### Claim 9 (depends on 8)

The camera of claim 8, comprising:
a plurality of cameras, each camera having a laser range finder coupled to the camera with zero degrees of freedom relative to the camera.

### Claim 10 (depends on 8)

The camera of claim 8, wherein the plurality of generally radially symmetric directions are eight or fewer generally radially symmetric directions.

### Claim 11 (depends on 8)

The camera of claim 8, wherein the orientation sensor comprises a magnetometer.

### Claim 12

A method of operating a distance-referencing camera, the method comprising:
positioning a camera in an indoor space at a first position;
orienting, from the first position, the camera in a first selected orientation;
capturing a first image of the indoor space with the camera from the first position in the first selected orientation;
measuring, from the first position, a first distance between a distance sensor and a first feature of the indoor space;
associating, in a tangible machine-readable memory, the measured distance with the image captured from the first position;
positioning the camera in the indoor space at a second position, the second position being substantially different from the first position;
orienting, from the second position, the camera in the first selected orientation;
capturing a second image of the indoor space with the camera from the second position in the first selected orientation;
measuring, from the second position, a second distance between the distance sensor and a second feature of the indoor space;
associating, in the tangible machine-readable memory, the second measured distance with the second image captured from the second position.

### Claim 13 (depends on 12)

The method of claim 12, wherein capturing the first image of the indoor space with the camera from the first position in the first selected orientation comprises:
capturing at least four images in at least four different directions each aligned with the first selected orientation, the at least four different directions being approximately radially symmetrically oriented.

### Claim 14 (depends on 13)

The method of claim 13, wherein measuring, from the first position, the first distance between the distance sensor and the first feature of the indoor space, comprises:
measuring four distances in four measurement directions, each measurement direction being generally aligned with the first selected orientation.

### Claim 15 (depends on 14)

The method of claim 14, comprising:
measuring four azimuths with a magnetometer, the fours azimuths corresponding to the four measurement directions; and
association, in the tangible machine readable memory, the measured four azimuths with the at least four images and the four distances measured at the first position.

### Claim 16 (depends on 12)

The method of claim 12, wherein the first feature and the second feature are the same feature.

### Claim 17 (depends on 12)

The method of claim 12, wherein the first distance is measured in a direction that corresponds to the second position.

### Claim 18

A method of determining the location from which images were captured, the method comprising:
obtaining a plurality of distance-referenced images, each distance-referenced image captured from a different position among a plurality of positions, and each distance-referenced image associated with two or fewer distances and two or fewer orientations;
determining the locations of the plurality of positions in a shared coordinate system based on the two or fewer distances and the two or fewer orientations; and
associating the determined locations with corresponding images of the distance-referenced images in spatially-referenced images stored by a spatially-referenced image repository.

### Claim 19 (depends on 18)

The method of claim 18, wherein determining the locations of the plurality of positions in a shared coordinate system based on the two or fewer distances and the two or fewer orientations comprises:
calculating a difference between a first distance and a second distance, wherein the first distance is measured from an earlier position among the plurality of positions in an orientation, and the second distance is measured from a later position among the plurality of positions in the same orientation as the first distance.

### Claim 20 (depends on 18)

The method of claim 18, comprising:
receiving, in an interactive tour web server, a request for a spatially-referenced image user interface;
retrieving, based on the request, from the spatially-referenced image repository a spatially-referenced image responsive to the request;
identifying, based on a position of the spatially-referenced image, another spatially-referenced image stored in the spatially-referenced image repository;
transmitting the spatially-referenced image and a selectable icon by which a user requests the other spatially-referenced image.