# US 2020/0007841 A1 · Transforming Locations in a Spherical Image Viewer

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0007841 A1
- Document ID: US20200007841A1
- Status: Published application
- Kind: A1
- Application number: 16/021,482
- Filed: 2018-06-28
- Published: 2020-01-02
- Earliest claimed: 2018-06-28
- Assignees: Eyespy360 Limited
- Applicants: Eyespy360 Limited
- Inventors: Peter Vassilev Sedeffow
- Web page: https://patentgrep.com/patent/US20200007841A1

## CPC

- G06T 3/06: Topological mapping of higher dimensional structures onto lower dimensional surfaces
- H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
- G06T 7/55: from multiple images
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method and apparatus are provided for transforming data provided in a spherical format. A spherical format is created of an image obtained by a camera, the spherical format comprising a notional sphere that has a centre corresponding to the position from which the image was obtained by the camera. A first surface represented in the image had a first orientation and was at a first distance from the camera when the image was obtained. A selected point in said spherical format is obtained, which is defined by spherical coordinates consisting of a yaw angle and a pitch angle defining a line from the centre of the sphere. A plane is identified that has the first orientation and is at the first distance from the centre. A location in a Cartesian coordinate system is calculated where the plane intersects with the line, wherein two of the axes of the Cartesian coordinate system are parallel to the first plane. Thereby, the position of the point on the first surface is identified. This location, and other locations corresponding to to other selected points, may be used to transform data in the spherical image for display.

## Claims

### Claim 1

A method of transforming data provided in a spherical format, comprising the steps of, at a processor:
creating a spherical format of an image obtained by a camera, wherein said spherical format has a centre that corresponds to the position from which said image was obtained by the camera, and wherein a first surface represented in said image had a first orientation and was at a first distance from the camera when the image was obtained;
obtaining a selected point in said spherical format, defined by spherical coordinates consisting of a yaw angle and a pitch angle defining a line from said centre;
obtaining said first orientation and said first distance;
identifying a plane having said first orientation that is at said first distance from said centre;
calculating a location in a Cartesian coordinate system where said plane intersects with said line, wherein two of the axes of said Cartesian coordinate system are parallel to said first plane, thereby identifying the position of the point on said first surface; and
using said location to transform data in said spherical image for display.

### Claim 2 (depends on 1)

A method according to claim 1, wherein said step of calculating said location comprises:
calculating a relative distance from the centre of said plane to the intersection of said plane with said line, said relative distance being relative to said first distance; and
using said relative distance to calculate relative coordinates in said Cartesian coordinate system.

### Claim 3 (depends on 2)

A method according to claim 2, wherein:
said step of calculating said relative distance comprises calculating the tangent of either the absolute value of the pitch angle, or the absolute value of the pitch angle subtracted from 90°; and
said step of calculating relative coordinates comprises calculating:
a) the cosine of the yaw angle, multiplied by said relative distance; and
b) the sine of the opposite of the yaw angle, multiplied by said relative distance.

### Claim 4 (depends on 2)

A method according to claim 2, wherein either:
a) said relative distance is modified using said first distance before said relative coordinates are calculated, and said relative coordinates are equal to said location; or
b) said relative coordinates are modified using said first distance to produce said location.

### Claim 5 (depends on 1)

A method according to claim 1, further comprising the step of obtaining an additional plurality of selected points in said spherical format and calculating an additional plurality of corresponding locations according to the method of claim 1;
and wherein said step of transforming data in said spherical image comprises:
rendering said locations as a two-dimensional plan, representing the location of said selected points on said first surface.

### Claim 6 (depends on 1)

A method according to claim 1, wherein a second surface represented in said image had said first orientation and was at a second distance from said first surface when the image was obtained, further comprising the steps of:
identifying a second plane having said first orientation that is at said second distance from said first plane; and
identifying a location in said Cartesian coordinate system on said second plane that is at the shortest distance from said selected point;
thereby obtaining Cartesian coordinates of points on parallel planes corresponding to said first and second surfaces.

### Claim 7 (depends on 1)

A method according to claim 1, wherein:
said first surface is the ground as seen in said image, and said first distance is obtained by identifying the height of the camera from the ground when the image was obtained.

### Claim 8 (depends on 1)

A method according to claim 1, further comprising the steps of:
obtaining data relating to the pose of the camera relative to said first surface when the image was obtained, said data comprising a pose-pitch angle and a pose-roll angle;
modifying each calculation related to a transformation between spherical coordinates and Cartesian coordinates and vice versa with said pose-pitch angle and said pose-roll angle.

### Claim 9 (depends on 1)

A method according to claim 1, further comprising the step of obtaining an additional plurality of selected points in said spherical format and calculating an additional plurality of corresponding locations according to the method of claim 1;
and wherein said step of transforming data in said spherical image comprises:
calculating angles between lines joining said locations, and summing the deviations of these angles from a predetermined angle; and
thereby estimating the deviation of the camera from vertical when the image was obtained.

### Claim 10

Apparatus for identifying a location on a surface represented in a spherical image, comprising a processor and a memory, wherein said processor is configured to:
create a spherical format of a first image obtained by a camera, wherein said spherical format has a centre that corresponds to the position from which said image was obtained by the camera, and wherein a first surface represented in said first image had a first orientation and was at a first distance from the camera when the image was obtained;
obtain a selected point in said spherical format, defined by spherical coordinates consisting of a yaw angle and a pitch angle defining a line from said centre;
retrieve from memory said first orientation and said first distance;
identify a plane having said first orientation that is at said first distance from said centre;
calculate a location in a Cartesian coordinate system where said plane intersects with said line, wherein two of the axes of said Cartesian coordinate system are parallel to said first plane, thereby identifying the position of the point on said first surface; and
use said location to transform data in said spherical image for display.

### Claim 11 (depends on 10)

Apparatus according to claim 10, wherein said processor is configured to calculate said location by:
calculating a relative distance from the centre of said plane to the intersection of said plane with said line, said relative distance being relative to said first distance; and
using said relative distance to calculate relative coordinates in said Cartesian coordinate system.

### Claim 12 (depends on 11)

Apparatus according to claim 11, wherein said processor is configured to:
calculate said relative distance by calculating the tangent of either the absolute value of the pitch angle, or the absolute value of the pitch angle subtracted from 90°; and
calculate said relative coordinates by calculating:
a) the cosine of the yaw angle, multiplied by said relative distance; and
b) the sine of the opposite of the yaw angle, multiplied by said relative distance.

### Claim 13 (depends on 11)

Apparatus according to claim 11, wherein said processor is configured to either:
a) modify said relative distance using said first distance before calculating said relative coordinates, wherein said relative coordinates are equal to said location; or
b) modify said relative coordinates using said first distance to produce said location.

### Claim 14 (depends on 1, 10)

Apparatus according to claim 10, wherein said processor is further configured to obtain an additional plurality of selected points in said spherical format and calculate an additional plurality of corresponding locations according to the method of claim 1;
and wherein said processor is configured to transform data in said spherical image by:
rendering said locations as a two-dimensional plan, representing the location of said selected points on said first surface.

### Claim 15 (depends on 10)

Apparatus according to claim 10, wherein said processor is further configured to:
identify a second plane having said first orientation that is at a second distance from said first plane; and
identify a location in said Cartesian coordinate system on said second plane that is at the shortest distance from said selected point;
thereby obtaining Cartesian coordinates of points on parallel planes corresponding to said first and second surfaces.

### Claim 16 (depends on 10)

A method according to claim 10, wherein:
said first surface is the ground as seen in said image, and said processor is configured to obtain said first distance by identifying the height of the camera from the ground when the image was obtained.

### Claim 17 (depends on 10)

Apparatus according to claim 10, wherein said processor is further configured to:
obtain data relating to the pose of the camera relative to said first surface when the image was obtained, said data comprising a pose-pitch angle and a pose-roll angle;
modify each calculation related to a transformation between spherical coordinates and Cartesian coordinates and vice versa with said pose-pitch angle and said pose-roll angle.

### Claim 18 (depends on 1, 10)

A method according to claim 10, wherein said processor is further configured to obtain an additional plurality of selected points in said spherical format and calculate an additional plurality of corresponding locations according to the method of claim 1;
and wherein said processor is configured to transform data in said spherical image by:
calculating angles between lines joining said locations, and summing the deviations of these angles from a predetermined angle; and
thereby estimate the deviation of the camera from vertical when the image was obtained.

### Claim 19

A non-transitory computer-readable medium with computer executable instructions stored thereon, wherein said instructions configure a processor to perform the method of:
creating a spherical format of an image obtained by a camera, wherein said spherical format has a centre that corresponds to the position from which said image was obtained by the camera, and wherein a first surface represented in said image had a first orientation and was at a first distance from the camera when the image was obtained;
obtaining a selected point in said spherical format, defined by spherical coordinates consisting of a yaw angle and a pitch angle defining a line from said centre;
obtaining said first orientation and said first distance;
identifying a plane having said first orientation that is at said first distance from said centre;
calculating a location in a Cartesian coordinate system where said plane intersects with said line, wherein two of the axes of said Cartesian coordinate system are parallel to said first plane, thereby identifying the position of the point on said first surface; and
using said location to transform data in said spherical image for display.
