# US 2014/0152655 A1 · Map Systems and Methods for Displaying Panoramic Images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0152655 A1
- Document ID: US20140152655A1
- Status: Published application
- Kind: A1
- Application number: 14/096,877
- Filed: 2013-12-04
- Published: 2014-06-05
- Earliest claimed: 2012-12-04
- Assignees: Nintendo Co., Ltd.
- Applicants: Ninetendo Co., Ltd.
- Inventors: Rory JOHNSTON; Corey Birnbaum
- Web page: https://patentgrep.com/patent/US20140152655A1

## CPC

- G06T 15/205: {Image-based rendering}
- G06F 1/1694: {the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/0412: {Digitisers structurally integrated in a display}
- G06T 3/12: Panospheric to cylindrical image transformations
- G06T 15/005: {General purpose rendering architectures}
- H04N 23/58: Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Operations for efficient display of panoramic images include receiving control information a plurality of tiles of a panoramic image, identifying padding included in the received tiles, and processing the tiles to remove the padding. For each of the processed tiles, at least one patch is generated which maps at least a portion of each tile to a three-dimensional surface formed in a memory. Each patch includes a plurality of polygonal areas, and corresponds to a respective portion of a sphere. The panoramic image is drawn using the patches.

## Claims

### Claim 1

A system, comprising an information processing device connectable over a network to a remote server, wherein the information processing device comprises:
first memory;
a network interface; and
a first processor arrangement including at least one processor, wherein the first processor arrangement is configured to perform first operations comprising:
receiving control information of a panoramic image and a plurality of image tiles of the panoramic image from a remote server through the network interface;
identifying, using at least the received control information, padding included in the received plurality of image tiles;
processing the received plurality of image tiles to remove the identified padding;
for each of the processed image tiles, generating at least one patch which maps at least a portion of said each processed image tile to a three-dimensional surface formed in the first memory, wherein each said generated patch comprises a plurality of polygonal areas, and wherein each said generated patch corresponds to a respective portion of a sphere; and
drawing the panoramic image using the generated patches.

### Claim 2 (depends on 1)

The system according to claim 1, further comprising
a movable display device communicably connected to the information processing device,
wherein the drawing performed by the first processor arrangement comprises:
forming, using the generated patches, a display image for the predicted geographic location; and
providing the display image to the movable display device,
and wherein the movable display device comprises:
a second memory; and
a second processor arrangement including at least one processor, wherein the second processor arrangement is configured to perform operations comprising displaying the provided display image.

### Claim 3 (depends on 1)

The system according to claim 1, wherein the first operations further comprise
repeatedly receiving indication of one or both of a current geographic location and a current view direction of a user;
receiving from the remote server the plurality of image tiles of the panoramic image of a predicted geographic location, wherein the predicted geographic location is located in proximity to the current geographic location; and
caching the received plurality of tiles in the first memory, wherein the processing is performed using the cached plurality of image tiles.

### Claim 4 (depends on 3)

The system according to claim 3, wherein the first operations further comprise decompressing the cached plurality of image tiles and storing the decompressed image tiles in the first memory, wherein the processing is performed using the decompressed tiles.

### Claim 5 (depends on 1)

The system according to claim 1, wherein the identifying comprises:
determining, from the received control information, a resolution of the panoramic image;
determining a total resolution of the received image tiles based upon the number of the received image tiles and the size of each received image tile; and
determining, at least by subtracting the resolution of the panoramic image from the determined total resolution, portions of the padding included along a length and/or width of respective ones of the received image tiles.

### Claim 6 (depends on 5)

The system according to claim 5, wherein the processing the received plurality of tiles comprises:
identifying a subset of the received image tiles, based upon their respective locations in the panoramic image, that include a portion of the padding; and
removing said padding in each received image tile in the identified subset in accordance with the determined portion of the padding included along the length and/or width of said received image tile.

### Claim 7 (depends on 1)

The system according to claim 1, wherein the generating the patch comprises forming the patch to include a number of said polygonal areas that is less than a configured maximum limit and same or greater than a configured minimum.

### Claim 8 (depends on 7)

The system according to claim 7, wherein the configured maximum limit is based upon dividing a maximum number of drawable polygons of a display device by the number of tiles.

### Claim 9 (depends on 8)

The system according to claim 8, wherein the generating the patch further comprises matching seams along a length and/or width of the patch with adjoining patches.

### Claim 10 (depends on 7)

The system according to claim 7, wherein the generating the patch further comprises forming the patch such that if the patch corresponds to an area at the north or south pole of the three-dimensional spherical surface, a concentration of points in the patch is higher than if the patch corresponds to an area in a middle of the three-dimensional spherical surface.

### Claim 11

A method, comprising:
receiving control information of a panoramic image and a plurality of image tiles of the panoramic image from a remote server through the network interface;
identifying, using at least the received control information, padding included in the received plurality of image tiles;
processing the received plurality of image tiles to remove the identified padding;
for each of the processed image tiles, generating at least one patch which maps at least a portion of said each processed image tile to a three-dimensional surface formed in the first memory, wherein each said generated patch comprises a plurality of polygonal areas, and wherein each said generated patch corresponds to a respective portion of a sphere; and
drawing the panoramic image using the generated patches.

### Claim 12 (depends on 11)

The method of claim 11, wherein the identifying comprises:
determining, from the received control information, a resolution of the panoramic image;
determining a total resolution of the received image tiles based upon the number of the received image tiles and the size of each received image tile; and
determining, at least by subtracting the resolution of the panoramic image from the determined total resolution, portions of the padding included along a length and/or width of respective ones of the received image tiles.

### Claim 13 (depends on 12)

The method of claim 12, wherein the processing the received plurality of tiles comprises:
identifying a subset of the received image tiles, based upon their respective locations in the panoramic image, that include a portion of the padding; and
removing said padding in each received image tile in the identified subset in accordance with the determined portion of the padding included along the length and/or width of said received image tile.

### Claim 14 (depends on 12)

The method of claim 12, wherein the padding includes a portion of the panoramic image in one of the received image tiles, wherein the portion overlaps with another portion in another one of the received image tiles.

### Claim 15 (depends on 11)

The method of claim 11, wherein the generating the patch comprises forming the patch to include a number of polygonal areas that is less than a configured maximum limit and same or greater than a configured minimum.

### Claim 16 (depends on 15)

The method of claim 15, wherein the configured maximum limit is based upon dividing a maximum number of drawable polygons of a display device by the number of tiles.

### Claim 17 (depends on 15)

The method of claim 15, wherein the generating the patch further comprises forming the patch such that if the patch corresponds to an area at the north or south pole of the spherical surface, a concentration of points in the patch is higher than if the patch corresponds to an area in a middle area of a three-dimensional spherical surface.

### Claim 18 (depends on 11)

The method of claim 11, further comprising:
storing the received plurality of image tiles in a cache, wherein
the processing is performed using the stored plurality of image tiles.

### Claim 19

A non-transitory computer-readable storage medium having instructions stored thereon, which when executed by a processor of a computer, causes the computer to perform operations comprising:
receiving control information of a panoramic image and a plurality of image tiles of the panoramic image from a remote server through the network interface;
identifying, using at least the received control information, padding included in the received plurality of image tiles;
processing the received plurality of image tiles to remove the identified padding;
for each of the processed image tiles, generating at least one patch which maps at least a portion of said each processed image tile to a three-dimensional surface formed in the first memory, wherein each said generated patch comprises a plurality of polygonal areas, and wherein each said generated patch corresponds to a respective portion of a sphere; and
drawing the panoramic image using the generated patches.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium according to claim 19, wherein the instructions cause the computer to perform operations further comprising:
storing the received plurality of image tiles in a cache, wherein
the processing is performed using the stored plurality of image tiles.