# US 8,279,296 B2 · Information processing apparatus, image-capturing system, reproduction control method, recording control method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,279,296 B2
- Document ID: US8279296B2
- Status: Granted
- Kind: B2
- Application number: 12/481,901
- Filed: 2009-06-10
- Published: 2009-12-17
- Granted: 2012-10-02
- Earliest claimed: 2008-06-11
- Assignees: Sony Corporation
- Inventors: Manabu Kimura; Tomoki Kanesaka; Yusuke Tani
- Web page: https://patentgrep.com/patent/US8279296B2

## CPC

- H04N 5/2624: {for obtaining an image which is composed of whole input images, e.g. splitscreen}
- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- H04N 9/3147: {Multi-projection systems (video walls G06F3/1446)}
- H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras}
- H04N 21/21815: {comprising local storage units}
- H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/2343: involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements {(video transcoding H04N19/40; media packet handling at the source H04L65/762)}
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An information processing apparatus includes a video obtaining unit configured to obtain a plurality of items of video data captured by a plurality of image-capturing devices; a position information obtaining unit configured to obtain position information including an orientation and an angle of view of each of the image-capturing devices; a reproduction position calculation unit configured to calculate a reproduction position of each item of the video data so that a projection direction and an angle of view of the item of the video data matches the orientation and the angle of view of a corresponding one of the image-capturing devices by using the position information; and a reproduction controller configured to reproduce each item of the video data at the corresponding reproduction position.

## Claims

### Claim 1

An information processing apparatus comprising:
a video obtaining unit configured to obtain a plurality of items of video data captured by a plurality of image-capturing devices;
a position information obtaining unit configured to obtain, position information including an orientation and an angle of view of each of the image-capturing devices;
a reproduction position calculation unit configured to calculate a reproduction position of each item of the video data so that a projection direction and an angle of view of the item of the video data matches the orientation and the angle of view of a corresponding one of the image-capturing devices by using the position information; and
a reproduction controller configured to reproduce each item of the video data at the corresponding reproduction position,
wherein to reproduce each item of the video data apex coordinates of a polygon, representing an item of the video data, are calculated on the basis of the position information and a viewpoint information of one of the plurality of image-capturing devices, and
wherein viewpoint information is information specified by a user and describes which viewpoint the reproduced video should be viewed.

### Claim 2 (depends on 1)

The information processing apparatus according to claim 1,
wherein the reproduction position calculation unit calculates a plane area on which a corresponding item of the video, data is projected as the reproduction position in a virtual three-dimensional space including the depth direction of a display screen, and
wherein the reproduction controller reproduces each item of the video data in such a manner that the item of the video data is projected on the corresponding plane area in the virtual three-dimensional space, the plane area being calculated as the corresponding reproduction position.

### Claim 3 (depends on 1)

The information processing apparatus according to claim 1, further comprising:
a direction controller configured to control the orientations of the plurality of image-capturing devices so that portions of image-capturing ranges thereof overlap each other;
an angle-of-view adjustment unit configured to adjust the angle of view of each of the image-capturing devices;
a position information recorder configured to record, as position information, the orientation of each of the image-capturing devices controlled by the direction controller and the angle of view of each of the image-capturing devices adjusted by the angle-of-view adjustment unit; and
a storage unit in which video data captured by each of the image-capturing devices and the position information are recorded,
wherein the video obtaining unit obtains the video data from the storage unit; and the position information obtaining unit obtains the position information from the storage unit.

### Claim 4

An information processing apparatus comprising:
a direction controller configured to control the orientations of a plurality of image-capturing devices so that portions of an image-capturing range overlap each other;
an angle-of-view adjustment unit configured to adjust the angle of view of each of the image-capturing devices;
a position information recorder configured to record, as position information; the orientation of each of the image-capturing devices controlled by the direction controller and the angle of view of each of the image-capturing devices adjusted by the angle-of-view adjustment unit; and
a storage unit in which video data captured by each of the image-capturing devices and the position information are recorded,
wherein to reproduce each item of the video data, apex coordinates of a polygon, representing an item of the video data, are calculated on the basis of the position information and a viewpoint information of one of the plurality of image-capturing devices, and
wherein viewpoint information is information is specified by a user and describes which viewpoint the reproduced video should be viewed.

### Claim 5

An image-capturing system comprising:
a plurality of image-capturing units capable of changing their individual image-capturing directions and angles of view;
a direction controller configured to control the orientation of each of the plurality of image-capturing units so that portions of image-capturing ranges thereof overlap each other;
an angle-of-view adjustment unit configured to adjust the angle of view of each of the image-capturing units;
a position, information recorder configured to record, as position information, the orientation of each of the image-capturing units controlled by the direction controller and the angle of view of each of the image-capturing units adjusted by the angle-of-view adjustment unit; and
a storage unit in which video data captured by each of the image-capturing units and the position information are recorded,
wherein to reproduce each item of the video data, apex coordinates of a polygon, representing an item of the video data, are calculated on the basis of the position information and a viewpoint information of one of the plurality of image-capturing devices, and
wherein viewpoint information is information is specified by a user and describes which viewpoint the reproduced video should be viewed.

### Claim 6

A reproduction control method comprising the steps of:
obtaining a plurality of items of video data captured by a plurality of image-capturing devices;
obtaining position information including an orientation and an angle of view of each of the image-capturing devices;
calculating the reproduction position of each item of video data so that the projection direction and the angle of view of the item of the video data matches the orientation and the angle of view of a corresponding one of the image-capturing devices by using the position information; and
performing control so that each item of the video data is reproduced at the corresponding reproduction position,
wherein to reproduce each item of the video data, apex coordinates of a polygon, representing an item of the video data, are calculated on the basis of the position information and a viewpoint information of one of the plurality of image-capturing devices, and
wherein viewpoint information is information is specified by a user and describes which viewpoint the reproduced video should be viewed.

### Claim 7

A recording control method comprising the steps of:
controlling orientations of a plurality of image-capturing devices so that portions of image-capturing ranges overlap each other;
adjusting the angle of view of each of the image-capturing devices;
recording, as position information, the orientation of each of the image-capturing devices controlled in the step of controlling orientations and the angle of view of each of the image-capturing devices adjusted in the step of adjusting an angle of view; and
recording video data captured by each of the image-capturing devices,
wherein to reproduce each item of the video data, apex coordinates of a polygon, representing an item of the video data, are calculated on the basis of the position information and a viewpoint information of one of the plurality of image-capturing devices, and
wherein viewpoint information is information is specified by a user and describes which viewpoint the reproduced video should be viewed.

### Claim 8

A non-transitory computer readable medium for storing a program for causing a computer to implement, a reproduction control method comprising the steps of:
obtaining a plurality of items of video data captured by a plurality of image-capturing devices;
obtaining position information including an orientation and an angle of view of each of the image-capturing devices;
calculating the reproduction position of each item of video data so that the projection direction and the angle of view of the item of the video data matches the orientation and the angle of view of a corresponding one of the image-capturing devices by using the position information; and
performing control so that each item of the video data is reproduced at the corresponding reproduction position,
wherein to reproduce each item of the video data, apex coordinates of a polygon, representing an item of the video data, are calculated on the basis of the position information and a viewpoint information of one of the plurality of image-capturing devices, and
wherein viewpoint information is information is specified by a user and describes which viewpoint the reproduced video should be viewed.

### Claim 9

A non-transitory computer readable medium for storing a program for causing a computer to implement a recording control method comprising the steps of:
controlling orientations of a plurality of image-capturing devices so that portions of image-capturing ranges overlap each other;
adjusting the angle of view of each of the image-capturing devices;
recording, as position information, the orientation of each of the image-capturing devices controlled in the step of controlling orientations and the angle of view of each of the image capturing, devices adjusted in the step of adjusting an angle of view; and
recording video data captured by each of the image-capturing devices,
wherein to reproduce each item of the video data, apex coordinates of a polygon, representing an item of the video data, are calculated on the basis of the position information and a viewpoint information of one of the plurality of image-capturing devices, and
wherein viewpoint information is information is specified by a user and describes which viewpoint the reproduced video should be viewed.
