# US 2019/0174109 A1 · Camerawork generating method and video processing device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0174109 A1
- Document ID: US20190174109A1
- Status: Published application
- Kind: A1
- Application number: 16/270,931
- Filed: 2019-02-08
- Published: 2019-06-06
- Earliest claimed: 2016-08-10
- Assignees: Panasonic Intellectual Property Corporation of America
- Applicants: Panasonic Intellectual Property Corporation of America
- Inventors: Satoshi YOSHIKAWA; Toshiyasu SUGIO; Toru MATSUNOBU; Tatsuya KOYAMA; Shuuhei SASAKURA; Chisato YAMASHITA
- Web page: https://patentgrep.com/patent/US20190174109A1

## CPC

- H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
- G06T 15/20: Perspective computation
- H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
- H04N 13/243: using three or more two-dimensional [2D] image sensors
- H04N 13/275: from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals
- H04N 13/279: the virtual viewpoint locations being selected by the viewers or determined by tracking
- H04N 21/218: Source of audio or video content {, e.g. local disk arrays (details of retrieval in video databases G06F16/739)}
- H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras}
- H04N 21/234: Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A camerawork generating method includes: generating, from a multi-view video of a same scene captured from different viewpoints, a three-dimensional model for the scene; determining a target scene which is included in the scene and is a target for which a free-viewpoint video of the three-dimensional model as viewed from a virtual camera is to be generated; and generating a camerawork representing a temporal change in a position and an orientation of the virtual camera in the free-viewpoint video.

## Claims

### Claim 1

A camerawork generating method, comprising:
generating, from a multi-view video of a same scene captured from different viewpoints, a three-dimensional model for the scene;
determining a target scene that is a target for which a free-viewpoint video of the three-dimensional model as viewed from a virtual camera is to be generated, the target scene being included in the scene; and
generating a camerawork representing a temporal change in a position and an orientation of the virtual camera in the free-viewpoint video.

### Claim 2 (depends on 1)

The camerawork generating method according to claim 1, wherein
in the determining of the target scene, the target scene is determined using one of the multi-view video and the three-dimensional model.

### Claim 3 (depends on 1)

The camerawork generating method according to claim 1, wherein
in the determining of the target scene, the target scene is determined using sound of the scene.

### Claim 4 (depends on 1)

The camerawork generating method according to claim 1, wherein
in the generating of the camerawork, the position and the orientation of the virtual camera are determined such that an object of a predetermined type associated with the target scene is included in the free-viewpoint video.

### Claim 5 (depends on 1)

The camerawork generating method according to claim 1, wherein
in the generating of the camerawork, the position and the orientation of the virtual camera are determined based on precisions of a plurality of portions included in the three-dimensional model.

### Claim 6 (depends on 1)

The camerawork generating method according to claim 1, further comprising:
displaying the camerawork.

### Claim 7 (depends on 1)

The camerawork generating method according to claim 1, further comprising:
generating the free-viewpoint video from one of the three-dimensional model and the multi-view video, using the camerawork.

### Claim 8 (depends on 7)

The camerawork generating method according to claim 7, wherein
in the generating of the free-viewpoint video, a shake is added to the free-viewpoint video.

### Claim 9 (depends on 1)

The camerawork generating method according to claim 1, further comprising:
acquiring an evaluation of the camerawork by a user, wherein
in the generating of the camerawork, the camerawork is generated based on the evaluation.

### Claim 10

A video processing device, comprising:
a model generation unit that generates, from a multi-view video of a same scene captured from different viewpoints, a three-dimensional model for the scene;
an analyzer that determines a target scene that is a target for which a free-viewpoint video of the three-dimensional model as viewed from a virtual camera is to be generated, the target scene being included in the scene; and
a camerawork generation unit that generates a camerawork representing a temporal change in a position and an orientation of the virtual camera in the free-viewpoint video.