# US 2025/0085543 A1 · Video display system, information processing method, and recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0085543 A1
- Document ID: US20250085543A1
- Status: Published application
- Kind: A1
- Application number: 18/286,346
- Filed: 2022-04-18
- Published: 2025-03-13
- Earliest claimed: 2021-04-16
- Assignees: Panasonic Intellectual Property Management Co., Ltd.
- Applicants: Panasonic Intellectual Property Management Co., Ltd.
- Inventors: Masayuki KOZUKA; Yoshihiro MORI; Junya SUZUki; Yoshiichiro KASHIWAGI; Yasutoshi YAMAMOTO; Toshiro NISHO
- Web page: https://patentgrep.com/patent/US20250085543A1

## CPC

- G09G 5/377: for mixing or overlaying two or more graphic patterns (G09G5/02, G09G5/397 take precedence)
- G02B 27/0172: {characterised by optical features}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G06Q 10/40: Business processes related to social networking or social networking services
- G06Q 30/0643: {graphically representing goods, e.g. 3D product representation}
- G06Q 50/10: Services
- G06Q 90/20: {Destination assistance within a business structure or complex}
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 15/00: Three-dimensional [3D] image rendering
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G09G 5/36: characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
- H04N 7/183: {for receiving images from a single remote source}
- H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras}
- H04N 21/2353: {specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata}
- H04N 21/4312: {involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations}
- H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window}
- H04N 21/439: Processing of audio elementary streams
- H04N 21/472: End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content {(end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612)}
- H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- H04N 21/8106: {involving special audio data, e.g. different tracks for different languages}
- H04N 21/816: {involving special video data, e.g 3D video}
- G06T 2207/20221: Image fusion; Image merging
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A video display system includes: an observation device, a data obtainment unit that obtains data related to a movement direction of the capturing unit, and a metadata composition unit that obtains metadata based on the obtained data; and a VR device including: a reception unit that receives the metadata; an orientation estimation unit that estimates an orientation of the display device; a differential calculation unit that calculates a relative movement direction of the capturing unit based on a difference between the orientation of the display device and the movement direction of the capturing unit in the metadata; a presentation unit that presents the calculated relative movement direction to a user of a display device; a video generation unit that generates a display video; and a VR device including a display device that displays the display video.

## Claims

### Claim 1

A video display system for displaying a display video via a display device, the video display system comprising:
an observation device including a capturing unit configured to capture a wide-angle video, a data obtainment unit configured to obtain data related to a movement direction of the capturing unit, a metadata composition unit configured to obtain metadata based on the data obtained, and a transmission unit configured to transmit, together with the metadata, the wide-angle video captured; and
a VR device including:
a reception unit configured to receive the wide-angle video and the metadata;
an orientation estimation unit configured to estimate an orientation of the display device;
a differential calculation unit configured to calculate a relative movement direction based on a difference between the orientation of the display device estimated and the movement direction of the capturing unit in the metadata, the relative movement direction being a movement direction of the capturing unit relative to the orientation of the display device;
a presentation unit configured to present the relative movement direction calculated to a user of the display device;
a video generation unit configured to generate, from the wide-angle video received, the display video including a portion corresponding to a viewing area corresponding to the orientation of the display device as estimated by the orientation estimation unit; and
the display device that displays the display video.

### Claim 2 (depends on 1)

The video display system according to claim 1, wherein
the presentation unit is configured to:
generate and output a graphic indicating the relative movement direction calculated; and
cause the video generation unit to superimpose the graphic onto the portion to present the relative movement direction.

### Claim 3 (depends on 2)

The video display system according to claim 2, wherein
the graphic displays an arrow indicating the relative movement direction on the display video.

### Claim 4 (depends on 2)

The video display system according to claim 2, wherein
the graphic displays, as a mask, an image for covering at least part of the display video other than a part corresponding to the relative movement direction of the mobile object.

### Claim 5 (depends on 1)

The video display system according to claim 1, wherein
the orientation of the display device estimated is a discrete display direction that changes in units of a first discrete value based on an actual orientation of the display device, and
the differential calculation unit is configured to calculate a difference between the discrete display direction and the movement direction of the capturing unit in the metadata.

### Claim 6 (depends on 5)

The video display system according to claim 5, wherein
the first discrete value varies from a minimum when the actual orientation of the display device and an inclination in a horizontal plane in the movement direction of the capturing unit in the metadata are coincident, to a maximum when a difference between the actual orientation of the display device and the inclination in the horizontal plane in the movement direction of the capturing unit in the metadata is a maximum.

### Claim 7 (depends on 1)

The video display system according to claim 1, wherein
the movement direction of the capturing unit in the metadata is a discrete movement direction that changes in units of a second discrete value based on an actual orientation of the display device, and
the differential calculation unit is configured to calculate a difference between the orientation of the display device estimated and the discrete movement direction.

### Claim 8 (depends on 1)

The video display system according to claim 1, wherein
the observation device includes an input interface for input by an operator who moves the capturing unit, and
the data obtainment unit is configured to obtain the data input by the operator via the input interface.

### Claim 9 (depends on 1)

The video display system according to claim 1, wherein
the observation device includes a position detection unit configured to detect a position of the capturing unit, and
the data obtainment unit is configured to obtain the data based on the position of the capturing unit detected over time by the position detection unit.

### Claim 10 (depends on 1)

The video display system according to claim 1, wherein
the capturing unit is configured to capture the wide-angle video as a virtual video by capturing video in a virtual image space constructed by computer graphics.

### Claim 11 (depends on 1)

The video display system according to claim 1, further comprising
an information processing device that includes at least one of functions included in the observation device and the VR device, is connected to the observation device and the VR device over a network, and takes over one or more processes of the observation device or the VR device.

### Claim 12 (depends on 11)

The video display system according to claim 11, wherein
the information processing device includes:
a reception unit configured to receive the wide-angle video and the data from the observation device;
a metadata composition unit configured to generate metadata based on the wide-angle video and the data;
a movement information calculation unit configured to calculate movement information related to movement of the capturing unit in the metadata;
a presentation unit configured to generate and output a graphic that indicates the movement information calculated and presents, to a user of the display device, the movement direction of the capturing unit by being superimposed on a portion of the wide-angle video that corresponds to a viewing area corresponding to the orientation of the display device estimated; and
a transmission unit configured to transmit the wide-angle video, the graphic, and the metadata.

### Claim 13 (depends on 11)

The video display system according to claim 11, wherein
the information processing device includes:
a reception unit configured to receive the wide-angle video and the data from the observation device;
a metadata composition unit configured to generate metadata based on the wide-angle video and the data;
a movement information calculation unit configured to calculate movement information related to movement of the capturing unit in the metadata; and
a transmission unit configured to transmit the wide-angle video, the movement information, and the metadata.

### Claim 14 (depends on 11)

The video display system according to claim 11, wherein
the information processing device includes:
a reception unit configured to receive the wide-angle video, the data, and data related to the orientation of the display device;
a metadata composition unit configured to generate metadata based on the wide-angle video, the data, and the data related to the orientation of the display device;
a differential calculation unit configured to calculate a relative movement direction based on a difference between the orientation of the display device and movement information related to movement of the capturing unit, the relative movement direction being a movement direction of the capturing unit relative to the orientation of the display device;
a presentation unit configured to generate and output a graphic that indicates the relative movement direction calculated and presents, to a user of the display device, the relative movement direction by being superimposed on a portion of the wide-angle video that corresponds to a viewing area corresponding to the orientation of the display device estimated; and
a transmission unit configured to transmit the wide-angle video, the graphic, and the metadata.

### Claim 15 (depends on 11)

The video display system according to claim 11, wherein
the information processing device is provided in a cloud connected to a wide-area network, and is connected to the observation device and the VR device over the wide-area network.

### Claim 16

An information processing method for displaying a display video on a display device, the method comprising:
receiving metadata based on data related to a movement direction of a capturing unit configured to capture a wide-angle video; and
calculating and outputting a relative movement direction based on a difference between an estimated orientation of the display device and the movement direction of the capturing unit in the metadata, the relative movement direction being a movement direction of the capturing unit relative to the orientation of the display device.

### Claim 17 (depends on 16)

A non-transitory computer-readable recording medium having recorded thereon a computer program for causing a computer to execute the information processing method according to claim 16.
