# US 12,217,366 B2 · Media distribution device and media distribution method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,366 B2
- Document ID: US12217366B2
- Status: Granted
- Kind: B2
- Application number: 17/998,500
- Filed: 2021-05-11
- Published: 2023-07-06
- Granted: 2025-02-04
- Earliest claimed: 2020-05-25
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: Toshiya Hamada; Takumi Tsuru
- Web page: https://patentgrep.com/patent/US12217366B2

## CPC

- G06T 19/003: {Navigation within 3D models or images}
- G10L 13/027: Concept to speech synthesisers; Generation of natural phrases from machine-based concepts (generation of parameters for speech synthesis out of text G10L13/08)
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 15/20: Perspective computation
- G10L 13/04: Details of speech synthesis systems, e.g. synthesiser structure or memory management
- G10L 19/167: {Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes}
- H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/234: Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- G06T 2210/61: Scene description
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present disclosure relates to a media distribution device, a media distribution method, and a program enabling to generate a guide voice more appropriately. A media distribution device includes a guide voice generation unit that generates a guide voice describing a rendered image viewed from a viewpoint in a virtual space by using a scene description as information describing a scene in the virtual space and a user viewpoint information indicating a position and a direction of the viewpoint of a user; and an audio encoding unit that mixes the guide voice with original audio, and encodes the guide voice. The present technology can be applied to, for example, a media distribution system that distributes 6DoF media.

## Claims

### Claim 1

A media distribution device, comprising:
a rendering processing unit configured to generate a rendered image;
a guide voice generation unit configured to generate a guide voice that describes the rendered image viewed from a viewpoint in a virtual space, wherein
the guide voice is generated based on a scene description and a user viewpoint information,
the scene description describes a scene in the virtual space,
the user viewpoint information indicates a position and a direction of the viewpoint of a user, and
the guide voice generation unit includes:
an encoding parameter generation unit configured to generate an encoding parameter based on information supplied from the rendering processing unit;
a scene description text generation unit configured to generate a description text based on the rendered image and the encoding parameter, wherein the description text includes the scene description that describes the scene in the virtual space; and
a voice conversion unit configured to convert the description text into a voice; and
an audio encoding unit configured to:
mix the guide voice with original audio; and
encode the mixed guide voice.

### Claim 2 (depends on 1)

The media distribution device according to claim 1, wherein
the encoding parameter generation unit is further configured to generate a QP map as the encoding parameter, and
the QP map represents a distribution of interest degrees based on a resolution of the rendered image.

### Claim 3 (depends on 1)

The media distribution device according to claim 1, wherein
the encoding parameter generation unit is further configured to generate a motion vector map as the encoding parameter, and
the motion vector map represents a distribution of motions of an object in the rendered image.

### Claim 4

A media distribution method, comprising:
in a media distribution device;
generating a rendered image;
generating an encoding parameter based on the rendered image;
generating a description text based on the rendered image and the encoding parameter, wherein the description text includes scene description that describes a scene in a virtual space;
converting the description text into a voice;
generating a guide voice based on the voice, wherein the guide voice describes the rendered image viewed from a viewpoint in the virtual space, wherein
the guide voice is generated based on the scene description and a user viewpoint information,
the scene description describes the scene in the virtual space, and
the user viewpoint information indicates a position and a direction of the viewpoint of a user;
mixing the guide voice with original audio; and
encoding the mixed guide voice.

### Claim 5

A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a computer, cause the computer to execute operations, the operations comprising:
generating a rendered image;
generating an encoding parameter based on the rendered image;
generating a description text based on the rendered image and the encoding parameter, wherein the description text includes scene description that describes a scene in a virtual space;
converting the description text into a voice;
generating a guide voice based on the voice, wherein the guide voice describes the rendered image viewed from a viewpoint in the virtual space, wherein
the quide voice is generated based on the by using a scene description and a user viewpoint information,
the scene description describes the as information describing a scene in the virtual space, and
the user viewpoint information indicating indicates a position and a direction of the viewpoint of a user;
mixing the guide voice with original audio; and
encoding the mixed guide voice.