# US 2022/0286756 A1 · Data processor and transport of user control data to audio decoders and renderers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0286756 A1
- Document ID: US20220286756A1
- Status: Published application
- Kind: A1
- Application number: 17/664,397
- Filed: 2022-05-20
- Published: 2022-09-08
- Earliest claimed: 2014-05-28
- Assignees: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
- Applicants: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
- Inventors: Stephan SCHREINER; Simone NEUKAM; Harald FUCHS; Jan PLOGSTIES; Stefan DOEHLA
- Web page: https://patentgrep.com/patent/US20220286756A1

## CPC

- H04N 21/8106: {involving special audio data, e.g. different tracks for different languages}
- G10L 19/00: Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis (in musical instruments G10H)
- G10L 19/167: {Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes}
- H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
- H04N 21/4363: Adapting the video stream to a specific local network, e.g. a Bluetooth® network
- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/44222: {Analytics of user selections, e.g. selection of programmes or purchase activity (monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33)}
- H04N 21/44227: {Monitoring of local network, e.g. connection or bandwidth variations; Detecting new devices in the local network (configuring of peripheral devices in general G06F9/4411; monitoring connectivity in data switched networks H04L43/0811)}
- H04N 21/4852: {for modifying audio parameters, e.g. switching between mono and stereo}
- H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Audio data processor, having: a receiver interface for receiving encoded audio data and metadata related to the encoded audio data; a metadata parser for parsing the metadata to determine an audio data manipulation possibility; an interaction interface for receiving an interaction input and for generating, from the interaction input, interaction control data related to the audio data manipulation possibility; and a data stream generator for obtaining the interaction control data and the encoded audio data and the metadata and for generating an output data stream, the output data stream having the encoded audio data, at least a portion of the metadata, and the interaction control data.

## Claims

### Claim 1

An audio data processor, comprising:
a receiver interface for receiving encoded audio data and metadata related to the encoded audio data;
a metadata parser for parsing the metadata to determine an audio data manipulation possibility;
an interaction interface for receiving an interaction input and for generating, from the interaction input, interaction control data related to the audio data manipulation possibility; and
a data stream generator for acquiring the interaction control data and the encoded audio data and the metadata and for generating an output data stream, the output data stream comprising the encoded audio data, at least a portion of the metadata, and the interaction control data.

### Claim 2 (depends on 1)

The audio data processor of claim 1, wherein the encoded audio data comprises separate encoded audio objects, wherein at least a portion of the metadata is related to a corresponding audio object,
wherein the metadata parser is configured to parse the corresponding portion for the encoded audio objects to determine, for at least an audio object, the object manipulation possibility,
wherein the interaction interface is configured to generate, for the at least one encoded audio object, the interaction control data from the interaction input related to the at least one encoded audio object.

### Claim 3 (depends on 1)

The audio data processor of claim 1, wherein the interaction interface is configured to present, to a user, the audio data manipulation possibility derived from the metadata by the metadata parser, and to receive, from the user, a user input on the specific data manipulation of the data manipulation possibility.

### Claim 4 (depends on 1)

The audio data processor of claim 1,
wherein the data stream generator is configured to process a data stream comprising the encoded audio data and the metadata received by the receiver interface without decoding the encoded audio data,
or to copy the encoded audio data and at least a portion of the metadata without changes in the output data stream,
wherein the data stream generator is configured to add an additional data portion comprising the interaction control data to the encoded audio data and/or the metadata in the output data stream.

### Claim 5 (depends on 1)

The audio data processor of claim 1,
wherein the data stream generator is configured to generate, in the output data stream, the interaction control data in the same format as the metadata.

### Claim 6 (depends on 1)

The audio data processor of claim 1,
wherein the data stream generator is configured to associate, with the interaction control data, an identifier in the output data stream, the identifier being different from an identifier associated with the metadata.

### Claim 7 (depends on 1)

The audio data processor of claim 1,
wherein the data stream generator is configured to add, to the interaction control data, signature data, the signature data indicating information on an application, a device or a user performing an audio data manipulation or providing the interaction input.

### Claim 8 (depends on 1)

The audio data processor of claim 1,
wherein the metadata parser is configured to identify a disabling possibility for one or more audio objects represented by the encoded audio data,
wherein the interaction interface is configured for receiving a disabling information for the one or more audio objects, and
wherein the data stream generator is configured for marking the one or more audio objects as disabled in the interaction control data or for removing the disabled one or more audio objects from the encoded audio data so that the output data stream does not comprise encoded audio data for the disabled one or more audio objects.

### Claim 9 (depends on 1)

The audio data processor of claim 1, wherein the data stream generator is configured to dynamically generate the output data stream, wherein in response to a new interaction input, the interaction control data is updated to match the new interaction input, and wherein the data stream generator is configured to comprise the updated interaction control data in the output data stream.

### Claim 10 (depends on 1)

The audio data processor of claim 1, wherein the receiver interface is configured to receive a main audio data stream comprising the encoded audio data and metadata related to the encoded audio data, and to additionally receive optional audio data comprising an optional audio object,
wherein the metadata related to said optional audio object is comprised in said main audio data stream.

### Claim 11 (depends on 1)

The audio data processor of claim 1,
wherein the metadata parser is configured to determine the audio manipulation possibility for a missing audio object not comprised in the encoded audio data,
wherein the interaction interface is configured to receive an interaction input for the missing audio object, and
wherein the receiver interface is configured to request audio data for the missing audio object from an audio data provider or to receive the audio data for the missing audio object from a different substream comprised in a broadcast stream or an internet protocol connection.

### Claim 12 (depends on 1)

The audio data processor of claim 1,
wherein the data stream generator is configured to assign, in the output data stream, a further packet type to the interaction control data, the further packet type being different from packet types for the encoded audio data and the metadata, or
wherein the data stream generator is configured to add, into the output data stream, fill data in a fill data packet type, wherein an amount of fill data is determined based on a data rate requirement determined by an output interface of the audio data processor.

### Claim 13 (depends on 1)

The audio data processor of claim 1, being implemented as a separate device, wherein the receiver interface forms an input to the separate device via a wired or wireless connection, wherein the audio data processor further comprises an output interface connected to the data stream generator, the output interface being configured for outputting the output data stream, wherein the output interface performs an output of the device and comprises a wireless interface or a wire connector.

### Claim 14

A method for processing audio data, the method comprising:
receiving encoded audio data and metadata related to the encoded audio data;
parsing the metadata to determine an audio data manipulation possibility;
receiving an interaction input and generating, from the interaction input, interaction control data related to the audio data manipulation possibility; and
acquiring the interaction control data and the encoded audio data and the metadata and generating an output data stream, the output data stream comprising the encoded audio data, at least a portion of the metadata, and the interaction control data.

### Claim 15

A non-transitory digital storage medium having stored thereon a computer program for performing a method for processing audio data, the method comprising:
receiving encoded audio data and metadata related to the encoded audio data;
parsing the metadata to determine an audio data manipulation possibility;
receiving an interaction input and generating, from the interaction input, interaction control data related to the audio data manipulation possibility; and
acquiring the interaction control data and the encoded audio data and the metadata and generating an output data stream, the output data stream comprising the encoded audio data, at least a portion of the metadata, and the interaction control data,
when said computer program is run by a computer.
