Methods and systems for generating and interactively rendering object based audio

A method of rendering audio content determined by an object based audio program, wherein the program is indicative of a bed of speaker channels, a set of M replacement speaker channels, a set of object channels, and metadata, wherein the set of object channels consists of N object channels, a first subset of the set of object channels is indicative of first non-ambient content, the first subset consists of M object channels of the set of object channels, each of N and M is an integer greater than zero, and M is equal to or less than N,

• the bed of speaker channels is indicative of a default mix of audio content, including second non-ambient content different than the first non-ambient content, where an object based speaker channel subset consisting of M of the speaker channels of the bed is indicative of the second non-ambient content or a mix of the second non-ambient content and at least some other audio content of the audio content of the default mix,
• each replacement speaker channel in the set of M replacement speaker channels is indicative of none or some, but not all, of the content of a corresponding speaker channel of the object based speaker channel subset, and
• the metadata is indicative of at least one selectable predetermined alternative mix of content of at least one of the object channels and content of predetermined ones of the speaker channels of the bed and/or the replacement speaker channels, where the metadata includes rendering parameters for each said alternative mix, and at least one said alternative mix is a replacement mix including at least some of the audio content of the bed and the first non-ambient content, but not the second non-ambient content, said method including steps of:
• (a) providing the object based audio program to an audio processing unit;
• and
• (b) in the audio processing unit, parsing the bed of speaker channels and rendering the default mix in response to the bed of speaker channels without use of the metadata.

A system for generating an object based audio program indicative of audio content including first non-ambient content, second non-ambient content different than the first non-ambient content, and third content different than the first non-ambient content and the second non-ambient content, said system including:

• a first subsystem configured to determine:
• a set of object channels consisting of N object channels, where a first subset of the set of object channels is indicative of the first non-ambient content, the first subset consists of M object channels of the set of object channels, each of N and M is an integer greater than zero, and M is equal to or less than N,
• a bed of speaker channels indicative of a default mix of audio content, where an object based speaker channel subset consisting of M of the speaker channels of the bed is indicative of the second non-ambient content or a mix of the second non-ambient content and at least some other audio content of the audio content of the default mix, and
• a set of M replacement speaker channels, where each replacement speaker channel in the set of M replacement speaker channels is indicative of none or some, but not all, of the content of a corresponding speaker channel of the object based speaker channel subset, wherein the first subsystem is also configured to generate metadata indicative of at least one selectable predetermined alternative mix of content of at least one of the object channels and content of predetermined ones of the speaker channels of the bed and/or the replacement speaker channels, where the metadata includes rendering parameters for each said alternative mix, and at least one said alternative mix is a replacement mix indicative of at least some of the audio content of the bed and the first non-ambient content, but not the second non-ambient content;
• and
• an encoding subsystem coupled to the first subsystem and configured to generate the object based audio program, such that said object based audio program includes the bed of speaker channels, the set of M replacement speaker channels, the set of object channels, and the metadata, and such that the bed of speaker channels is renderable without use of the metadata to provide sound perceivable as the default mix, and the replacement mix is renderable, in response to at least some of the metadata, to provide sound perceivable as a mix including said at least some of the audio content of the bed and the first non-ambient content but not the second non-ambient content.

An audio processing unit configured to render audio content determined by an object based audio program, wherein the program is indicative of a bed of speaker channels, a set of M replacement speaker channels, a set of object channels, and metadata, wherein the set of object channels consists of N object channels, a first subset of the set of object channels is indicative of first non-ambient content, the first subset consists of M object channels of the set of object channels, each of N and M is an integer greater than zero, and M is equal to or less than N,

• the bed of speaker channels is indicative of a default mix of audio content, including second non-ambient content different than the first non-ambient content, where an object based speaker channel subset consisting of M of the speaker channels of the bed is indicative of the second non-ambient content or a mix of the second non-ambient content and at least some other audio content of the audio content of the default mix,
• each replacement speaker channel in the set of M replacement speaker channels is indicative of none or some, but not all, of the content of a corresponding speaker channel of the object based speaker channel subset, and
• the metadata is indicative of at least one selectable predetermined alternative mix of content of at least one of the object channels and content of predetermined ones of the speaker channels of the bed and/or the replacement speaker channels, where the metadata includes rendering parameters for each said alternative mix, and at least one said alternative mix is a replacement mix including at least some of the audio content of the bed and the first non-ambient content, but not the second non-ambient content, said audio processing unit including:
• a first subsystem coupled to receive the object based audio program and configured to parse the bed of speaker channels, the replacement speaker channels, the object channels, and the metadata of the program;
• and
• a rendering subsystem, coupled to the first subsystem and operable in a first mode to render the default mix in response to the bed of speaker channels without using the metadata, wherein the rendering subsystem is also operable in a second mode to render the replacement mix using at least some of the metadata, including by selecting, and mixing content of, the first subset of the set of object channels and at least one said replacement speaker channel in response to at least some of the metadata.

An audio processing unit, including:

• a buffer memory;
• and
• at least one audio processing subsystem coupled to the buffer memory, wherein the buffer memory stores at least one segment of an object based audio program, wherein the program is indicative of a bed of speaker channels, a set of M replacement speaker channels, a set of object channels, and metadata, wherein the set of object channels consists of N object channels, a first subset of the set of object channels is indicative of first non-ambient content, the first subset consists of M object channels of the set of object channels, each of N and M is an integer greater than zero, and M is equal to or less than N,
• the bed of speaker channels is indicative of a default mix of audio content, including second non-ambient content different than the first non-ambient content, where an object based speaker channel subset consisting of M of the speaker channels of the bed is indicative of the second non-ambient content or a mix of the second non-ambient content and at least some other audio content of the audio content of the default mix,
• each replacement speaker channel in the set of M replacement speaker channels is indicative of none or some, but not all, of the content of a corresponding speaker channel of the object based speaker channel subset, and
• the metadata is indicative of at least one selectable predetermined alternative mix of content of at least one of the object channels and content of predetermined ones of the speaker channels of the bed and/or the replacement speaker channels, where the metadata includes rendering parameters for each said alternative mix, and at least one said alternative mix is a replacement mix including at least some of the audio content of the bed and the first non-ambient content, but not the second non-ambient content,
• and wherein each said segment includes data indicative of audio content of the bed of speaker channels, data indicative of audio content of the replacement speaker channels, and data indicative of audio content of the object channels, and at least a part of the metadata.