# US 10,375,131 B2 · Selectively transforming audio streams based on audio energy estimate > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,375,131 B2 - Document ID: US10375131B2 - Status: Granted - Kind: B2 - Application number: 15/599,677 - Filed: 2017-05-19 - Published: 2018-11-22 - Granted: 2019-08-06 - Earliest claimed: 2017-05-19 - Assignees: Cisco Technology, Inc. - Applicants: Cisco Technology, Inc. - Inventors: Marcello Caramma - Web page: https://patentgrep.com/patent/US10375131B2 ## CPC - H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) - G10L 19/167: {Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes} - H04L 65/765: {intermediate} - H04W 88/02: Terminal devices - G10L 19/012: Comfort noise or silence coding - G10L 19/0212: {using orthogonal transformation} - H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A server receives, from each of a plurality of participant devices in a communication session, a respective one of a plurality of audio streams. The server estimates an audio energy of each of the plurality of audio streams and determines whether to perform a transform on at least one of the plurality of audio streams. If so, the server performs the transform on the at least one of the plurality of audio streams and transmits the at least one of the plurality of audio streams to at least one of the plurality of participant devices. ## Claims ### Claim 1 A method comprising: receiving, from each of a plurality of participant devices in a communication session, a respective one of a plurality of audio streams; estimating an audio energy of each of the plurality of audio streams; based on the estimated audio energy of each of the plurality of audio streams, determining whether to perform a transform on at least one of the plurality of audio streams, the determining including determining whether performing the transform would require a processing load that is less than a processing load threshold; and if it is determined to perform the transform on the at least one of the plurality of audio streams: performing the transform on the at least one of the plurality of audio streams; and transmitting the at least one of the plurality of audio streams to at least one of the plurality of participant devices. ### Claim 2 (depends on 1) The method of claim 1, wherein the audio streams are modified discrete cosine transformed frequency domain audio streams, the method further comprising: partially decoding the plurality of audio streams so as to determine modified discrete cosine transform frequency coefficients corresponding to each of the plurality of audio streams, wherein: estimating includes estimating the audio energy of each of the plurality of audio streams based on the coefficients; determining includes determining whether to perform an inverse modified discrete cosine transform on the at least one of the plurality of audio streams; and performing includes performing the inverse modified discrete cosine transform on the at least one of the plurality of audio streams. ### Claim 3 (depends on 2) The method of claim 2, wherein determining whether to perform the inverse modified discrete cosine transform on the at least one of the plurality of audio streams further includes: determining whether the at least one of the plurality of audio streams among the plurality of audio streams has a highest estimated audio energy. ### Claim 4 (depends on 2) The method of claim 2, wherein determining whether to perform the inverse modified discrete cosine transform on the at least one of the plurality of audio streams further includes: determining whether the at least one of the plurality of audio streams has an estimated audio energy that exceeds an audio energy threshold. ### Claim 5 (depends on 1) The method of claim 1, wherein: estimating includes estimating the audio energy of each of the plurality of audio streams in the time domain; determining includes determining whether to perform a modified discrete cosine transform on the at least one of the plurality of audio streams; and performing includes performing the modified discrete cosine transform on the at least one of the plurality of audio streams. ### Claim 6 (depends on 5) The method of claim 5, further comprising: if it is determined not to perform the transform on an audio stream of the plurality of audio streams, replacing the audio stream with comfort noise. ### Claim 7 (depends on 1) The method of claim 1, wherein the audio streams are modified discrete cosine transformed frequency domain audio streams, and wherein: performing includes performing the inverse modified discrete cosine transform on the at least one of the plurality of audio streams. ### Claim 8 An apparatus comprising: a network interface configured to enable communications over a network in order to receive, from each of a plurality of participant devices in a communication session, a respective one of a plurality of audio streams; and one or more processors coupled to the network interface, wherein the one or more processors are configured to: estimate an audio energy of each of the plurality of audio streams; based on the estimated audio energy of each of the plurality of audio streams, determine whether to perform a transform on at least one of the plurality of audio streams, including determining whether performing the transform would require a processing load that is less than a processing load threshold; and if it is determined to perform the transform on the at least one of the plurality of audio streams: perform the transform on the at least one of the plurality of audio streams; and transmit the at least one of the plurality of audio streams to at least one of the plurality of participant devices. ### Claim 9 (depends on 8) The apparatus of claim 8, wherein the audio streams are modified discrete cosine transformed frequency domain audio streams, the one or more processors further configured to: partially decode the plurality of audio streams so as to determine modified discrete cosine transform frequency coefficients corresponding to each of the plurality of audio streams, wherein the one or more processors are configured to: estimate by estimating the audio energy of each of the plurality of audio streams based on the coefficients; determine by determining whether to perform an inverse modified discrete cosine transform on the at least one of the plurality of audio streams; and perform by performing the inverse modified discrete cosine transform on the at least one of the plurality of audio streams. ### Claim 10 (depends on 9) The apparatus of claim 9, wherein the one or more processors are further configured to determine whether to perform the inverse modified discrete cosine transform on the at least one of the plurality of audio streams by: determining whether the at least one of the plurality of audio streams among the plurality of audio streams has a highest estimated audio energy. ### Claim 11 (depends on 9) The apparatus of claim 9, wherein the one or more processors are further configured to determine whether to perform the inverse modified discrete cosine transform on the at least one of the plurality of audio streams by: determining whether the at least one of the plurality of audio streams has an estimated audio energy that exceeds an audio energy threshold. ### Claim 12 (depends on 8) The apparatus of claim 8, wherein the one or more processors are configured to: estimate by estimating the audio energy of each of the plurality of audio streams in the time domain; determine by determining whether to perform a modified discrete cosine transform on the at least one of the plurality of audio streams; and perform by performing the modified discrete cosine transform on the at least one of the plurality of audio streams. ### Claim 13 (depends on 12) The apparatus of claim 12, wherein the one or more processors are further configured to: if it is determined not to perform the transform on an audio stream of the plurality of audio streams, replace the audio stream with comfort noise. ### Claim 14 (depends on 8) The apparatus of claim 8, wherein the audio streams are modified discrete cosine transformed frequency domain audio streams, and wherein the one or more processors are configured to: perform by performing the inverse modified discrete cosine transform on the at least one of the plurality of audio streams. ### Claim 15 One or more non-transitory computer readable storage media encoded with instructions that, when executed by a processor, cause the processor to: obtain, from each of a plurality of participant devices in a communication session, a respective one of a plurality of audio streams; estimate an audio energy of each of the plurality of audio streams; based on the estimated audio energy of each of the plurality of audio streams, determine whether to perform a transform on at least one of the plurality of audio streams, including determining whether performing the transform would require a processing load that is less than a processing load threshold; and if it is determined to perform the transform on the at least one of the plurality of audio streams: perform the transform on the at least one of the plurality of audio streams; and transmit the at least one of the plurality of audio streams to at least one of the plurality of participant devices. ### Claim 16 (depends on 15) The non-transitory computer readable storage media of claim 15, wherein the audio streams are modified discrete cosine transformed frequency domain audio streams, the instructions further causing the processor to: partially decode the plurality of audio streams so as to determine modified discrete cosine transform frequency coefficients corresponding to each of the plurality of audio streams, wherein the instructions cause the processor to: estimate by estimating the audio energy of each of the plurality of audio streams based on the coefficients; determine by determining whether to perform an inverse modified discrete cosine transform on the at least one of the plurality of audio streams; and perform by performing the inverse modified discrete cosine transform on the at least one of the plurality of audio streams. ### Claim 17 (depends on 16) The non-transitory computer readable storage media of claim 16, wherein the instructions that cause the processor to determine whether to perform the inverse modified discrete cosine transform on the at least one of the plurality of audio streams cause the processor to: determine whether the at least one of the plurality of audio streams among the plurality of audio streams has a highest estimated audio energy. ### Claim 18 (depends on 16) The non-transitory computer readable storage media of claim 16, wherein the instructions that cause the processor to determine whether to perform the inverse modified discrete cosine transform on the at least one of the plurality of audio streams cause the processor to: determine whether the at least one of the plurality of audio streams has an estimated audio energy that exceeds an audio energy threshold. ### Claim 19 (depends on 15) The non-transitory computer readable storage media of claim 15, the instructions further causing the processor to: estimate by estimating the audio energy of each of the plurality of audio streams in the time domain; determine by determining whether to perform a modified discrete cosine transform on the at least one of the plurality of audio streams; and perform by performing the modified discrete cosine transform on the at least one of the plurality of audio streams. ### Claim 20 (depends on 15) The non-transitory computer readable storage media of claim 15, wherein the audio streams are modified discrete cosine transformed frequency domain audio streams, and wherein the instructions cause the processor to: perform by performing the inverse modified discrete cosine transform on the at least one of the plurality of audio streams.