# US 11,308,975 B2 · Mixing device, mixing method, and non-transitory computer-readable recording medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,975 B2 - Document ID: US11308975B2 - Status: Granted - Kind: B2 - Application number: 17/047,504 - Filed: 2019-04-11 - Published: 2021-05-20 - Granted: 2022-04-19 - Earliest claimed: 2018-04-17 - Assignees: The University of Electro-Communications; Hibino Corporation - Applicants: The University of Electro-Communications; Hibino Corporation - Inventors: Kota Takahashi; Tsukasa Miyamoto; Yoshiyuki Ono; Yoji Abe; Hiroshi Inoue - Web page: https://patentgrep.com/patent/US11308975B2 ## CPC - G10L 21/0332: involving modification of waveforms - G10L 21/0364: for improving intelligibility - H04R 3/00: Circuits for transducers (arrangements for producing a reverberation or echo sound G10K15/08; amplifiers H03F) - H04R 25/43: {Electronic input selection or mixing based on input signal analysis, e.g. mixing or selection between microphone and telecoil or between microphones with different directivity characteristics (H04R25/407 takes precedence)} - H04S 1/007: {in which the audio signals are in digital form (data reduction aspects thereof based on psychoacoustics G10L19/02)} - G10H 1/46: Volume control - G10H 7/008: {Means for controlling the transition from one tone waveform to another} - H03G 3/20: Automatic control ({H03G3/005 takes precedence;} combined with volume compression or expansion H03G7/00) - H04H 60/04: Studio equipment; Interconnection of studios - H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)} - H04R 2227/009: Signal processing in [PA] systems to enhance the speech intelligibility - H04R 2420/01: Input selection or mixing for amplifiers or loudspeakers - H04S 7/00: Indicating arrangements; Control arrangements, e.g. balance control - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - H04S: STEREOPHONIC SYSTEMS - G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE - H03G: CONTROL OF AMPLIFICATION - H04H: BROADCAST COMMUNICATION (multiplex communication H04J; pictorial communication aspects of broadcast systems H04N) ## Abstract A mixing device of a first signal and a second signal on a time-frequency plane, includes a control signal generation unit to generate a control signal indicating whether to perform prioritized mixing that includes amplification of the first signal and attenuation of the second signal; and a gain derivation unit to derive a first gain for amplifying the first signal and a second gain for attenuating the second signal based on the control signal. The control signal takes at least a first value and a second value different from the first value, wherein the first value is not continuous beyond a predetermined bandwidth on a frequency axis. The mixing device applies the prioritized mixing to the first and second signals when the control signal indicates the first value, and applies simple addition to the first and second signals when the control signal indicates the second value. ## Claims ### Claim 1 A mixing device of a first signal and a second signal on a time-frequency plane, comprising: a processor that generates a control signal indicating whether to perform prioritized mixing that includes amplification of the first signal and attenuation of the second signal, and derives a first gain for amplifying the first signal and a second gain for attenuating the second signal based on the control signal, wherein the control signal takes at least a first value and a second value different from the first value, wherein the first value is not continuous beyond a predetermined bandwidth on a frequency axis, wherein the mixing device applies the prioritized mixing to the first signal and the second signal in response to the control signal indicating the first value, and applies simple addition to the first signal and the second signal in response to the control signal indicating the second value, and wherein the processor performs first frequency processing with respect to an intensity of the first signal on the time-frequency plane, to obtain a first spectrum representing an absolute amount of the first signal, second frequency processing with respect to the first spectrum, to obtain a second spectrum indicating local changes of the first spectrum, and signal generation to generate the control signal based on the first spectrum and the second spectrum. ### Claim 2 (depends on 1) The mixing device as claimed in claim 1, wherein the processor performs threshold value processing with respect to the first spectrum and the second spectrum, and outputs a smaller one of a first threshold value processing result with respect to the first spectrum and a second threshold value processing result with respect to the second spectrum. ### Claim 3 (depends on 2) The mixing device as claimed in claim 2, wherein the processor generates a first control signal that takes the first value in response to the first spectrum being greater than or equal to the first threshold value and takes the second value in response to the second spectrum being less than the second threshold value, and a second control signal that takes the first value in response to the second spectrum being greater than or equal to a third threshold value and takes the second value in response to the second spectrum being less than a fourth threshold value, and outputs a smaller one of the first control signal and the second control signal as the control signal. ### Claim 4 (depends on 1) The mixing device as claimed in claim 1, wherein the processor switches on and off application of the first spectrum and the second spectrum depending on states of the first signal and the second signal. ### Claim 5 (depends on 4) The mixing device as claimed in claim 4, wherein the processor uses only the second spectrum in response to the first signal being silent or weak. ### Claim 6 (depends on 4) The mixing device as claimed in claim 4, further comprising: a user interface configured to allow a user to select whether to perform application of the first spectrum and application of the second spectrum, wherein the processor performs or does not perform the application of the first spectrum and the second spectrum according to input through the user interface. ### Claim 7 (depends on 1) The mixing device as claimed in claim 1, further comprising: a bandpass filter configured to repeatedly invert an intensity signal of the first signal on the time-frequency plane in a frequency direction, to transmit the intensity signal, wherein the processor generates the control signal based on output of the bandpass filter. ### Claim 8 (depends on 1) The mixing device as claimed in claim 1, wherein the processor transforms a linear frequency axis to an axis based on a sense of hearing, to generate the control signal on the axis based on the sense of hearing. ### Claim 9 (depends on 1) The mixing device as claimed in claim 1, wherein the control signal takes a third value between the first value and the second value, and wherein a degree of the prioritized mixing is adjusted depending on the third value. ### Claim 10 A mixing method of a first signal and a second signal on a time-frequency plane, the mixing method comprising: generating a control signal indicating whether to perform prioritized mixing that includes amplification of the first signal and attenuation of the second signal, the control signal taking at least a first value and a second value different from the first value, wherein the first value is not continuous beyond a predetermined bandwidth on a frequency axis; deriving a first gain for amplifying the first signal and a second gain for attenuating the second signal based on the control signal; and applying the prioritized mixing to the first signal and the second signal in response to the control signal indicating the first value, and applying simple addition to the first signal and the second signal in response to the control signal indicating the second value, wherein the generating a control signal includes: performing first frequency processing with respect to an intensity of the first signal on the time-frequency plane, to obtain a first spectrum representing an absolute amount of the first signal, performing second frequency processing with respect to the first spectrum, to obtain a second spectrum indicating local changes of the first spectrum, and generating the control signal based on the first spectrum and the second spectrum. ### Claim 11 A non-transitory computer-readable recording medium having computer-readable instructions stored thereon, which when executed, causes a computer to execute a mixing process of a first signal and a second signal on a time-frequency plane, the mixing process comprising: generating a control signal indicating whether to perform prioritized mixing that includes amplification of the first signal and attenuation of the second signal, the control signal taking at least a first value and a second value different from the first value, wherein the first value is not continuous beyond a predetermined bandwidth on a frequency axis; deriving a first gain for amplifying the first signal and a second gain for attenuating the second signal based on the control signal; and applying the prioritized mixing to the first signal and the second signal in response to the control signal indicating the first value, and applying simple addition to the first signal and the second signal in response to the control signal indicating the second value, wherein the generating a control signal includes: performing first frequency processing with respect to an intensity of the first signal on the time-frequency plane, to obtain a first spectrum representing an absolute amount of the first signal, performing second frequency processing with respect to the first spectrum, to obtain a second spectrum indicating local changes of the first spectrum, and generating the control signal based on the first spectrum and the second spectrum.