# US 2017/0373777 A1 · Systems and methods for signal mixing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0373777 A1
- Document ID: US20170373777A1
- Status: Published application
- Kind: A1
- Application number: 15/192,687
- Filed: 2016-06-24
- Published: 2017-12-28
- Earliest claimed: 2016-06-24
- Assignees: Harman International Industries, Incorporated
- Applicants: Harman International Industries, Incorporated
- Inventors: Vinod Channegowda
- Web page: https://patentgrep.com/patent/US20170373777A1

## CPC

- H04H 60/04: Studio equipment; Interconnection of studios
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G10L 15/25: using position of the lips, movement of the lips or face analysis
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G10L 25/48: specially adapted for particular use
- H04H: BROADCAST COMMUNICATION (multiplex communication H04J; pictorial communication aspects of broadcast systems H04N)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems and methods for a signal mixing device are provided. An example method includes generating a mixing request during a live performance based on sensed mouth movement when sensed audio from the mouth is less than a threshold. In such an example method, the sensed mouth movement may be generated from a sensor mounted separately from an audio sensor of a microphone, and may be displayed at a mixing console during the live performance.

## Claims

### Claim 1

A method, comprising:
generating, at a sensor processing unit, a mixing request from a user during a live performance based on a mouth movement of the user sensed by a sensor communicatively connected to the sensor processing unit when audio from a mouth of the user sensed by an audio detection device is detected to be less than a threshold, the mixing request including one or more of a requested change to a sound signal associated with the live performance, an indication of the sound signal to which the requested change is targeted, and an indication of a selected output device for outputting the sound signal with the requested change.

### Claim 2 (depends on 1)

The method of claim 1, wherein the sensed mouth movement is generated from the sensor mounted separately from the audio detection device of a microphone, the audio detection device communicatively connected to the sensor processing unit.

### Claim 3 (depends on 1)

The method of claim 1, further comprising displaying the generated mixing request at a mixing console during the live performance.

### Claim 4 (depends on 1)

The method of claim 1, wherein the sensor sensing mouth movement includes a camera.

### Claim 5 (depends on 4)

The method of claim 4, wherein the camera captures mouth movement in response to a trigger.

### Claim 6 (depends on 5)

The method of claim 5, wherein the trigger is generated by a performer during the live performance.

### Claim 7 (depends on 5)

The method of claim 5, wherein the trigger includes the sensed audio being determined to be less than the threshold, the threshold being a non-zero positive value.

### Claim 8 (depends on 7)

The method of claim 7, further comprising: translating the mixing request into each of an indication of a requested change in volume, an indication of the sound signal to which the requested change in volume is to be targeted, and an indication of the selected output device of one or more output devices for outputting the sound signal with the requested change in volume depending on the sensed mouth movement.

### Claim 9

A signal mixing device comprising:
an audio interface configured to be coupled to one or more audio detection devices for receiving sound signals from each of the one or more audio detection devices;
a sensor interface configured to be coupled to one or more sensors, the one or more sensors detecting user movement corresponding to a message mouthed by a user;
a processor;
a sensor processing unit comprising instructions executable by the processor to determine a mixing request from the user corresponding to the message mouthed by the user, the mixing request being determined based on signals received from the one or more sensors;
an output interface configured to be coupled to one or more first output devices and one or more second output devices, the output interface receiving feedback signals from the sensor processing unit indicating the mixing request for presenting the mixing request via the one or more first output devices, and the output interface receiving the sound signals from each of the one or more audio detection devices for outputting via the one or more second output devices; and
a mixing interface comprising one or more user inputs for selecting adjustments to signals output by the one or more second output devices based on the mixing request, the mixing interface coupled to the output interface and configured to transmit instructions indicating the adjustments to the signals output by the one or more second output devices.

### Claim 10 (depends on 9)

The signal mixing device of claim 9, wherein the one or more audio detection devices includes a microphone and the one or more sensors comprises a camera embedded in or coupled to the microphone.

### Claim 11 (depends on 9)

The signal mixing device of claim 9, wherein the sensor processing unit further comprises instructions executable by the processor to selectively transmit instructions for operating each of the one or more sensors based on a trigger.

### Claim 12 (depends on 9)

The signal mixing device of claim 9, wherein the sensor processing unit further comprises instructions executable to selectively determine the mixing request from the user based on a trigger.

### Claim 13 (depends on 9)

The signal mixing device of claim 9, wherein the mixing request is determined by detecting user movement from a selected sensor while an amplitude of a sound signal from the one or more audio detection devices associated with the selected sensor is below a threshold.

### Claim 14 (depends on 13)

The signal mixing device of claim 13, wherein the user movement comprises one or more of lip movement and neck muscle movement, and wherein determining the mixing request comprises comparing the user movement to a database of stored user movements to determine one or more of words, phrases, and phonemes associated with the detected user movement.

### Claim 15 (depends on 9)

The signal mixing device of claim 9, wherein the sensor processing unit further comprises instructions executable to prioritize signals received from the one or more sensors based on one or more of the sound signals received from the one or more audio detection devices, a determined mixing request indicated by the signals from the one or more sensors, and a user associated with the one or more sensors.

### Claim 16 (depends on 9)

The signal mixing device of claim 9, wherein the mixing request comprises an indication of a requested change in volume, an indication of a sound signal to which the requested change in volume is to be targeted, and an indication of a selected output device of the one or more second output devices for outputting the sound signal with the requested change in volume.

### Claim 17 (depends on 16)

The signal mixing device of claim 16, wherein the one or more second output devices comprises one of an audience-facing speaker and an in-ear monitor speaker.

### Claim 18

A signal mixing device comprising:
an audio interface configured to be coupled to a plurality of audio detection devices for receiving sound signals from each of the plurality of audio detection devices, each of the plurality of audio detection devices being associated with a sound source;
a sensor interface configured to be coupled to a plurality of sensors, each of the plurality of sensors detecting user movement corresponding to a message mouthed by an associated user;
a processor;
a sensor processing unit comprising instructions executable by the processor to determine a mixing request from each of a plurality of users, for each user of the plurality of users, the mixing request for that user being determined based on signals received from one or more selected sensors of the plurality of sensors, the one or more selected sensors being associated with that user, and the sensor processing unit further comprising instructions executable to prioritize each of the mixing requests from the plurality of users based on one or more of a requested adjustment indicated by the mixing request, an identity of a user providing the mixing request, and a target of the requested adjustment for the mixing request;
an output interface configured to be coupled to one or more first output devices and one or more second output devices, the output interface receiving feedback signals from the sensor processing unit indicating the mixing request for each of the plurality of users for presenting the mixing request via the one or more first output devices according to the prioritization of the mixing requests, and the output interface receiving the sound signals from each of the plurality of audio detection devices for outputting via the one or more second output devices; and
a mixing interface comprising one or more user inputs for selecting adjustments to signals output by the one or more second output devices based on each of the mixing requests, the mixing interface coupled to the output interface and configured to transmit instructions indicating the adjustments to the signals output by the one or more second output devices.

### Claim 19 (depends on 18)

The signal mixing device of claim 18, wherein the plurality of audio detection devices includes a microphone and the one or more selected sensors comprises a camera embedded in or coupled to the microphone.

### Claim 20 (depends on 18)

The signal mixing device of claim 18, wherein determining the mixing request for each user comprises detecting user movement from a first selected sensor of the one or more selected sensors while an amplitude of a sound signal from the plurality of audio detection devices associated with the first selected sensor is below a threshold.
