# US 9,520,141 B2 · Keyboard typing detection and suppression

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,520,141 B2
- Document ID: US9520141B2
- Status: Granted
- Kind: B2
- Application number: 13/781,262
- Filed: 2013-02-28
- Published: 2014-08-28
- Granted: 2016-12-13
- Earliest claimed: 2013-02-28
- Assignees: Google Inc.
- Applicants: Google Inc.
- Inventors: Jens Enzo Nyby Christensen; Simon J. Godsill; Jan Skoglund
- Web page: https://patentgrep.com/patent/US9520141B2

## CPC

- G10L 25/48: specially adapted for particular use
- G10L 21/0216: characterised by the method used for estimating noise
- G10L 21/02: Speech enhancement, e.g. noise reduction or echo cancellation (reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08)
- G10L 21/0208: Noise filtering
- G10L 25/84: for discriminating voice from noise
- G10L 25/93: Discriminating between voiced and unvoiced parts of speech signals (G10L25/90 takes precedence)
- G10L 2025/935: {Mixed voiced class; Transitions}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Provided are methods and systems for detecting the presence of a transient noise event in an audio stream using primarily or exclusively the incoming audio data. Such an approach offers improved temporal resolution and is computationally efficient. The methods and systems presented utilize some time-frequency representation of an audio signal as the basis in a predictive model in an attempt to find outlying transient noise events and interpret the true detection state as a Hidden Markov Model (HMM) to model temporal and frequency cohesion common amongst transient noise events.

## Claims

### Claim 1

A method performed by a teleconference computing device for suppressing transient noise in an audio signal, the method comprising:
extracting one or more voiced parts from an audio signal input from an audio capture device to yield a residual part of the audio signal;
decomposing the residual part of the signal into a sparse set of coefficients corresponding to noise pulses in the residual part of the signal;
modeling each of the coefficients as a switched noise pulse combined with additive noise;
estimating initial probabilities of detection states for each of the modeled coefficients;
calculating transition probabilities between each of the detection states;
determining a probable detection state for each of the coefficients based on the initial probabilities of the detection states for each of the coefficients, the calculated transition probabilities between each of the detection states, and observation probabilities determined from observed data associated with the noise pulses;
filtering out transient noise from the residual part of the signal based on the probable detection states determined for the coefficients; and
combining the filtered residual part of the signal with the one or more extracted voiced parts of the signal,
wherein the transient noise is at least one of feedback noise, fan noise, and button-clicking noise due to mechanical connection between the audio capture device and a keyboard or trackpad of the teleconferencing computing device.

### Claim 2 (depends on 1)

The method of claim 1, wherein extracting the one or more voiced parts of the audio signal includes recursively subtracting tonal components from the audio signal.

### Claim 3 (depends on 1)

The method of claim 1, wherein the residual part of the signal is decomposed into a sparse set of coefficients using a wavelet packet transform.

### Claim 4 (depends on 1)

The method of claim 1, wherein estimating the initial probability of the one or more detection states for each of the coefficients includes modeling the switched noise pulse and the additive noise as zero-mean Gaussian distributions.

### Claim 5 (depends on 4)

The method of claim 4, wherein the switched noise pulse is modeled using a changing variance model based on an envelope of the changing variance of the noise pulse.

### Claim 6 (depends on 1)

The method of claim 1, wherein estimating the initial probability of the one or more detection states for each of the coefficients includes modeling the additive noise using an autoregressive (AR) model with estimated parameters.

### Claim 7 (depends on 1)

The method of claim 1, wherein the probable detection states for the coefficients are determined using a Hidden Markov Model (HMM).

### Claim 8 (depends on 1)

The method of claim 1, further comprising determining, based on the combined residual part and the one or more extracted voiced parts, whether to perform further transient noise suppression on the audio signal.

### Claim 9 (depends on 1)

The method of claim 1, further comprising, prior to combining the filtered residual part of the signal and the one or more extracted voiced parts of the signal:
determining that the one or more extracted voiced parts include low-frequency components of transient noise; and
filtering out the low-frequency components of transient noise from the one or more extracted voiced parts.

### Claim 10 (depends on 1)

The method of claim 1, further comprising identifying the one or more voiced parts of the audio signal by detecting spectral peaks in the frequency domain of the audio signal.

### Claim 11 (depends on 10)

The method of claim 10, wherein the spectral peaks are detected by thresholding a median filter output.

### Claim 12 (depends on 1)

The method of claim 1, further comprising performing the extraction of voiced parts of the audio signal multiple times using different frame sizes.

### Claim 13 (depends on 1)

The method of claim 1, further comprising performing the extraction of voiced parts of the audio signal multiple times using different thresholds for a median filter output.

### Claim 14 (depends on 1)

The method of claim 1, wherein filtering out transient noise from the residual part of the audio signal includes:
identifying corrupted samples of the residual part of the audio signal based on the probable detection states determined for the coefficients; and
removing the corrupted samples from the audio signal.

### Claim 15 (depends on 14)

The method of claim 14, further comprising restoring the corrupted samples removed from the audio signal.

### Claim 16 (depends on 1)

The method of claim 1, further comprising:
determining, based on the residual part of the audio signal, that additional voiced parts remain in the residual part of the audio signal; and
extracting one or more of the additional voiced parts from the residual part of the audio signal.

### Claim 17 (depends on 1)

The method of claim 1, wherein the noise pulses in the residual part of the audio signal correspond to mechanical impulses caused by keystrokes on a keypad.

### Claim 18

A teleconferencing computing system for suppressing transient noise in an audio signal, the system comprising:
at least one processor; and
a non-transitory computer-readable medium coupled to the at least one processor having instructions stored thereon that, when executed by the at least one processor, causes the at least one processor to:
extract one or more voiced parts from an audio signal input from an audio capture device to yield a residual part of the audio signal;
decompose the residual part of the signal into a sparse set of coefficients corresponding to noise pulses in the residual part of the signal;
model each of the coefficients as a switched noise pulse combined with additive noise;
estimate initial probabilities of detection states for each of the modeled coefficients;
calculate transition probabilities between each of the detection states;
determine a probable detection state for each of the coefficients based on the initial probabilities of the detection states for each of the coefficients, the calculated transition probabilities between each of the detection states, and observation probabilities determined from observed data associated with the noise pulses;
filter out transient noise from the residual part of the signal based on the probable detection states determined for the coefficients; and
combine the filtered residual part of the signal with the one or more extracted voiced parts of the signal,
wherein the transient noise is at least one of feedback noise, fan noise, and button-clicking noise due to mechanical connection between the audio capture device and a keyboard or trackpad of the teleconferencing computing system.

### Claim 19 (depends on 18)

The system of claim 18, wherein the at least one processor is further caused to:
prior to combining the filtered residual part of the signal and the one or more extracted voiced parts of the signal, determine that the one or more extracted voiced parts include low-frequency components of transient noise; and
filter out the low-frequency components of transient noise from the one or more extracted voiced parts.

### Claim 20 (depends on 18)

The system of claim 18, wherein the probable detection states for the coefficients are determined using a Hidden Markov Model (HMM).