# US 2016/0365099 A1 · Method and system for consonant-vowel ratio modification for improving speech perception

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0365099 A1
- Document ID: US20160365099A1
- Status: Published application
- Kind: A1
- Application number: 15/121,599
- Filed: 2015-01-27
- Published: 2016-12-15
- Earliest claimed: 2014-03-04
- Assignees: Indian Institute of Technology Bombay
- Applicants: Indian Institute of Technology Bombay
- Inventors: Prem Chand Pandey; Ammanath Ramakrishnan Jayan; Nitya Tiwari
- Web page: https://patentgrep.com/patent/US20160365099A1

## CPC

- G10L 21/0232: Processing in the frequency domain
- G10L 21/0264: characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques
- G10L 21/0364: for improving intelligibility
- G10L 25/21: the extracted parameters being power information
- G10L 25/87: Detection of discrete points within a voice signal
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Increasing the level of the consonant segments relative to the nearby vowel segments, known as consonant-vowel ratio (CVR) modification, is reported to be effective in improving speech intelligibility by listeners in noisy backgrounds and by hearing-impaired listeners. A method along with a system for real-time CVR modification using the rate of change of spectral centroid for detection of spectral transitions is disclosed. A preferred embodiment of the invention using a 16-bit fixed point processor with on-chip FFT hardware is also presented for real-time signal processing. It can be integrated with other FFT-based signal processing in communication devices, hearing aids, and other systems for improving speech perception under adverse listening conditions.

## Claims

### Claim 1

A method for improving speech perception, by processing a digital speech signal, the method comprising:
detecting perceptually salient segments in said digital speech signal;
calculating time-varying gain in accordance with a location and energy of said detected segments in said digital speech signal; and
applying said time-varying gain to said digital speech signal,
wherein detecting said perceptually salient segments and calculating said time-varying gain comprises:
windowing samples of said digital speech signal to form overlapping frames and calculating energy of said frames;
smoothening said frame energy by a moving-average filter to obtain smoothened short-time energy;
applying a peak detector with exponential decay on said frame energy to track peak energy;
generating a low-frequency tone and multiplying said low-frequency tone with said peak energy and adding a resulting scaled tone to said digital speech signal to obtain a tone-added signal;
windowing said tone-added signal and applying Discrete Fourier transform (DFT) to obtain short-time magnitude spectrum of said tone-added signal;
applying a moving-average filter on said short-time magnitude spectrum to obtain a smoothened short-time magnitude spectrum;
calculating a spectral centroid of said smoothened short-time magnitude spectrum;
smoothening said spectral centroid by median filtering to obtain a smoothened spectral centroid;
calculating a first-difference of said smoothened spectral centroid to obtain a rate of change of said smoothened spectral centroid; and
selecting said time-varying gain using said smoothened short-time energy, said peak energy, and said rate of change of said smoothened spectral centroid
wherein modification of perceptually salient segments is carried out without significantly increasing loudness level of said digital speech signal.

### Claim 2 (depends on 1)

The method for improving speech perception as claimed in claim 1, wherein said perceptually salient segments for modification comprise sharp spectral transitions associated with major changes in a vocal tract configuration and occur at a release of closures in stops and affricates and in fricatives and nasals.

### Claim 3 (depends on 1)

The method for improving speech perception as claimed in claim 1, wherein said peak detector with exponential decay tracks a vowel energy and retains it during one or more of stop closures and low energy clusters.

### Claim 4 (depends on 1)

The method for improving speech perception as claimed in claim 1, wherein said rate of change of said smoothened spectral centroid of said digital speech signal is used to detect said perceptually salient segments with sharp spectral transitions to avoid effects of speaker variability.

### Claim 5 (depends on 1)

The method for improving speech perception as claimed in claim 1, wherein said spectral centroid is calculated from said short-time magnitude spectrum of said digital speech signal added with said low-frequency tone for reducing insertion errors in the detection of said perceptually salient segments.

### Claim 6 (depends on 1)

The method for improving speech perception as claimed in claim 1, wherein said spectral centroid is smoothened by said median filter for suppressing ripples without significantly smearing changes corresponding to major spectral transitions.

### Claim 7 (depends on 1)

The method for improving speech perception as claimed in claim 1, wherein said perceptually salient segments are enhanced by said time-varying gain to reduce computational complexity and memory requirement associated with labelling of the segments and analysis-synthesis based processing.

### Claim 8 (depends on 1)

The method for improving speech perception as claimed in claim 1, wherein said gain to be applied at a frame position is calculated using said rate of change of said smoothened spectral centroid, said smoothened short-time energy, and said peak energy.

### Claim 9 (depends on 8)

The method for improving speech perception as claimed in claim 8, wherein said gain is selected using a hysteresis-based thresholding of rate of change of said smoothened spectral centroid for detection of sharp spectral transitions with a high temporal accuracy and low insertion error.

### Claim 10 (depends on 8)

The method for improving speech perception as claimed in claim 8, wherein said gain is selected to keep a signal level of said perceptually salient segment below that of a preceding vowel.

### Claim 11 (depends on 8)

The method for improving speech perception as claimed in claim 8, wherein said gain is changed from a current value to a target value in logarithmic steps to avoid perceptible distortions caused by abrupt changes.

### Claim 12 (depends on 1)

The method for improving speech perception as claimed in claim 1, wherein an output signal is obtained by multiplying said time-varying gain with said digital speech signal which is delayed to compensate for a processing delay in selection of said time-varying gain.

### Claim 13

A system for improving speech perception by processing a digital speech signal available in a form of digital samples at regular intervals or in the form of data packets, the system comprising:
a digital input-output port configured to receive said digital speech signal and output a processed digital speech signal; and
a digital processor interfaced to said digital input-output port configured to process said digital speech signal;
wherein said digital speech signal is processed using the a method comprising:
detecting perceptually salient segments in said digital speech signal;
calculating time-varying gain in accordance with a location and energy of said detected segments in said digital speech signal; and
applying said time-varying gain to said digital speech signal.

### Claim 14

A system for improving speech perception by processing an input analog speech signal, the system comprising:
an analog-to-digital converter configured to convert said input analog speech signal to a digital speech signal;
a digital signal processor configured to process said digital speech signal; and
digital-to-analog converter configured to convert the processed digital speech signal as an output analog speech signal,
wherein said digital speech signal is processed using a method comprising:
detecting perceptually salient segments in said digital speech signal;
calculating time-varying gain in accordance with a location and energy of said detected segments in said digital speech signal; and
applying said time-varying gain to said digital speech signal.

### Claim 15 (depends on 14)

The system for improving speech perception as claimed in claim 14, wherein said digital signal processor comprises on-chip FFT hardware and said analog-to-digital converter and said digital-to-analog converter are configured for signal input and output operations, respectively, using DMA (direct memory access) and cyclic buffering for computationally efficient processing.

### Claim 16 (depends on 13)

The system for improving speech perception as claimed in claim 13, wherein detecting said perceptually salient segments and calculating said time-varying gain comprises:
windowing samples of said digital speech signal to form overlapping frames and calculating energy of said frames;
smoothening said frame energy by a moving-average filter to obtain a smoothened short-time energy;
applying a peak detector with exponential decay on said frame energy to track peak energy;
generating a low-frequency tone and multiplying said low-frequency tone with said peak energy and adding a resulting scaled tone to said digital speech signal to obtain a tone-added signal;
windowing said tone-added signal and applying Discrete Fourier transform (DFT) to obtain a short-time magnitude spectrum of said tone-added signal;
applying a moving-average filter on said short-time magnitude spectrum to obtain smoothened short-time magnitude spectrum;
calculating a spectral centroid of said smoothened short-time magnitude spectrum;
smoothening said spectral centroid by median filtering to obtain a smoothened spectral centroid;
calculating a first-difference of said smoothened spectral centroid to obtain a rate of change of said smoothened spectral centroid; and
selecting said time-varying gain using said smoothened short-time energy, said peak energy, and said rate of change of said smoothened spectral centroid.

### Claim 17 (depends on 14)

The system for improving speech perception as claimed in claim 14, wherein detecting said perceptually salient segments and calculating said time-varying gain comprises:
windowing samples of said digital speech signal to form overlapping frames and calculating energy of said frames;
smoothening said frame energy by a moving-average filter to obtain a smoothened short-time energy;
applying a peak detector with exponential decay on said frame energy to track peak energy;
generating a low-frequency tone and multiplying said low-frequency tone with said peak energy and adding a resulting scaled tone to said digital speech signal to obtain a tone-added signal;
windowing said tone-added signal and applying Discrete Fourier transform (DFT) to obtain a short-time magnitude spectrum of said tone-added signal;
applying a moving-average filter on said short-time magnitude spectrum to obtain smoothened short-time magnitude spectrum;
calculating a spectral centroid of said smoothened short-time magnitude spectrum;
smoothening said spectral centroid by median filtering to obtain a smoothened spectral centroid;
calculating a first-difference of said smoothened spectral centroid to obtain a rate of change of said smoothened spectral centroid; and
selecting said time-varying gain using said smoothened short-time energy, said peak energy, and said rate of change of said smoothened spectral centroid.
