# US 2024/0194184 A1 · Testing cascaded deep learning pipelines comprising a speech-to-text model and a text intent classifier

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0194184 A1
- Document ID: US20240194184A1
- Status: Published application
- Kind: A1
- Application number: 18/062,857
- Filed: 2022-12-07
- Published: 2024-06-13
- Earliest claimed: 2022-12-07
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Swagatam Haldar; Diptikalyan Saha; Deepak Vijaykeerthy; Aniya Aggarwal; Nishtha Madaan
- Web page: https://patentgrep.com/patent/US20240194184A1

## CPC

- G10L 15/01: Assessment or evaluation of speech recognition systems
- G06N 3/08: Learning methods
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/32: Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
- G10L 25/18: the extracted parameters being spectral information of each sub-band
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to a process to facilitate testing a cascaded pipeline. A system can comprise a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components can comprise an input component, a cascaded pipeline, and an evaluation component. The input component can receive a test case associated with a label from labeled speech data represented by waveform. The evaluation component can feed the test case to the cascaded pipeline to obtain an output of the cascaded pipeline. The evaluation component can evaluate a robustness of the cascaded pipeline by comparing the output of the cascaded pipeline and the label. The cascaded pipeline can include a first model and a second model, and the first model can be different than the second model.

## Claims

### Claim 1

A system comprising:
a memory that stores computer executable components; and
a processor, operably coupled to the memory, and that executes the computer executable components stored in the memory, wherein the computer executable components comprise:
an input component that receives a test case associated with a label from labeled speech data represented by waveform;
a cascaded pipeline; and
an evaluation component that feeds the test case to the cascaded pipeline to obtain an output;
wherein the evaluation component determines whether the output is different from the label to test the cascaded pipeline.

### Claim 2 (depends on 1)

The system of claim 1, wherein the cascaded pipeline comprises a first model and a second model, and the first model is different than the second model.

### Claim 3 (depends on 2)

The system of claim 2, wherein the first model is a speech-to-text model, and the second model is an intent classification model.

### Claim 4 (depends on 3)

The system of claim 3, further comprising a generation component that generates the test case by adding noise to a sample from the labeled speech data.

### Claim 5 (depends on 4)

The system of claim 4, wherein the noise added to the sample is such a degree that the output of the cascaded pipeline is different from the label.

### Claim 6 (depends on 5)

The system of claim 5, wherein the generation component generates a magnitude spectrum from the test case and adds the noise to the magnitude spectrum.

### Claim 7 (depends on 6)

The system of claim 6, wherein the generation component reconstructs the test case via the magnitude spectrum.

### Claim 8

A computer implemented method of testing a cascaded pipeline, comprising:
generating, using a processor operatively coupled to memory, a test case associated with a label from labeled speech data represented by waveform; and
feeding, using the processor, the test case to a cascaded pipeline to obtain an output; and
testing, using the processor, the cascaded pipeline by determining whether the output is different from the label.

### Claim 9 (depends on 8)

The computer implemented method of claim 8, wherein the cascaded pipeline comprises a first model and a second model, the first model is different than the second model.

### Claim 10 (depends on 9)

The computer implemented method of claim 9, wherein the first model is a speech-to-text model, and the second model is an intent classification model.

### Claim 11 (depends on 10)

The computer implemented method of claim 10, further comprising:
generating, using the processor, the test case by adding noise to a sample from the labeled speech data.

### Claim 12 (depends on 11)

The computer implemented method of claim 11, wherein the noise added to the sample is such a degree that the output of the cascaded pipeline is substantially different from the label.

### Claim 13 (depends on 12)

The computer implemented method of claim 12, further comprising:
generating, using the processor, a magnitude spectrum from the test case and adding the noise to the magnitude spectrum.

### Claim 14 (depends on 13)

The computer implemented method of claim 13, further comprising:
reconstructing, using the processor, the test case via the magnitude spectrum.

### Claim 15

A computer program product for testing a cascaded pipeline, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
generate, using the processor, a test case associated with a label from labeled speech data represented by waveform;
feed, using the processor, the test case to a cascaded pipeline to obtain an output;
test, using the processor, the cascaded pipeline by determining whether the output is different from the label associated with the test case.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the cascaded pipeline comprises a speech-to-text model and an intent classification model.

### Claim 17 (depends on 16)

The computer program product of claim 16, wherein the program instructions further cause the processor to:
generate, using the processor, the test case by adding noise to a sample from the labeled speech data.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein the noise added to the sample is such a degree that the output of the cascaded pipeline is substantially different from the label.

### Claim 19 (depends on 18)

The computer program product of claim 18, wherein the program instructions further cause the processor to:
generate, using the processor, a magnitude spectrum from the test case and adding the noise to the magnitude spectrum.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein the program instructions further cause the processor to:
reconstruct, using the processor, the test case via the magnitude spectrum.
