# US 2026/0099663 A1 · CARTGPT: Improving CART Captioning Using Large Language Models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0099663 A1
- Document ID: US20260099663A1
- Status: Published application
- Kind: A1
- Application number: 19/349,821
- Filed: 2025-10-03
- Published: 2026-04-09
- Earliest claimed: 2024-10-04
- Assignees: The Regents of the University of Michigan
- Applicants: Regents of the University of Michigan
- Inventors: Liang-yuan Wu; Dhruv Jain
- Web page: https://patentgrep.com/patent/US20260099663A1

## CPC

- G06F 40/166: Editing, e.g. inserting or deleting
- G06F 40/194: Calculation of difference between files
- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods to generate corrected Communication Access Real-time Translation (CART) captions are provided herein. The systems and methods may include receiving, an uncorrected CART transcript and an automatic speech recognition (ASR) transcript. The uncorrected CART transcript and ASR transcript may be aligned by segmenting the uncorrected CART transcript and ASR transcript into clauses, segmenting the clauses, and determining similarity values between the plurality of the CART transcript clauses and the ASR transcript clauses, with alignment of the uncorrected CART transcript and ASR transcript based on the similarity values. Errors in the uncorrected CART transcript may be detected and replaced with placeholder characters. The uncorrected CART transcript, the ASR transcript, and a prompt including context may be provided to a large language model (LLM) to generate a corrected CART transcript. Non-error substitutions may be removed from the corrected CART transcript, and the corrected CART transcript may be displayed.

## Claims

### Claim 1

A method for real-time correction of communication access real time translation (CART) captions comprising:
receiving, by one or more processors, an uncorrected CART transcript and an automatic speech recognition (ASR) transcript;
segmenting, by the one or more processors, the uncorrected CART transcript into a plurality of CART transcript clauses and the ASR transcript into a plurality of ASR transcript clauses;
embedding, by the one or more processors, the plurality of CART transcript clauses and the plurality of ASR transcript clauses;
determining, by the one or more processors, similarity values between the plurality of the CART transcript clauses and the plurality of the ASR transcript clauses;
aligning, by the one or more processors, the plurality of the CART transcript clauses with the plurality of the ASR transcript clauses based on the similarity values;
detecting, by the one or more processors, an error in the uncorrected CART transcript;
replacing, by the one or more processors, the error in the uncorrected CART transcript with a placeholder character;
providing, by the one or more processors, the uncorrected CART transcript, the ASR transcript, and a prompt including context to a large language model (LLM) to generate a corrected CART transcript by replacing the placeholder character based on the ASR transcript, the alignment of the CART transcript clauses with the plurality of the ASR transcript clauses, and the context;
removing, by the one or more processors, one or more non-error substitutions in the corrected CART transcript; and
displaying, by the one or more processors, the corrected CART transcript.

### Claim 2 (depends on 1)

The method of claim 1, wherein removing the one or more non-error substitutions includes:
comparing, by the one or more processors, the corrected CART transcript to the uncorrected CART transcript;
detecting, by the one or more processors, the one or more non-error substitutions in the corrected CART transcript; and
replacing, by the one or more processors, the one or more non-error substitutions with a corresponding word from the uncorrected CART transcript.

### Claim 3 (depends on 1)

The method of claim 1, wherein detecting the one or more errors includes detecting one or more error keywords, the error keywords including at least one of: (i) “[inaudible]”, (ii) “[indiscernible]”, or (iii) “(?)”.

### Claim 4 (depends on 1)

The method of claim 1, wherein the errors include at least one of: (i) an omission, or (ii) an untranslate error.

### Claim 5 (depends on 1)

The method of claim 1, wherein the context includes two paragraphs of the uncorrected CART transcript preceding a paragraph containing an error of the one or more errors.

### Claim 6 (depends on 1)

The method of claim 1, wherein segmenting the uncorrected CART transcript is based on punctuation and pause cues.

### Claim 7 (depends on 1)

The method of claim 1, wherein aligning the plurality of the CART transcript clauses with the plurality of the ASR transcript clauses includes:
for each clause of the plurality of the CART transcript clauses:
determining, at least partially based on a position in the CART transcript of each clause of the plurality of the CART transcript clauses, a subset of ASR transcript clauses from the plurality of the ASR transcript clauses;
determining, based on the similarity values, a matching clause from the subset of ASR transcript clauses.

### Claim 8

A computing system for real-time correction of communication access real time translation (CART) captions comprising:
one or more processors; and
one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing system to:
receive an uncorrected CART transcript and an automatic speech recognition (ASR) transcript;
segment the uncorrected CART transcript into a plurality of CART transcript clauses and the ASR transcript into a plurality of ASR transcript clauses;
embed the plurality of CART transcript clauses and the plurality of ASR transcript clauses;
determine similarity values between the plurality of the CART transcript clauses and the plurality of the ASR transcript clauses;
align the plurality of the CART transcript clauses with the plurality of the ASR transcript clauses based on the similarity values;
detect an error in the uncorrected CART transcript;
replace the error in the uncorrected CART transcript with a placeholder character;
provide the uncorrected CART transcript, the ASR transcript, and a prompt including context to a large language model (LLM) to generate a corrected CART transcript by replacing the placeholder character based on the ASR transcript, the alignment of the plurality CART transcript clauses with the plurality of the ASR transcript clauses, and the context;
remove one or more non-error substitutions in the corrected CART transcript; and
display the corrected CART transcript.

### Claim 9 (depends on 8)

The computing system of claim 8, wherein removing the one or more non-error substitutions by:
comparing the corrected CART transcript to the uncorrected CART transcript;
detecting the one or more non-error substitutions in the corrected CART transcript; and
replacing the one or more non-error substitutions with a corresponding word from the uncorrected CART transcript.

### Claim 10 (depends on 7)

The computing system of claim 7, wherein detecting the one or more errors includes detecting one or more error keywords.

### Claim 11 (depends on 7)

The computing system of claim 7, wherein the errors include at least one of: (i) an omission, or (ii) an untranslate error.

### Claim 12 (depends on 7)

The computing system of claim 7, wherein the context includes two paragraphs of the uncorrected CART transcript preceding a paragraph containing an error of the one or more errors.

### Claim 13 (depends on 7)

The computing system of claim 7, wherein segmenting the uncorrected CART transcript is based on punctuation and pause cues.

### Claim 14 (depends on 7)

The computing system of claim 7, wherein aligning the plurality of the CART transcript clauses with the plurality of the ASR transcript clauses includes:
for each clause of the plurality of the CART transcript clauses:
determining, at least partially based on a position in the CART transcript of each clause of the plurality of the CART transcript clauses, a subset of ASR transcript clauses from the plurality of the ASR transcript clauses;
determining, based on the similarity values, a matching clause from the subset of ASR transcript clauses.

### Claim 15

One or more non-transitory computer-readable media having stored thereon instructions that when executed, cause a computer to:
receive an uncorrected CART transcript and an automatic speech recognition (ASR) transcript;
segment the uncorrected CART transcript into a plurality of CART transcript clauses and the ASR transcript into a plurality of ASR transcript clauses;
embed the plurality of CART transcript clauses and the plurality of ASR transcript clauses;
determine similarity values between the plurality of the CART transcript clauses and the plurality of the ASR transcript clauses;
align the plurality of the CART transcript clauses with the plurality of the ASR transcript clauses based on the similarity values;
detect an error in the uncorrected CART transcript;
replace the error in the uncorrected CART transcript with a placeholder character;
provide the uncorrected CART transcript, the ASR transcript, and a prompt including context to a large language model (LLM) to generate a corrected CART transcript by replacing the placeholder character based on the ASR transcript, the alignment of the plurality CART transcript clauses with the plurality of the ASR transcript clauses, and the context;
remove one or more non-error substitutions in the corrected CART transcript; and
display the corrected CART transcript.

### Claim 16 (depends on 15)

The non-transitory computer-readable media of claim 15, wherein removing the one or more non-error substitutions by:
comparing the corrected CART transcript to the uncorrected CART transcript;
detecting the one or more non-error substitutions in the corrected CART transcript; and
replacing the one or more non-error substitutions with a corresponding word from the uncorrected CART transcript.

### Claim 17 (depends on 15)

The non-transitory computer-readable media of claim 15, wherein detecting the one or more errors includes detecting one or more error keywords.

### Claim 18 (depends on 15)

The non-transitory computer-readable media of claim 15, wherein the context includes two paragraphs of the uncorrected CART transcript preceding a paragraph containing an error of the one or more errors.

### Claim 19 (depends on 15)

The non-transitory computer-readable media of claim 15, wherein segmenting the uncorrected CART transcript is based on punctuation and pause cues.

### Claim 20 (depends on 15)

The non-transitory computer-readable media of claim 15, wherein aligning the plurality of the CART transcript clauses with the plurality of the ASR transcript clauses includes:
for each clause of the plurality of the CART transcript clauses:
determining, at least partially based on a position in the CART transcript of each clause of the plurality of the CART transcript clauses, a subset of ASR transcript clauses from the plurality of the ASR transcript clauses;
determining, based on the similarity values, a matching clause from the subset of ASR transcript clauses.
