CARTGPT: Improving CART Captioning Using Large Language Models
AssigneeThe Regents of the University of Michigan
InventorsLiang-yuan Wu, Dhruv Jain
Life of this patent
Estimating the term…
Loading patent…
AssigneeThe Regents of the University of Michigan
InventorsLiang-yuan Wu, Dhruv Jain
Estimating the term…
Systems and methods to generate corrected Communication Access Real-time Translation (CART) captions are provided herein. The systems and methods may include receiving, an uncorrected CART transcript and an automatic speech recognition (ASR) transcript. The uncorrected CART transcript and ASR transcript may be aligned by segmenting the uncorrected CART transcript and ASR transcript into clauses, segmenting the clauses, and determining similarity values between the plurality of the CART transcript clauses and the ASR transcript clauses, with alignment of the uncorrected CART transcript and ASR transcript based on the similarity values. Errors in the uncorrected CART transcript may be detected and replaced with placeholder characters. The uncorrected CART transcript, the ASR transcript, and a prompt including context may be provided to a large language model (LLM) to generate a corrected CART transcript. Non-error substitutions may be removed from the corrected CART transcript, and the corrected CART transcript may be displayed.
A method for real-time correction of communication access real time translation (CART) captions comprising:
A computing system for real-time correction of communication access real time translation (CART) captions comprising:
One or more non-transitory computer-readable media having stored thereon instructions that when executed, cause a computer to:
Dependencies are inferred from the claim text.