Augmented reality speech-to-text captioning
AssigneeInternational Business Machines Corporation
InventorsJill S. Dhillon, Jennifer M. Hatfield, Tushar Agrawal, Jeremy R. Fox
Life of this patent
Estimating the term…
Abstract
An approach for improving communications for hearing challenged individuals by dynamically optimizing presentation of speech-derived captions. The approach captures audio communication associated with a speaker presentation. The approach converts the audio communication to text. The approach presents the text as captions on an augmented reality (AR) device. The approach monitors user processing behavior associated with consumption of the captions. The approach dynamically modifies, in real time and based on the monitored user processing behavior, at least one presentation parameter of the captions to improve comprehension of the audio communication.
Claims20
A method for dynamically optimizing presentation of speech-derived captions, the method comprising:
- capturing audio communication associated with a speaker presentation;
- converting the audio communication to text;
- presenting the text as captions on an augmented reality (AR) device;
- monitoring user processing behavior associated with consumption of the captions;
- and
- dynamically modifying, in real time and based on the monitored user processing behavior, at least one presentation parameter of the captions to improve comprehension of the audio communication.
A computer system for dynamically optimizing presentation of speech-derived captions, the computer system comprising:
- one or more computer processors;
- and
- one or more computer readable storage media having program instructions stored therein;
- wherein:
- the one or more computer processors execute the program instructions to cause the one or more computer processors to perform a method comprising:
- capturing audio communication associated with a speaker presentation;
- converting the audio communication to text;
- presenting the text as captions on an augmented reality (AR) device;
- monitoring user processing behavior associated with consumption of the captions;
- and
- dynamically modifying, in real time and based on the monitored user processing behavior, at least one presentation parameter of the captions to improve comprehension of the audio communication.
A computer program product comprising one or more computer-readable storage media having a set of instructions stored therein which, when executed by a processor, causes the processor to dynamically optimize presentation of speech-derived captions by:
- capturing audio communication associated with a speaker presentation;
- converting the audio communication to text;
- presenting the text as captions on an augmented reality (AR) device;
- monitoring user processing behavior associated with consumption of the captions;
- and
- dynamically modifying, in real time and based on the monitored user processing behavior, at least one presentation parameter of the captions to improve comprehension of the audio communication.
Dependencies are inferred from the claim text.