Augmented reality speech-to-text captioning
AssigneeInternational Business Machines Corporation
InventorsJill S. Dhillon, Jennifer M. Hatfield, Tushar Agrawal, Jeremy R. Fox
Life of this patent
Estimating the term…
Abstract
An approach for improving communications for hearing challenged individuals. The approach captures audio communication and video communication associated with a speaker presentation. The approach converts the audio communication to text. The approach sends the text to an augmented reality (AR) device, associated with a user having hearing challenges, as captions. The approach stores the audio communication, the video communication, and the captions for replay by the user.
Claims17
A computer-implemented method for improving audio communications, the computer-implemented method comprising:
- capturing audio communication and video communication associated with a speaker presentation to a user;
- converting the audio communication to text, the converting performed responsive to a machine learning prediction that comprehension of the audio by the user will fall below a threshold level of comprehension;
- sending the text to an augmented reality (AR) device as captions for the audio communication;
- and
- displaying the captions along with the video communication on the AR device.
A computer system for improving audio communications, the computer system comprising:
- one or more computer processors;
- one or more non-transitory computer readable storage media;
- and
- program instructions stored on the one or more non-transitory computer readable storage media, the program instructions comprising:
- program instructions to capture audio communication and video communication associated with a speaker presentation to a user;
- program instructions to convert, responsive to a machine learning prediction that comprehension of the audio by the user will fall below a threshold level of comprehension, the audio communication to text;
- program instructions to send the text to an augmented reality (AR) device as captions for the audio communication;
- and
- program instructions to display the captions along with the video communication on the AR device.
A computer program product for improving audio communications, the computer program product comprising:
- one or more computer readable storage media;
- and
- program instructions stored on the one or more computer readable storage media, the program instructions comprising:
- program instructions to capture audio communication and video communication associated with a speaker presentation to a user;
- program instructions to convert, responsive to a machine learning prediction that comprehension of the audio by the user will fall below a threshold level of comprehension, the audio communication to text;
- program instructions to send the text to an augmented reality (AR) device as captions for the audio communication;
- and
- program instructions to display the captions along with the video communication on the AR device.
Dependencies are inferred from the claim text.