# US 2021/0344995 A1 · Video content conversion > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0344995 A1 - Document ID: US20210344995A1 - Status: Published application - Kind: A1 - Application number: 16/865,447 - Filed: 2020-05-04 - Published: 2021-11-04 - Earliest claimed: 2020-05-04 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Sarbajit K. Rakshit; Martin Franz; Vittorio Castelli - Web page: https://patentgrep.com/patent/US20210344995A1 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06N 20/00: Machine learning - G06V 20/47: {Detecting features for summarising video content} - G06V 40/18: Eye characteristics, e.g. of the iris - H04N 21/42203: {sound input device, e.g. microphone} - H04N 21/440236: {by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text} - H04N 21/44204: {Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched (monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31)} - H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)} - H04N 21/466: Learning process for intelligent management, e.g. learning user preferences for recommending movies {(services using the results of monitoring in broadcast systems H04H60/61)} - G10L 13/00: Speech synthesis; Text to speech systems - G10L 21/04: Time compression or expansion - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Described are techniques for video conversion for accessibility including a technique comprising determining, using data from at least one camera, that a user is distracted based on a direction of gaze of the user with respect to a display device presenting video content. The technique further comprises converting, by a machine learning model, the video content to audio content in response to determining the user is distracted, wherein the audio content comprises a description of the video content. The technique further comprises outputting, using at least one speaker, the audio content to the user while the user is distracted. ## Claims ### Claim 1 A computer-implemented method comprising: determining, using data from at least one camera, that a user is distracted based on a direction of gaze of the user with respect to a display device presenting video content; converting, by a machine learning model, the video content to audio content in response to determining the user is distracted, wherein converting the video content to the audio content further comprises: identifying salient features of the video content; converting, to the audio content, a subset of the salient features that do not match any preexisting audio content; and interleaving the audio content with the preexisting audio content by increasing a speed of the preexisting audio content in order to create respective portions of the video content without any preexisting audio content and inserting the audio content into the respective portions; and outputting, using at least one speaker, the audio content to the user while the user is distracted. ### Claim 2 (depends on 1) The method of claim 1, wherein determining the user is distracted further comprises: determining the direction of gaze of the user does not intersect the display device for a period of time above a time threshold. ### Claim 3 (depends on 1) The method of claim 1, wherein converting the video content to the audio content further comprises converting textual content in the video content to audio information in the audio content. ### Claim 4 (depends on 1) The method of claim 1, wherein converting the video content to the audio content further comprises converting motions in the video content to audio information in the audio content. ### Claim 5 (depends on 1) The method of claim 1, wherein converting the video content to the audio content further comprises converting scene information in the video content to audio information in the audio content. ### Claim 6 (canceled) ### Claim 7 (depends on 1) The method of claim 1, wherein interleaving the audio content with the preexisting audio content further comprises inserting the audio content into portions of the video content without any preexisting audio content. ### Claim 8 (canceled) ### Claim 9 (depends on 1) The method of claim 1, wherein the interleaving the audio content with the preexisting audio content further comprises pausing the video content and inserting the audio content while the video content is paused. ### Claim 10 (depends on 1) The method of claim 1, further comprising: determining the user is not distracted based on a second direction of gaze of the user with respect to the display device; and stopping outputting the audio content to the user. ### Claim 11 (depends on 1) The method of claim 1, further comprising: determining that the video content is a secondary priority to the user; and stopping outputting the audio content. ### Claim 12 (depends on 1) The method of claim 1, further comprising: receiving feedback from the user in response to outputting the audio content to the user; and updating the machine learning model based on the feedback. ### Claim 13 (depends on 12) The method of claim 12, wherein the feedback comprises verbal feedback received at a microphone. ### Claim 14 (depends on 1) The method of claim 1, further comprising: generating a textual summary of the video content while the user is distracted; and presenting the textual summary to the user. ### Claim 15 (depends on 14) The method of claim 14, wherein the textual summary is presented on the display device. ### Claim 16 (depends on 14) The method of claim 14, wherein the textual summary transmitted to a personal device associated with the user and presented on the personal device. ### Claim 17 (depends on 1) The method of claim 1, wherein the method is performed by a video conversion system according to software that is downloaded to the video conversion system from a remote data processing system. ### Claim 18 (depends on 17) The method of claim 17, wherein the method further comprises: metering a usage of the software; and generating an invoice based on metering the usage. ### Claim 19 A system comprising: one or more processors; and one or more computer-readable storage media storing program instructions which, when executed by the one or more processors, are configured to cause the one or more processors to perform a method comprising: determining, using data from at least one camera, that a user is distracted based on a direction of gaze of the user with respect to a display device presenting video content; converting, by a machine learning model, the video content to audio content in response to determining the user is distracted, wherein converting the video content to the audio content further comprises: identifying salient features of the video content; converting, to the audio content, a subset of the salient features that do not match any preexisting audio content; and interleaving the audio content with the preexisting audio content by increasing a speed of the preexisting audio content in order to create respective portions of the video content without any preexisting audio content and inserting the audio content into the respective portions; and outputting, using at least one speaker, the audio content to the user while the user is distracted. ### Claim 20 A computer program product comprising one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions comprising instructions configured to cause one or more processors to perform a method comprising: determining, using data from at least one camera, that a user is distracted based on a direction of gaze of the user with respect to a display device presenting video content; converting, by a machine learning model, the video content to audio content in response to determining the user is distracted, wherein converting the video content to the audio content further comprises: identifying salient features of the video content; converting, to the audio content, a subset of the salient features that do not match any preexisting audio content; and interleaving the audio content with the preexisting audio content by increasing a speed of the preexisting audio content in order to create respective portions of the video content without any preexisting audio content and inserting the audio content into the respective portions; and outputting, using at least one speaker, the audio content to the user while the user is distracted.