# US 8,786,781 B2 · Detection and enhancement of in-video text > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,786,781 B2 - Document ID: US8786781B2 - Status: Granted - Kind: B2 - Application number: 12/421,216 - Filed: 2009-04-09 - Published: 2010-10-14 - Granted: 2014-07-22 - Earliest claimed: 2009-04-09 - Assignees: ATI Technologies ULC - Applicants: Philip Swan - Inventors: Philip Swan - Web page: https://patentgrep.com/patent/US8786781B2 ## CPC - H04N 21/234336: {by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text} - G06V 20/635: {Overlay text, e.g. embedded captions in a TV programme} - H04N 21/47: End-user applications - H04N 21/4884: {for displaying subtitles} - G06V 30/10: Character recognition - H04N 7/0885: {for the transmission of subtitles} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The present disclosure relates to methods and apparatus for detecting text information in a video signal that includes subtitles, captions, credits, or other text, and also for applying enhancements to the display of text areas in video. The sharpness and/or contrast ratio of subtitles of detected text areas may be improved. Text areas may be displayed in a magnified form in a separate window on a display, or on a secondary display. Further disclosed are methods and apparatus for extending the duration for which subtitles appear on the display, for organizing subtitles to be displayed in a scrolling format, for allowing the user to control when a subtitle advances to the next subtitle using a remote control, and for allowing a user to scroll back to a past subtitle in cases where the user has not finished reading a subtitle. Additionally, optical character recognition (OCR) technology may be applied to detected areas of a video signal that include text, and the text may then be displayed in a more readable font, displayed in a translated language, or rendered using voice synthesis technology. ## Claims ### Claim 1 A video processing device, comprising: an input interface, configured to receive a video frame; a processor, configured to: modify the received video frame to generate an output video frame if a probability that the received video frame includes text exceeds a threshold; and generate the output video frame if the probability that the received video frame includes text is below the threshold; and an output interface, configured to display the output video frame. ### Claim 2 (depends on 1) The video processing device of claim 1, further comprising: a motion estimator, configured to generate a plurality of motion vectors based on the received video frame; wherein the processor is further configured to generate the probability based on an analysis of the plurality of motion vectors. ### Claim 3 (depends on 1) The video processing device of claim 1, wherein the processor is further configured to modify the received video frame by performing any one or more of: apply an enhancement to a region of the received video frame that includes the text; send the region of the received video frame that includes the text to a different location in the video frame; or send the region of the received video frame that includes the text to a different display. ### Claim 4 (depends on 3) The video processing device of claim 3, wherein the processor is further configured to apply the enhancement by performing any one or more of: adjust a sharpness or a contrast ratio of the region that includes the text; modify a duration for which the text is displayed; select among a plurality of text regions to be displayed; reorder a display order of the plurality of text regions; or apply character recognition to the text. ### Claim 5 (depends on 4) The video processing device of claim 4, wherein the processor is further configured to apply character recognition to the text by performing any one or more of: display the text in a different font; translate the text to a different language; or create an audio version of the text for a separate audio output. ### Claim 6 A method for use in video processing, comprising: receiving a video frame; modifying the received video frame to generate an output video frame if a probability that the received video frame includes text exceeds a threshold; generating the output video frame if the probability that the received video frame includes text is below the threshold; and displaying the output video frame. ### Claim 7 (depends on 6) The method of claim 6, wherein the probability is determined by: generating a plurality of motion vectors based on the received video frame; and generating the probability based on an analysis of the plurality of motion vectors. ### Claim 8 (depends on 6) The method of claim 6, wherein the modifying includes any one or more of: applying an enhancement to a region of the received video frame that includes the text; sending the region of the received video frame that includes the text to a different location in the video frame; or sending the region of the received video frame that includes the text to a different display. ### Claim 9 (depends on 8) The method of claim 8, wherein the enhancement includes any one or more of: adjusting a sharpness or a contrast ratio of the region that includes the text; modifying a duration for which the text is displayed; selecting among a plurality of text regions to be displayed; reordering a display order of the plurality of text regions; or applying character recognition to the text. ### Claim 10 (depends on 9) The method of claim 9, wherein the applying character recognition to the text includes any one or more of: displaying the text in a different font; translating the text to a different language; or creating an audio version of the text for a separate audio output. ### Claim 11 A non-transitory computer-readable storage medium storing a set of instructions for execution by a general purpose computer to perform method for use in video processing, the set of instructions comprising: a receiving code segment for receiving a video frame; a modifying code segment for modifying the received video frame to generate an output video frame if a probability that the received video frame includes text exceeds a threshold; a generating code segment for generating the output video frame if the probability that the received video frame includes text is below the threshold; and a displaying code segment for displaying the output video frame. ### Claim 12 (depends on 11) The non-transitory computer-readable storage medium of claim 11, wherein the probability is determined by: a second generating code segment for generating a plurality of motion vectors based on the received video frame; and a third generating code segment for generating the probability based on an analysis of the plurality of motion vectors. ### Claim 13 (depends on 11) The non-transitory computer-readable storage medium of claim 11, wherein the modifying code segment includes any one or more of: an applying code segment for applying an enhancement to a region of the received video frame that includes the text; a first sending code segment for sending the region of the received video frame that includes the text to a different location in the video frame; or a second code segment for sending the region of the received video frame that includes the text to a different display. ### Claim 14 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the enhancement includes any one or more of: an adjusting code segment for adjusting a sharpness or a contrast ratio of the region that includes the text; a second modifying code segment for modifying a duration for which the text is displayed; a selecting code segment for selecting among a plurality of text regions to be displayed; a reordering code segment for reordering a display order of the plurality of text regions; or a second applying code segment for applying character recognition to the text. ### Claim 15 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein the second applying code segment includes any one or more of: a second displaying code segment for displaying the text in a different font; a translating code segment for translating the text to a different language; or a creating code segment for creating an audio version of the text for a separate audio output.