# US 9,520,102 B2 · Text extraction from graphical user interface content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,520,102 B2 - Document ID: US9520102B2 - Status: Granted - Kind: B2 - Application number: 13/872,172 - Filed: 2013-04-29 - Published: 2014-10-30 - Granted: 2016-12-13 - Earliest claimed: 2013-04-29 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Amir Geva; Mattias Marder - Web page: https://patentgrep.com/patent/US9520102B2 ## CPC - G06V 30/413: Classification of content, e.g. text, photographs or tables - G06V 10/267: {by performing operations on regions, e.g. growing, shrinking or watersheds} - G09G 5/02: characterised by the way in which colour is displayed {(details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607)} - H04N 1/40012: {Conversion of colour to monochrome} - H04N 1/40062: {Discrimination between different image types, e.g. two-tone, continuous tone} - G06V 2201/02: Recognising information on displays, dials, clocks - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems and methods for extracting text from images rendered on a display screen, the method comprising capturing a color image rendered on a display screen; and transforming the color image to binary color image, preserving text-like graphic components and filtering out non-text-like graphical components. The transforming comprises scanning one or more areas of the color image; and detecting continuous bi-tonal regions in the scanned one or more areas, wherein the continuous bi-tonal regions have large variances. ## Claims ### Claim 1 A method for extracting text from images rendered on a display screen, the method comprising: capturing a color image rendered on a display screen; and transforming the color image to binary color image, preserving textual graphic components and filtering out non-textual graphical components, said transforming comprises: scanning one or more areas of the color image; detecting continuous regions in the scanned one or more areas, the continuous regions having two distinguishable shades, wherein a first shade represents the foreground and a second shade represents the background, and wherein the continuous regions comprise multiple components; representing a set having a component from the continuous regions; and extracting the component as text component, wherein the component is recognized based on switches between runs where a single color part of the run comprises a lower number of pixels than all other single color parts of the run. ### Claim 2 (depends on 1) The method of claim 1, wherein the binary image comprises black and white colors. ### Claim 3 (depends on 1) The method of claim 1, wherein the binary image is a gray scaled image. ### Claim 4 (depends on 1) The method of claim 1, wherein the detecting comprises collecting horizontal sets of pixels in the continuous regions that contain two shades. ### Claim 5 (depends on 4) The method of claim 4, wherein the collected horizontal sets of pixels are combined into bi-level regions. ### Claim 6 (depends on 5) The method of claim 5, wherein pixels associated with a foreground color are distinguished from pixels from a background color, in response to determining that a bi-level region contains text. ### Claim 7 A system for extracting text from images rendered on a display screen, the system comprising: a logic unit for capturing a color image rendered on a display screen; and a logic unit for transforming the color image to binary color image, preserving textual graphic components and filtering out non-textual graphical components, wherein the transforming comprises: scanning one or more areas of the color image; detecting continuous regions in the scanned one or more areas, the continuous regions having two distinguishable shades, wherein a first shade represents the foreground and a second shade represents the background, and wherein the continuous regions comprise multiple components; representing a set having a component from the one or more continuous regions; and extracting the component as text component, wherein the component is recognized based on switches between runs where a single color part of the run comprises a lower number of pixels than all other single color parts of the run. ### Claim 8 A computer program product comprising a non-transitory computer readable storage medium having a computer readable program, wherein the computer readable program when executed on a computer causes the computer to: capture a color image rendered on a display screen; and transform the color image to binary color image, preserving textual graphic components and filtering out non-textual graphical components, wherein the transforming comprises: scanning one or more areas of the color image; detecting continuous regions in the scanned one or more areas, the continuous regions having two distinguishable shades, wherein a first shade represents the foreground and a second shade represents the background wherein the continuous regions comprise multiple components; representing a set having a component from the one or more continuous regions; and extracting the component as text component, wherein the component is recognized based on switches between runs where a single color part of the run comprises a lower number of pixels than all other single color parts of the run.