# US 12,183,102 B2 · Enhanced optical character recognition (OCR) image segmentation system and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,102 B2 - Document ID: US12183102B2 - Status: Granted - Kind: B2 - Application number: 17/871,261 - Filed: 2022-07-22 - Published: 2023-01-12 - Granted: 2024-12-31 - Earliest claimed: 2020-06-29 - Assignees: Capital One Services, LLC - Applicants: Capital One Services, LLC - Inventors: Ian Whitestone; Brian Chun-Lai So; Sourabh Mittal - Web page: https://patentgrep.com/patent/US12183102B2 ## CPC - G06V 30/1475: {Inclination or skew detection or correction of characters or of image to be recognised} - G06F 18/2413: based on distances to training or reference patterns - G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes} - G06V 10/30: Noise filtering - G06V 10/32: Normalisation of the pattern dimensions - G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching} - G06V 10/82: using neural networks - G06V 30/1478: {of characters or characters lines} - G06V 30/19: Recognition using electronic means - G06V 30/10: Character recognition - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Optical character recognition (OCR) based systems and methods for extracting and automatically evaluating contextual and identification information and associated metadata from an image utilizing enhanced image processing techniques and image segmentation. A unique, comprehensive integration with an account provider system and other third party systems may be utilized to automate the execution of an action associated with an online account. The system may evaluate text extracted from a captured image utilizing machine learning processing to classify an image type for the captured image, and select an optical character recognition model based on the classified image type. They system may compare a data value extracted from the recognized text for a particular data type with an associated online account data value for the particular data type to evaluate whether to automatically execute an action associated with the online account linked to the image based on the data value comparison. ## Claims ### Claim 1 An optical character recognition method, comprising: optimizing the image quality of an image; performing optical character recognition on the optimized image; evaluating text extracted from the image utilizing a deep learning image classifier to classify an image type for the image, wherein the deep leaning image classifier is trained with additional images generated by scaling and translating original images; selecting an optical character recognition model based on the classified image type; performing optical character recognition on the optimized image utilizing the selected model to generate recognized text and to extract identification and metadata information associated with the image from the recognized text; evaluating the extracted identification and metadata information associated with the image from the recognized text to link the image to an online account associated with the image; comparing a data value extracted from the recognized text for a particular data type with an associated online account data value for the particular data type; and evaluating whether to automatically execute an action associated with the online account linked to the image based on the comparison, wherein the action comprises at least one selected from the group of adding a return mail flag. ### Claim 2 (depends on 1) The optical character recognition method of claim 1, wherein the action comprises executing a visual workflow builder. ### Claim 3 (depends on 1) The optical character recognition method of claim 1, wherein the action comprises flagging the online account for processing. ### Claim 4 (depends on 1) The optical character recognition method of claim 1, further comprising generating a confidence score based on the comparison of the data value extracted from the recognized text with the associated online account data value. ### Claim 5 (depends on 4) The optical character recognition method of claim 4, further comprising evaluating the confidence score based on a determined cutoff value to determine whether to automatically execute the action associated with the online account linked to the image. ### Claim 6 (depends on 1) The optical character recognition method of claim 1, wherein optimizing the image quality of the image comprises computing an angle of skewed text and rotating the image by the computed angle to correct the skew. ### Claim 7 (depends on 1) The optical character recognition method of claim 1, wherein optimizing the image quality of the image comprises at least one selected from the group of de-noising, resizing, and thresholding. ### Claim 8 An optical character recognition system, comprising: a processor; and a memory comprising instructions operable to be executed by the processor, wherein the processor is configured to: optimize the image quality of an image, perform optical character recognition on the optimized image, evaluate text extracted from the image utilizing a deep learning image classifier to classify an image type for the image, wherein the deep leaning image classifier is trained with additional images generated by scaling and translating original images, select an optical character recognition model based on the classified image type, perform optical character recognition on the optimized image utilizing the selected model to generate recognized text and to extract identification and metadata information associated with the image from the recognized text evaluate the extracted identification and metadata information associated with the image from the recognized text to link the image to an online account associated with the image; compare a data value extracted from the recognized text for a particular data type with an associated online account data value for the particular data type; and evaluate whether to automatically execute an action associated with the online account linked to the image based on the comparison, wherein the action comprises at least one selected from the group of adding a return mail flag. ### Claim 9 (depends on 8) The optical character recognition system of claim 8, wherein: the deep learning image classifier comprises a convolutional neural network, and an architecture of the convolutional neural network includes three layers of convolution followed by activation and pooling, a fully connected layer followed by activation and dropout, and a fully connected layer followed by sigmoid activation. ### Claim 10 (depends on 8) The optical character recognition system of claim 8, wherein: the image comprises a digitized document, and the image is received from at least one selected from the group of a scanner, a sensor, a camera, and a mobile device. ### Claim 11 (depends on 8) The optical character recognition system of claim 8, wherein the image type comprises at least one selected from the group of a statement, a letter, a welcome kit, and an unspecified mail. ### Claim 12 A non-transitory computer-accessible medium storing instructions operable to be executed by a processor, wherein, upon execution of the instructions, the processor is configured to perform procedures comprising: optimizing the image quality of an image; performing optical character recognition on the optimized image; evaluating text extracted from the image utilizing a deep learning image classifier to classify an image type for the image, wherein the deep leaning image classifier is trained with additional images generated by scaling and translating original images; selecting an optical character recognition model based on the classified image type; performing optical character recognition on the optimized image utilizing the selected model to generate recognized text and to extract identification and metadata information associated with the image from the recognized text evaluating the extracted identification and metadata information associated with the image from the recognized text to link the image to an online account associated with the image; comparing a data value extracted from the recognized text for a particular data type with an associated online account data value for the particular data type; and evaluating whether to automatically execute an action associated with the online account linked to the image based on the comparison, wherein the action comprises at least one selected from the group of adding a return mail flag. ### Claim 13 (depends on 12) The computer-accessible medium of claim 12, the procedures further comprising: utilizing machine learning to process and cluster the extracted data and associated processing and action information; and generating additional training data for the optical character recognition model using the extracted data. ### Claim 14 (depends on 13) The computer-accessible medium of claim 13, wherein information used to process and cluster comprises at least one selected from the group of a type of mail and an address match. ### Claim 15 (depends on 13) The computer-accessible medium of claim 13, wherein the machine learning comprises at least one selected from the group of gradient boosting machine and logistic regression. ### Claim 16 (depends on 12) The computer-accessible medium of claim 12, the procedures further comprising: generating regions of interest that associate specific areas of the image with particular information; and performing optical character recognition only on the regions of interest.