US 11,769,239 B1Granted

Model based document image enhancement

AssigneeIntuit Inc.

InventorsJiaxin Zhang, Tharathorn Joy Rimchala, Lalla Mouatadid, Kamalika Das, Sricharan Kallur Palli Kumar

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods are disclosed for model based document image enhancement. Instead of requiring paired dirty and clean images for training a model to clean document images (which may cause privacy concerns), two models are trained on the unpaired images such that only the dirty images are accessed or only the clean images are accessed at one time. One model is a first implicit model to translate the dirty images from a source space to a latent space, and the other model is a second implicit model to translate the images from the latent space to clean images in a target space. The second implicit model is trained based on translating electronic document images in the target space to the latent space. In some implementations, the implicit models are diffusion models, such as denoising diffusion implicit models based on solving ordinary differential equations.

Claims12

Dependent claims

A computer-implemented method for document image enhancement, the method comprising:

obtaining an electronic document image by a machine learning (ML) model, wherein the electronic document image is generated from scanning a physical document;
and
enhancing the electronic document image by the ML model, including:
translating the electronic document image in a source space to a latent space by a first implicit probabilistic model of the ML model, wherein the first implicit probabilistic model is trained based on translating electronic document images in the source space to the latent space;
and
translating the electronic document image in the latent space to a target space by a second implicit probabilistic model of the ML model, wherein:
the second implicit probabilistic model is trained independently from the first implicit probabilistic model;
the second implicit probabilistic model is trained based on translating electronic document images in the target space to the latent space;
and
the second implicit probabilistic model includes a second cycle consistent model to translate the electronic document image between the latent space and the target space, wherein the second cycle consistent model is configured to reverse the function of translating a target space document image to a latent space document image to translate the electronic document image in the latent space to the target space;
and
providing the enhanced electronic document image in the target space to an object character recognition (OCR) engine to perform OCR.

9 more elements

A computing system for document image enhancement, the computing system comprising:

one or more processors;
and
a memory storing instructions that, when executed by the one or more processors, causes the computing system to perform operations comprising:
obtaining an electronic document image by a machine learning (ML) model of the computing system, wherein the electronic document image is generated from scanning a physical document;
enhancing the electronic document image by the ML model, including:
translating the electronic document image in a source space to a latent space by a first implicit probabilistic model of the ML model, wherein the first implicit probabilistic model is trained based on translating electronic document images in the source space to the latent space;
and
translating the electronic document image in the latent space to a target space by a second implicit probabilistic model of the ML model, wherein:
the second implicit probabilistic model is trained independently from the first implicit probabilistic model;
the second implicit probabilistic model is trained based on translating electronic document images in the target space to the latent space;
and
the second implicit probabilistic model includes a second cycle consistent model to translate the electronic document image between the latent space and the target space, wherein the second cycle consistent model is configured to reverse the function of translating a target space document image to a latent space document image to translate the electronic document image in the latent space to the target space;
and
providing the enhanced electronic document image in the target space to an object character recognition (OCR) engine to perform OCR.

11 more elements

Dependencies are inferred from the claim text.