US 2013/0170749 A1Published application

Method and apparatus for document image indexing and retrieval using multi-level document image structure and local features

AssigneeKonica Minolta Laboratory U.S.A., Inc.

InventorYibin TIAN

Life of this patent

Estimating the term…

Loading patent…

Abstract

An image based document index and retrieval method is described. During document indexing, each source document is analyzed to generate index information at document, page, region and unit levels. Region and unit level index information is generated by segmenting each text region into units, constructing unit length or unit density histograms, and analyzing the units in a few most frequent bins of the histogram. The index information and the source document images are stored in a database. During document retrieval, a target document is analyzed to generate target index information in the same way as during document indexing. The target index information is compared to stored index information in a progressive manner (from higher to lower levels) to identify source documents with index information that matches the target index information. Fuzzy logic is used in the comparison steps to increase the robustness of the document retrieval.

Claims18

Dependent claims

An image-based document indexing method comprising:

(a) obtaining a document image including at least one page;
(b) segmenting the page of the document image into one or more text regions;
(c) analyzing the page of the document image to generate page-level index information;
(d) for each text region in the page:
(d1) segmenting the region into a plurality of units each having a unit length and/or a unit density;
and
(d2) generating a unit length histogram and/or a unit density histogram, wherein the unit length histogram has a plurality of bins and represents a relationship between the unit length and a frequency of occurrence of the units within the region, the unit density histogram has a plurality of bins and represents a relationship between the unit density and a frequency of occurrence of the units within the region, and wherein each unit of the region belongs to a bin of the unit length histogram or the unit density histogram;
wherein the unit length histogram or unit density histogram forms region-level index information for the region;
(e) for each text region, and for each of a number of designated bins of the unit length histogram or unit density histogram, including a most frequent bin and zero or more bins around the most frequent bin:
(e1) characterizing each unit belonging to the bin into a unit type and obtain a number of unit types in the bin;
and
(e2) for each of a number of designated unit types, obtaining a number of units belonging to the unit type, locations of such units, and a unit feature vector or unit image patch of the unit type;
wherein the identity of the designated bins, the numbers of unit types in each designated bin, the number of units belonging to each designated unit type and the locations of such units, and the unit feature vector or unit image patch of each designated unit type collectively form unit-level index information for the region;
and
wherein the page-level index information, the region-level index information and the unit-level index information collectively form the document index information of the document image.

12 more elements

An image-based document retrieval method comprising:

(a) obtaining index information for a target document image, the index information including page-level index information that includes geometric characteristics of a page of the target document, region-level index information that includes geometric characteristics of text regions of the page of the target document, and unit-level index information that includes geometric characteristics of text units in each text region of the page of the target document;
(b) accessing a database which stores index information for a plurality of source documents, the index information including, for each source document, page-level index information that includes geometric characteristics of a page of the source document, region-level index information that includes geometric characteristics of text regions of the page of the source document, and unit-level index information that includes geometric characteristics of text units in each text region of the page of the source document;
(c) comparing the page-level index information for the page of the target document image with stored page-level index information for a plurality of source documents to identify one or more pages of one or more source documents with page-level index information that matches the page-level index information of the page of the target document;
(d) comparing the region-level index information for the page of the target document image with stored region-level index information for the one or more pages identified in step (c) to further identify one or more pages of one or more source documents with region-level index information that matches the region-level index information of the page of the target document;
(e) comparing the unit-level index information for the page of the target document image with stored unit-level index information for the one or more pages identified in step (d) to further identify one or more pages of one or more source documents with unit-level index information that matches the unit-level index information of the page of the target document.

2 more elements

A computer program product comprising a computer usable non-transitory medium having a computer readable program code embedded therein for controlling a data processing apparatus, the computer readable program code configured to cause the data processing apparatus to execute an image-based document indexing process which comprises:

(a) obtaining a document image including at least one page;
(b) segmenting the page of the document image into one or more text regions;
(c) analyzing the page of the document image to generate page-level index information;
(d) for each text region in the page:
(d1) segmenting the region into a plurality of units each having a unit length and/or a unit density;
and
(d2) generating a unit length histogram and/or a unit density histogram, wherein the unit length histogram has a plurality of bins and represents a relationship between the unit length and a frequency of occurrence of the units within the region, the unit density histogram has a plurality of bins and represents a relationship between the unit density and a frequency of occurrence of the units within the region, and wherein each unit of the region belongs to a bin of the unit length histogram or the unit density histogram;
wherein the unit length histogram or unit density histogram forms region-level index information for the region;
(e) for each text region, and for each of a number of designated bins of the unit length histogram or unit density histogram, including a most frequent bin and zero or more bins around the most frequent bin:
(e1) characterizing each unit belonging to the bin into a unit type and obtain a number of unit types in the bin;
and
(e2) for each of a number of designated unit types, obtaining a number of units belonging to the unit type, locations of such units, and a unit feature vector or unit image patch of the unit type;
wherein the identity of the designated bins, the numbers of unit types in each designated bin, the number of units belonging to each designated unit type and the locations of such units, and the unit feature vector or unit image patch of each designated unit type collectively form unit-level index information for the region;
and
wherein the page-level index information, the region-level index information and the unit-level index information collectively form the document index information of the document image.

12 more elements

(a) obtaining index information for a target document image, the index information including page-level index information that includes geometric characteristics of a page of the target document, region-level index information that includes geometric characteristics of text regions of the page of the target document, and unit-level index information that includes geometric characteristics of text units in each text region of the page of the target document;
(b) accessing a database which stores index information for a plurality of source documents, the index information including, for each source document, page-level index information that includes geometric characteristics of a page of the source document, region-level index information that includes geometric characteristics of text regions of the page of the source document, and unit-level index information that includes geometric characteristics of text units in each text region of the page of the source document;
(c) comparing the page-level index information for the page of the target document image with stored page-level index information for a plurality of source documents to identify one or more pages of one or more source documents with page-level index information that matches the page-level index information of the page of the target document;
(d) comparing the region-level index information for the page of the target document image with stored region-level index information for the one or more pages identified in step (c) to further identify one or more pages of one or more source documents with region-level index information that matches the region-level index information of the page of the target document;
(e) comparing the unit-level index information for the page of the target document image with stored unit-level index information for the one or more pages identified in step (d) to further identify one or more pages of one or more source documents with unit-level index information that matches the unit-level index information of the page of the target document.

2 more elements

Dependencies are inferred from the claim text.

Facts

Application: 13/340,513Filed 29 Dec 2011
Published: 4 Jul 2013
Term ends: Not estimatedThe source records don’t support an estimate.

Classification6 codes

G06F 16/93
Details of database functions independent of the retrieved data types: document management systems
G06V 30/18086
Extraction of features or characteristics of the image: by performing operations within image blocks or by using histograms
G06V 30/413
Analysis of document content: classification of content, e.g. text, photographs or tables
G06V 30/10
Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition: character recognition