Back to results
US 12,591,743 B2Granted
Information processing apparatus, information processing method, and storage medium for extracting a named entity from a document
AssigneeCanon Kabushiki Kaisha
InventorTomoaki Higo
Life of this patent
Estimating the term…
Abstract
An information processing apparatus for converting text data from a document image read from a document into a token string and calculates a number of processing times necessary for performing processing in a natural language processing model based on the token string. Then, at the time of division, the information processing apparatus divides the token string into blocks so that at least a portion overlaps between adjacent blocks based on the calculated number of processing times and for each token belonging to the overlap portion between the adjacent blocks, selects one of estimation results obtained from each block.
Claims15
An information processing apparatus that extracts a named entity from a document by using a natural language processing model, the apparatus comprising:
- one or more memories storing instructions;
- and
- one or more processors executing the instructions to:
- (a) obtain a maximum number (L) of tokens that can be input at a time to the natural language processing model;
- (b) obtain text data from a document image read from the document;
- (c) convert the text data into a token string by performing processing to decompose the text data into each token;
- (d) divide the token string into blocks that can be processed by the natural language processing model, each of the block is a group of tokens;
- (e) calculate a number of processing times (P) necessary for performing processing in the natural language processing model based on a length (T) of the token string, the maximum number of tokens (L), and a number of tokens (M) desired to be overlapped between adjacent blocks among blocks generated by dividing the token string;
- (f) estimate a named entity by performing processing to input each of the blocks to the natural language processing model;
- and
- (g) provide a user interface for causing a user to select a desired named entity from estimation results obtained from each of the blocks, wherein
- (h) in the calculating, a minimum number of processing times necessary for processing all of the tokens included in the token string as the number of processing times (P) necessary for performing processing in the natural language processing model is calculated;
- and
- (i) in the dividing, based on the calculated minimum number of processing times (P), the token string is divided into the blocks so that M tokens to be overlapped between the two adjacent blocks and
- (j) in the processing, for each of the token belonging to the overlap portion which are overlapped M tokens between the two adjacent blocks, one of estimation results obtained from each of the blocks is selected and
- (k) in the processing, on a condition that the estimation results obtained from the tokens belonging to the overlap portion are different and an estimation result obtained from one of the tokens belonging to the overlap portion is an estimation result representing not being a named entity, estimation results representing a named entity are selected.
13 more elements
An information processing method of extracting a named entity from a document by using a natural language processing model, the method comprising the steps of:
- obtaining a maximum number (L) of tokens that can be input at a time to the natural language processing model;
- obtaining text data from a document image read from the document;
- converting the text data into a token string by performing processing to decompose the text data into each token;
- dividing the token string into blocks that can be processed by the natural language processing model, each of the block is a group of tokens;
- calculating a number of processing times (P) necessary for performing processing in the natural language processing model based on a length (T) of the token string, the maximum number of tokens (L), and a number of tokens (M) desired to be overlapped between adjacent blocks among blocks generated by dividing the token string;
- estimating a named entity by performing processing to input each of the blocks to the natural language processing model;
- and
- providing a user interface for causing a user to select a desired named entity from estimation results obtained from each of the blocks, wherein
- in the calculating, a minimum number of processing times necessary for processing all of the tokens included in the token strings as the number of processing times (P) necessary for performing processing in the natural language processing model is calculated;
- and
- in the dividing, based on the calculated minimum number of processing times (P), the token string is divided into the blocks so that M tokens to be overlapped between the two adjacent blocks and
- in the processing, for each of the token belonging to the overlap portion which are overlapped M tokens between the two adjacent blocks, one of estimation results obtained from each of the blocks is selected and
- in the processing, on a condition that the estimation results obtained from the tokens belonging to the overlap portion are different and an estimation result obtained from one of the tokens belonging to the overlap portion is an estimation result representing not being a named entity, estimation results representing a named entity are selected.
10 more elements
A non-transitory computer readable storage medium storing a program for causing a computer to perform an information processing method of extracting a named entity from a document by using a natural language processing model, the method comprising the steps of:
- obtaining a maximum number (L) of tokens that can be input at a time to the natural language processing model;
- obtaining text data from a document image read from the document;
- converting the text data into a token string by performing processing to decompose the text data into each token;
- dividing the token string into blocks that can be processed by the natural language processing model, each of the blocks is a group of tokens;
- calculating a number of processing times (P) necessary for performing processing in the natural language processing model based on a length (T) of the token string, the maximum number of tokens (L), and a number of tokens (M) desired to be overlapped between adjacent blocks among blocks generated by dividing the token string;
- estimating a named entity by performing processing to input each of the blocks to the natural language processing model;
- and
- providing a user interface for causing a user to select a desired named entity from estimation results obtained from each of the blocks, wherein
- in the calculating, a minimum number of processing times necessary for processing all of the tokens included in the token string as the number of processing times (P) necessary for performing processing in the natural language processing model is calculated;
- and
- in the dividing, based on the calculated minimum number of processing times (P), the token string is divided into the blocks so that M tokens to be overlapped between the two adjacent blocks and
- in the processing, for each of the token belonging to the overlap portion which are overlapped M tokens between the two adjacent blocks, one of estimation results obtained from each of the blocks is selected and
- in the processing, on a condition that the estimation results obtained from the tokens belonging to the overlap portion are different and an estimation result obtained from one of the tokens belonging to the overlap portion is an estimation result representing not being a named entity, estimation results representing a named entity are selected.
10 more elements
Dependencies are inferred from the claim text.