US 10,769,484 B2 · Character detection method and apparatus

Abstract

Disclosed embodiments relate to a character detection method and apparatus. In some embodiments, the method includes: using an image including an annotated word as an input to a machine learning model; selecting, based on a predicted result of characters inside an annotation region of the annotated word predicted and annotation information of the annotated word, characters for training the machine learning model from the characters inside the annotation region of the annotated word predicted; and training the machine learning model based on features of the selected characters. This implementation manner implements the full training of a machine learning model by using existing word level annotated images, to obtain a machine learning model capable of detecting characters in images, thereby reducing the costs for the training of a machine learning model capable of detecting characters in images.

A character detection method, comprising:

using an image including an annotated word as an input to a machine learning model, comprising:
using a word level annotated image in a word level annotated dataset as the image including the annotated word, the word level annotated image comprising an annotation box surrounding the word for annotating the position of the word;
selecting, based on a predicted result for characters being inside an annotation region of the annotated word and predicted by the machine learning model and annotation information of the annotated word, characters for training the machine learning model from the characters being inside the annotation region of the annotated word and predicted by the machine learning model, wherein the predicted result comprises:
bounding boxes corresponding to the characters being inside the annotation region of the annotated word and confidence levels corresponding to the characters being inside the annotation region of the annotated word, and the annotation information comprises a bounding box corresponding to the annotated word;
training the machine learning model based on features of the selected characters;
and
detecting characters in an image by using the trained machine learning model,
wherein the selecting characters for training the machine learning model from the characters being inside the annotation region of the annotated word and predicted by the machine learning model comprises:
calculating k neighbors for the bounding boxes corresponding to the characters being inside the annotation region of the annotated word and predicted by the machine learning model, to obtain a connection relationship between the characters, wherein each of the characters is connected to k other characters;
calculating a weight wij between two connected characters by using the following formula:
w
ij
=
exp
⁡
(
-
d
⁡
(
i
,
j
)
d
_
)
·
(
t
i
+
t
j
)
wherein the two connected characters constitute a character connection pair, d(i, j) represents a distance between the two connected characters, d represents an average distance between characters in all character connection pairs, and ti and tj represent respective confidence levels corresponding to the two connected characters;
finding a maximum spanning tree, wherein the maximum spanning tree comprises sequentially connected characters predicted by the machine learning model, and the sum of the weights between the characters is the greatest;
executing the following selection operation:
pruning each character connection pair in a current tree to obtain multiple subtrees, wherein when the selection operation is executed for the first time, the current tree is the maximum spanning tree;
calculating a score s of a subtree or the current tree by using the following formula:
s
=
w
·
s
⁢
⁢
1
+
(
1
-
w
)
·
s
⁢
⁢
2
s
⁢
⁢
1
=
area
⁡
(
B
chars
)
area
⁡
(
B
anno
)
s
⁢
⁢
2
=
1
-
λ
2
λ
1
wherein Bchars represents a bounding box corresponding to a character in the subtree or the current tree, Banno represents the bounding box corresponding to the annotated word, area(Bchars) represents the area of the bounding boxes corresponding to the characters in the subtree or the current tree, area(Banno) represents the area of the bounding box corresponding to the annotated word, λ1 and λ2 respectively represent the greatest feature value and the second greatest feature value of a covariance matrix of the center coordinates of Bchars, w is a preset weight when the selection operation is executed for the first time, and w is the weight between the two characters in the character connection pair corresponding to the subtree when the selection operation is not executed for the first time;
determining whether the greatest of the scores of the subtrees is greater than the score of the current tree;
and
if yes, using the subtree with the greatest score as the current tree, and executing the selection operation again;
or
if not, using the characters in the current tree as the characters for training the machine learning model.

90 more elements

Character detection method and apparatus

Life of this patent

Character detection method and apparatus

Life of this patent

Abstract

Claims13