US 12,424,010 B2Granted

Character recognition model training method and apparatus, character recognition method and apparatus, device and storage medium

AssigneeBeijing Baidu Netcom Science Technology Co., Ltd.

InventorsPengyuan Lv, Chengquan Zhang, Shanshan Liu, Meina Qiao, Yangliu Xu, Liang Wu, Xiaoyan Wang, Kun Yao, Junyu Han, Errui Ding, Jingdong Wang, Tian Wu, Haifeng Wang

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

The present disclosure provides a character recognition model training method and apparatus, a character recognition method and apparatus, a device and a medium, relating to the technical field of artificial intelligence, and specifically to the technical fields of deep learning, image processing and computer vision, which can be applied to scenarios such as character detection and recognition technology. The specific implementing solution is: partitioning an untagged training sample into at least two sub-sample images; dividing the at least two sub-sample images into a first training set and a second training set; where the first training set includes a first sub-sample image with a visible attribute, and the second training set includes a second sub-sample image with an invisible attribute; performing self-supervised training on a to-be-trained encoder by taking the second training set as a tag of the first training set, to obtain a target encoder.

Claims17

Dependent claims

A character recognition method being applied to a server and comprising:

partitioning an untagged training sample into at least two sub-sample images;
dividing the at least two sub-sample images into a first training set and a second training set;
wherein the first training set comprises a first sub-sample image with a visible attribute, and the second training set comprises a second sub-sample image with an invisible attribute;
performing self-supervised training on a to-be-trained encoder by taking the second training set as a tag of the first training set, to obtain a target encoder;
wherein the performing the self-supervised training on the to-be-trained encoder by taking the second training set as the tag of the first training set, to obtain the target encoder comprises:
initializing the to-be-trained encoder to obtain a first encoder;
extracting, based on the first encoder, a first visual feature of the first sub-sample image in the first training set and a second visual feature of the second sub-sample image in the second training set;
performing mask query calculation on the first visual feature, to obtain a third visual feature;
and
updating the first encoder according to a feature error between the third visual feature and the second visual feature until the feature error satisfies a first error condition, and determining a latest updated first encoder as the target encoder;
wherein the updating the first encoder according to the feature error between the third visual feature and the second visual feature until the feature error satisfies the first error condition, and the determining the latest updated first encoder as the target encoder comprise:
initializing a to-be-trained decoder to obtain a first decoder;
determining, based on the first decoder, an image error generated when image reconstruction is performed on the third visual feature;
determining the feature error between the third visual feature and the second visual feature;
and
updating the first encoder based on the feature error and the image error and updating the first decoder based on the image error until the feature error satisfies the first error condition and the image error satisfies a second error condition, and determining a latest obtained first encoder as the target encoder;
receiving a to-be-recognized image sent by a terminal device, and performing, based on the target encoder and the updated first decoder, image features extraction on the to-be-recognized image to obtain a target text;
and
sending the target text to the terminal device.

16 more elements

A character recognition model training method comprising:

partitioning a synthetic sample into at least two sub-synthetic images, wherein the synthetic sample comprises a synthetic text tag;
dividing the at least two sub-synthetic images as a first synthetic set and a second synthetic set;
wherein the first synthetic set comprises a first sub-synthetic image with a visible attribute, and the second synthetic set comprises a second sub-synthetic image with an invisible attribute;
and
performing, based on the first synthetic set and the second synthetic set, supervised training on a to-be-trained decoder to obtain a target decoder corresponding to the to-be-trained decoder;
wherein the performing, based on the first synthetic set and the second synthetic set, the supervised training on the to-be-trained decoder to obtain the target decoder corresponding to the to-be-trained decoder comprises:
extracting, based on a target encoder, a first feature sequence of the first sub-synthetic image in the first synthetic set;
wherein the target encoder is obtained by performing following steps:
partitioning an untagged training sample into at least two sub-sample images;
dividing the at least two sub-sample images into a first training set and a second training set;
wherein the first training set comprises a first sub-sample image with a visible attribute, and the second training set comprises a second sub-sample image with an invisible attribute;
and
performing self-supervised training on a to-be-trained encoder by taking the second training set as a tag of the first training set, to obtain the target encoder;
performing feature completion on the first feature sequence according to an image position, in the synthetic sample, of the second sub-synthetic image in the second synthetic set, to obtain a second feature sequence;
and
training, by taking that a predictive text of the second feature sequence predicted by the to-be-trained decoder is the same as a synthetic text of the second sub-synthetic image in the synthetic text tag as a training objective, to obtain the target decoder corresponding to the to-be-trained decoder.

13 more elements

A character recognition method comprising:

determining a target encoder and a target decoder;
wherein the target encoder is obtained by performing following steps:
partitioning an untagged training sample into at least two sub-sample images;
dividing the at least two sub-sample images into a first training set and a second training set;
wherein the first training set comprises a first sub-sample image with a visible attribute, and the second training set comprises a second sub-sample image with an invisible attribute;
and
performing self-supervised training on a to-be-trained encoder by taking the second training set as a tag of the first training set, to obtain the target encoder;
wherein the target decoder is obtained by performing following steps:
partitioning a synthetic sample into at least two sub-synthetic images, wherein the synthetic sample comprises a synthetic text tag;
dividing the at least two sub-synthetic images as a first synthetic set and a second synthetic set;
wherein the first synthetic set comprises a first sub-synthetic image with a visible attribute, and the second synthetic set comprises a second sub-synthetic image with an invisible attribute;
and
performing, based on the first synthetic set and the second synthetic set, supervised training on a to-be-trained decoder to obtain the target decoder corresponding to the to-be-trained decoder;
partitioning a to-be-recognized image into at least two sub-images;
extracting image features of the sub-images based on the target encoder, to obtain the image features corresponding to the at least two sub-images respectively;
performing, based on the target decoder and at least two query vectors, decoding calculation on the image features corresponding to the at least two sub-images respectively, to obtain a decoded feature sequence corresponding to the at least two sub-images;
and
obtaining a target text of the to-be-recognized image according to the decoded feature sequence.

15 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/168,759Filed 14 Feb 2023
Priority: CN 202210983230.916 Aug 2022
Published: US 2023/0215203 A16 Jul 2023
Granted: 23 Sep 2025
Term ends: Not estimatedThe source records don’t support an estimate.

Classification71 codes

G06V 10/82
Using pattern recognition or machine learning: using neural networks
G06V 30/19147
Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation:…
G06F 7/764
Arrangements for rearranging, permuting or selecting data according to predetermined rules, independently of the content of the data: masking
G06F 16/24
Of structured data, e.g. relational data: querying