# US 2023/0267755 A1 · Object recognition processing method, processing apparatus, electronic device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267755 A1
- Document ID: US20230267755A1
- Status: Published application
- Kind: A1
- Application number: 18/022,523
- Filed: 2021-07-27
- Published: 2023-08-24
- Earliest claimed: 2020-08-25
- Assignees: Hangzhou Dana Technology Inc.
- Applicants: Hangzhou Dana Technology Inc.
- Inventors: Tao HE; Fan SHI; Huan LUO; Mingquan CHEN
- Web page: https://patentgrep.com/patent/US20230267755A1

## CPC

- G06V 30/16: Image preprocessing
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06V 10/82: using neural networks
- G06V 20/00: Scenes; Scene-specific elements (control of digital cameras H04N23/60)
- G06N 5/046: Forward inferencing; Production systems
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Disclosed in the present disclosure are an object recognition processing method, a processing apparatus, an electronic device, and a non-transitory computer-readable storage medium. The method includes: obtaining an object to be recognized; recognizing a type of said object on the basis of a type recognition model; determining a processing rule corresponding to said object; in response to the fact that the type of said object is a basic type, taking said object as a target object to be recognized, and in response to the fact that the type of said object is a non-basic type, transforming said object by means of a transformer learning model, so as to transform said object into the target object to be recognized; and performing, by means of the transformer learning model, recognition processing on the target object to be recognized to obtain a target result corresponding to the target object to be recognized.

## Claims

### Claim 1

An object recognition processing method, comprising:
obtaining an object to be recognized;
recognizing a type of the object to be recognized based on a type recognition model;
determining, according to the type of the object to be recognized, a processing rule corresponding to the object to be recognized;
performing, according to the processing rule, a recognition processing on the object to be recognized by means of a transformer learning model, so as to obtain a target result corresponding to the object to be recognized,
wherein the step of performing, according to the processing rule, the recognition processing on the object to be recognized by means of the transformer learning model, so as to obtain the target result corresponding to the object to be recognized comprises:
in response to that the type of the object to be recognized is a basic type, taking, according to the processing rule, the object to be recognized as a target object to be recognized, and in response to that the type of the object to be recognized is a non-basic type, transforming the object to be recognized by means of the transformer learning model according to the processing rule, so as to transform the object to be recognized into the target object to be recognized;
performing the recognition processing on the target object to be recognized by means of the transformer learning model, so as to obtain the target result corresponding to the object to be recognized,
wherein a type of the target object to be recognized is the basic type.

### Claim 2 (depends on 1)

The object recognition processing method according to claim 1, wherein the step of performing the recognition processing on the target object to be recognized by means of the transformer learning model, so as to obtain the target result corresponding to the object to be recognized further comprises:
performing the recognition processing on the target object to be recognized by means of the transformer learning model, so as to obtain a processing result corresponding to the target object to be recognized;
processing the processing result according to the processing rule, so as to obtain the target result corresponding to the object to be recognized.

### Claim 3 (depends on 2)

The object recognition processing method according to claim 2, wherein the basic type comprises calculation-based questions or knowledge-based questions,
the non-basic type comprises calculation-based fill-in-the-blank questions, calculation-based true-false questions, calculation-based multiple-choice questions, knowledge-based fill-in-the-blank questions, knowledge-based true-false questions, or knowledge-based multiple-choice questions.

### Claim 4 (depends on 3)

The object recognition processing method according to claim 3, wherein the step of transforming the object to be recognized by means of the transformer learning model, so as to transform the object to be recognized into the target object to be recognized comprises:
in response to that the type of the object to be recognized is the calculation-based fill-in-the-blank questions or the knowledge-based fill-in-the-blank questions, transforming the object to be recognized into the target object to be recognized directly by means of the transformer learning model;
in response to that the type of the object to be recognized is the calculation-based true-false questions, deleting a judgment result from the object to be recognized by means of the transformer learning model, so as to transform the object to be recognized into a first intermediate object to be recognized, and transforming the first intermediate object to be recognized into the target object to be recognized, wherein a type of the first intermediate object to be recognized is the calculation-based fill-in-the-blank questions;
in response to that the type of the object to be recognized is the calculation-based multiple-choice questions or the knowledge-based multiple-choice questions, deleting respective options from the object to be recognized by means of the transformer learning model, and transforming a question stem in the object to be recognized into the target object to be recognized;
in response to that the type of the object to be recognized is the knowledge-based true-false questions, transforming the object to be recognized into a third intermediate object to be recognized by means of the transformer learning model, wherein a type of the third intermediate object to be recognized is the knowledge-based multiple-choice questions, deleting respective options from the third intermediate object to be recognized, transforming a question stem in the third intermediate object to be recognized into a second intermediate object to be recognized, and transforming the second intermediate object to be recognized into the target object to be recognized, wherein a type of the second intermediate object to be recognized is the knowledge-based fill-in-the-blank questions.

### Claim 5 (depends on 4)

The object recognition processing method according to claim 4, wherein the step of determining, according to the type of the object to be recognized, the processing rule corresponding to the object to be recognized comprises:
in response to that the type of the object to be recognized is the calculation-based fill-in-the-blank questions or the knowledge-based fill-in-the-blank questions, the processing rule corresponding to the object to be recognized comprises adding a keyword for the target object to be recognized;
in response to that the type of the object to be recognized is the calculation-based multiple-choice questions or the knowledge-based multiple-choice questions, the processing rule corresponding to the object to be recognized comprises selecting an option that is the same as the processing result from the respective options in the object to be recognized;
in response to that the type of the object to be recognized is the calculation-based true-false questions, the processing rule corresponding to the object to be recognized comprises comparing the processing result with the judgment result in the object to be recognized;
in response to that the type of the object to be recognized is the knowledge-based true-false questions, the processing rule corresponding to the object to be recognized comprises selecting an option that is the same as the processing result from the respective options in the third intermediate object to be recognized.

### Claim 6 (depends on 5)

The object recognition processing method according to claim 5, wherein the step of processing the processing result according to the processing rule, so as to obtain the target result corresponding to the object to be recognized comprises:
in response to that the type of the object to be recognized is the calculation-based questions or the knowledge-based questions, outputting the processing result directly as the target result;
in response to that the type of the object to be recognized is the calculation-based fill-in-the-blank questions or the knowledge-based fill-in-the-blank questions, taking a part corresponding to the keyword in the processing result as the target result;
in response to that the type of the object to be recognized is the calculation-based multiple-choice questions or the knowledge-based multiple-choice questions, selecting an option that is the same as the processing result from the respective options in the object to be recognized and taking a sequence number of the option that is the same as the processing result as the target result;
in response to that the type of the object to be recognized is the calculation-based true-false questions, comparing the processing result with the judgment result in the object to be recognized to obtain a comparison result, and taking the comparison result as the target result, wherein the comparison result comprises true, false or to be determined;
in response to that the type of the object to be recognized is the knowledge-based true-false questions, selecting an option that is the same as the processing result from the respective options in the third intermediate object to be recognized, and taking the option that is the same as the processing result as the target result.

### Claim 7 (depends on 5)

The object recognition processing method according to claim 5, wherein when the type of the object to be recognized is the calculation-based fill-in-the-blank questions, the keyword is “fill-in-the-blanks”,
when the type of the object to be recognized is the knowledge-based fill-in-the-blank questions, the keyword is “fill in text”.

### Claim 8 (depends on 1)

The object recognition processing method according to claim 1, wherein the step of obtaining the object to be recognized comprises:
obtaining an input image, wherein the input image comprises the object to be recognized;
processing the input image by means of a region recognition model, so as to obtain a region to be recognized comprising the object to be recognized;
processing the region to be recognized through a character recognition model, so as to obtain the object to be recognized.

### Claim 9 (depends on 8)

The object recognition processing method according to claim 8, wherein the input image is an image of a paper quiz, and the object to be recognized is a test question on the paper quiz.

### Claim 10 (depends on 1)

The object recognition processing method according to claim 1, wherein the transformer learning model is a model based on a neural network.

### Claim 11

A processing apparatus, comprising:
an acquisition module, configured to acquire an object to be recognized;
a type recognition module, configured to recognize a type of the object to be recognized based on a type recognition model;
a rule determining module, configured to determine a processing rule corresponding to the object to be recognized according to the type of the object to be recognized;
a transformer module, configured to recognize the object to be recognized through a transformer learning model according to the processing rule, so as to obtain a target result corresponding to the object to be recognized,
wherein when the transformer module executes an operation of recognizing the object to be recognized by means of the transformer learning model according to the processing rule to obtain the target result corresponding to the object to be recognized, the operation of recognizing comprises the following steps:
in response to that the type of the object to be recognized is a basic type, taking the object to be recognized as a target object to be recognized according to the processing rule, in response to that the type of the object to be recognized is a non-basic type, transforming the object to be recognized by means of the transformer learning model according to the processing rule, so as to transform the object to be recognized into the target object to be recognized;
recognizing the target object to be recognized by means of the transformer learning model, so as to obtain the target result corresponding to the object to be recognized,
wherein a type of the target object to be recognized is the basic type.

### Claim 12 (depends on 11)

The processing apparatus according to claim 11, wherein when the transformer module executes the operation of recognizing the object to be recognized by means of the transformer learning model according to the processing rule to obtain the target result corresponding to the object to be recognized, the operation of recognizing comprises the following steps:
performing a recognition processing on the target object to be recognized by means of the transformer learning model, so as to obtain a processing result corresponding to the target object to be recognized;
processing the processing result according to the processing rule, so as to obtain the target result corresponding to the object to be recognized.

### Claim 13 (depends on 1)

An electronic device, comprising:
a memory configured to store computer-readable instructions in a non-transitory manner;
a processor configured to execute the computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the object recognition processing method according to claim 1 is implemented.

### Claim 14 (depends on 1)

A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-readable instructions, when the computer-readable instructions are executed by a processor, the object recognition processing method according to claim 1 is implemented.
