# US 2023/0274662 A1 · Data processing method and apparatus, computer device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274662 A1
- Document ID: US20230274662A1
- Status: Published application
- Kind: A1
- Application number: 18/312,750
- Filed: 2023-05-05
- Published: 2023-08-31
- Earliest claimed: 2021-07-20
- Assignees: Beijing Sogou Technology Development Co., Ltd.
- Applicants: Beijing Sogou Technology Development Co., Ltd.
- Inventors: Wenjun DUAN; Jingrong BI; Mei LI; Fanbo MENG; Yun PENG; Kai LIU
- Web page: https://patentgrep.com/patent/US20230274662A1

## CPC

- G09B 21/009: {Teaching or communicating with deaf persons (G09B21/04 takes precedence)}
- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06V 30/153: {using recognition of characters or words}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

Provided herein are a data processing method and apparatus, a computer device, and a storage medium for sign language translation. Sign language is translated to text by: obtaining sign language action data; determining a sign language tagging sequence corresponding to the sign language action data by element analysis, and performing operation processing on the sign language action data based on the sign language tagging sequence. Text is translated to sign language by: performing word segmentation processing on a text sequence to obtain a natural word sequence; determining a basic sign language element and an element type corresponding to each natural word in the natural word sequence; sorting the basic sign language element and the element type to generate a sign language element sequence; determining a sign language tagging sequence; and performing sign language translation processing on the text sequence based on the sign language tagging sequence.

## Claims

### Claim 1

A data processing method, performed by at least one processor of a computer device and comprising:
obtaining sign language action data;
determining a sign language tagging sequence corresponding to the sign language action data by performing element analysis on the sign language action data, the element analysis being based on a pre-established sign language tagging system, the sign language tagging sequence comprising tagging information of basic sign language elements corresponding to the sign language action data; and
performing operation processing on the sign language action data based on the sign language tagging sequence.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising, prior to the performing of the element analysis:
establishing a sign language tagging system based on basic sign language elements and element types corresponding to each basic sign language element, the sign language tagging system comprising tagging information corresponding to the element types of each basic sign language element.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the basic sign language elements comprise at least one of a left or right arm feature, a one or both handshape feature, an orientation motion feature, a knuckle bending angle, a facial expression feature, or constraint information.

### Claim 4 (depends on 2)

The method according to claim 2, wherein the establishing of the sign language tagging system comprises:
performing disassembly and classification on sign language action data in a database, to obtain the basic sign language elements and the element types corresponding to each basic sign language element.

### Claim 5 (depends on 2)

The method according to claim 2, wherein the performing of the disassembly and of the classification comprises:
traversing the sign language action data in the database, performing action disassembly on the sign language action data, and determining a key part corresponding to each piece of the sign language action data and an action feature of the key part;
performing classification processing on the key part corresponding to each piece of the sign language action data in the database and the action feature of the key part, to obtain at least two class clusters, each class cluster corresponding to one basic sign language element; and
determining, based on an action feature of each class cluster, element types of a basic sign language element corresponding to the class cluster.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the action feature comprises at least one of rotation data, displacement data, a bending angle, a key feature, or an expression feature.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the determining of the sign language tagging sequence comprises:
performing the element analysis on the sign language action data, to determine a first basic sign language element corresponding to the sign language action data, and a first element type and a first timestamp of the first basic sign language element;
determining, based on the pre-established sign language tagging system, first tagging information of the first basic sign language element and second tagging information of the first element type; and
determining, based on the first timestamp, the first tagging information, and the second tagging information, the sign language tagging sequence corresponding to the sign language action data.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the performing of the operation processing comprises:
driving, based on the sign language tagging sequence, a pre-established three-dimensional character model to perform a sign language action corresponding to the sign language action data.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the performing of the operation processing comprises:
performing sign language translation processing on the sign language action data based on the sign language tagging sequence, to obtain a target text sequence corresponding to the sign language action data.

### Claim 10

A data processing method, performed by a computer device and comprising:
performing word segmentation processing on a to-be-translated text sequence, to obtain a natural word sequence corresponding to the text sequence;
determining a basic sign language element corresponding to each natural word in the natural word sequence and an element type corresponding to the basic sign language element;
sorting the basic sign language element and the element type, to generate a sign language element sequence conforming to a sign language grammar rule;
determining a sign language tagging sequence corresponding to the sign language element sequence based on a sign language tagging system; and
performing sign language translation processing on the text sequence based on the sign language tagging sequence, to obtain a sign language action corresponding to the text sequence.

### Claim 11

A data processing apparatus, comprising a memory and one or more processors, the memory storing computer-readable instructions, the computer-readable instructions, when executed by the one or more processors, causing the one or more processors to perform operations comprising:
obtaining sign language action data;
determining a sign language tagging sequence corresponding to the sign language action data by performing element analysis on the sign language action data, the element analysis being based on a pre-established sign language tagging system, the sign language tagging sequence comprising tagging information of basic sign language elements corresponding to the sign language action data; and
performing operation processing on the sign language action data based on the sign language tagging sequence.

### Claim 12 (depends on 11)

The apparatus according to claim 11, wherein the computer-readable instructions further cause the one or more processors to perform operations of:
establishing a sign language tagging system based on basic sign language elements and element types corresponding to each basic sign language element, the sign language tagging system comprising tagging information corresponding to the element types of each basic sign language element.

### Claim 13 (depends on 12)

The apparatus according to claim 12, wherein the basic sign language elements comprise at least one of a left or right arm feature, a one or both handshape feature, an orientation motion feature, a knuckle bending angle, a facial expression feature, or constraint information.

### Claim 14 (depends on 12)

The apparatus according to claim 12, wherein the computer-readable instructions further cause the one or more processors to perform disassembly and classification on sign language action data in a database, to obtain the basic sign language elements and the element types corresponding to each basic sign language element.

### Claim 15 (depends on 12)

The apparatus according to claim 12, wherein the computer-readable instructions cause the one or more processors to perform the disassembly and the classification by:
traversing the sign language action data in the database, performing action disassembly on the sign language action data, and determining a key part corresponding to each piece of the sign language action data and an action feature of the key part;
performing classification processing on the key part corresponding to each piece of the sign language action data in the database and the action feature of the key part, to obtain at least two class clusters, each class cluster corresponding to one basic sign language element; and
determining, based on an action feature of each class cluster, element types of a basic sign language element corresponding to the class cluster.

### Claim 16 (depends on 15)

The apparatus according to claim 15, wherein the action feature comprises at least one of rotation data, displacement data, a bending angle, a key feature, or an expression feature.

### Claim 17 (depends on 11)

The apparatus according to claim 11, wherein the computer-readable instructions cause the one or more processors to determine the sign language tagging sequence by:
performing the element analysis on the sign language action data, to determine a first basic sign language element corresponding to the sign language action data, and a first element type and a first timestamp of the first basic sign language element;
determining, based on the pre-established sign language tagging system, first tagging information of the first basic sign language element and second tagging information of the first element type; and
determining, based on the first timestamp, the first tagging information, and the second tagging information, the sign language tagging sequence corresponding to the sign language action data.

### Claim 18 (depends on 11)

The apparatus according to claim 11, wherein the computer-readable instructions cause the one or more processors to perform the operation processing by:
driving, based on the sign language tagging sequence, a pre-established three-dimensional character model to perform a sign language action corresponding to the sign language action data.

### Claim 19 (depends on 11)

The apparatus according to claim 11, wherein the computer-readable instructions cause the one or more processors to perform the operation processing by:
performing sign language translation processing on the sign language action data based on the sign language tagging sequence, to obtain a target text sequence corresponding to the sign language action data.
