# US 12,217,003 B2 · Apparatus and method for processing natural language

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,003 B2
- Document ID: US12217003B2
- Status: Granted
- Kind: B2
- Application number: 17/863,742
- Filed: 2022-07-13
- Published: 2023-03-16
- Granted: 2025-02-04
- Earliest claimed: 2021-09-08
- Assignees: Samsung Sds Co., Ltd.
- Applicants: Samsung Sds Co., Ltd.
- Inventors: Bong-Kyu Hwang; Ju-Dong Kim; Jae-Woong Yun; Hyun-Jae Lee; Hyun-Jin Choi; Seong-Ho Joe; Young-June Gwon
- Web page: https://patentgrep.com/patent/US12217003B2

## CPC

- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06F 16/35: Clustering; Classification
- G06F 40/117: Tagging; Marking up (details of markup languages G06F40/143); Designating a block; Setting of attributes (style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154)
- G06F 40/143: Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
- G06F 40/205: Parsing
- G06F 40/221: Parsing markup language streams (streaming G06F40/149)
- G06F 40/30: Semantic analysis
- G06F 40/279: Recognition of textual entities
- G06F 40/295: Named entity recognition
- G10L 15/18: using natural language modelling
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/183: using context dependencies, e.g. language models
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

An apparatus for processing natural language according to an embodiment includes a collection module that collects documents having tags, a parsing module that extracts text from the collected documents and extracts tag-related information on the tag surrounding each extracted text, and a preprocessing module that generates tokens of a preset unit by tokenizing each extracted text, generates token position information for each token in full text of the document, and sets the token and the token position information as training data in matching with the tag-related information.

## Claims

### Claim 1

An apparatus for processing natural language comprising:
at least one memory configured to store program code; and
at least one processor configured to read the program code and operate as instructed by the program code to perform:
collecting documents having tags;
extracting text from the collected documents and extracting tag-related information on a tag surrounding each extracted text; and
generating tokens of a preset unit by tokenizing each extracted text, generating token position information for each token in full text of a document, and setting the token and the token position information as training data in matching with the tag-related information,
wherein the tag-related information includes structural position information of a tag in which each text is positioned,
wherein the structural position information of the tag includes depth information and relative position information of a corresponding tag,
wherein the depth information is information indicating a depth level of the corresponding tag,
wherein the relative position information includes a relative position information value that is assigned to tags having the same depth level to sequentially increase or decrease according to a relative position or order between the tags having the same depth level, and
wherein relative position information values of two adjacent tags having different depth levels do not represent a relative position or order between the two adjacent tags.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the tag-related information further includes tag attribute information of the corresponding tag.

### Claim 3 (depends on 2)

The apparatus of claim 2,
wherein, in the extracting the tag-related information, the at least one processor is further configured to operate as instructed by the program code to assign the depth level to the corresponding tag according to whether or not a higher-level tag or a lower-level tag for the corresponding tag exists.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein, in the setting the token and the token position information, the at least one processor is further configured to operate as instructed by the program code to set the token and the token position information as the training data in matching with the tag attribute information and the relative position information.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein the at least one processor is further configured to operate as instructed by the program code to:
train a natural language understanding (NLU) model using the training data.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the at least one processor is further configured to operate as instructed by the program code to train the NLU model by assigning a weight to each tag among the training data.

### Claim 7

A method for processing natural language performed by a computing device that includes one or more processors and a memory for storing one or more programs executed by the one or more processors, the method comprising:
collecting documents having tags;
extracting text from the collected documents and extracting tag-related information on a tag surrounding each extracted text;
generating tokens of a preset unit by tokenizing each extracted text;
generating token position information for each token in full text of a document; and
setting the token and the token position information as training data by matching in matching with the tag-related information,
wherein the tag-related information includes structural position information of a tag in which each text is positioned,
wherein the structural position information of the tag includes depth information and relative position information of a corresponding tag,
wherein the depth information is information indicating a depth level of the corresponding tag,
wherein the relative position information includes a relative position information value that is assigned to tags having the same depth level to sequentially increase or decrease according to a relative position or order between the tags having the same depth level, and
wherein relative position information values of two adjacent tags having different depth levels do not represent a relative position or order between the two adjacent tags.

### Claim 8 (depends on 7)

The method of claim 7, wherein the tag-related information further includes tag attribute information of the corresponding tag.

### Claim 9 (depends on 8)

The method of claim 8, wherein
the extracting of the tag-related information includes assigning the depth level according to whether or not a higher-level tag or a lower-level tag for the corresponding tag exists.

### Claim 10 (depends on 8)

The method of claim 8, wherein in the setting of the token and the token position information as the training data, the token and the token position information are set as the training data in matching with the tag attribute information and the relative position information.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
training a natural language understanding (NLU) model by inputting the training data into the NLU model.

### Claim 12 (depends on 11)

The method of claim 11, wherein in the training of the NLU model, the NLU model is trained by assigning a weight to each tag among the training data.