# US 2022/0027608 A1 · Apparatus for detecting contextually-anomalous sentence in document, method therefor, and computer-readable recording medium having program for performing same method recorded thereon

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0027608 A1
- Document ID: US20220027608A1
- Status: Published application
- Kind: A1
- Application number: 17/413,825
- Filed: 2019-11-14
- Published: 2022-01-27
- Earliest claimed: 2018-12-14
- Assignees: ESTsoft Corp.
- Applicants: Estsoft Corp.
- Inventors: Hyeong Jin BYEON; Min Gwan SEO; Hae Bin SHIN
- Web page: https://patentgrep.com/patent/US20220027608A1

## CPC

- G06V 30/40: Document-oriented image-based pattern recognition
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06F 40/12: Use of codes for handling textual entities
- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06F 40/30: Semantic analysis
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06V 10/768: {using context analysis, e.g. recognition aided by known co-occurring patterns}
- G06V 10/82: using neural networks
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 30/1983: {Syntactic or structural pattern recognition, e.g. symbolic string recognition}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Disclosed are an apparatus and a method for detecting whether an anomalous sentence having a context different from that of other sentences exists in a document. The apparatus for detecting a contextually-anomalous sentence in a document according to the present invention includes: a sentence encoder for encoding individual sentences constituting document data by means of a predetermined rule (function) to generate encoding vectors; a context embedder neural network for converting the generated encoding vector into embedding vectors corresponding thereto; and a context anomaly detector neural network for detecting whether an anomalous sentence exists in the converted document data.

## Claims

### Claim 1

An apparatus for detecting a contextually-anomalous sentence in a document, the apparatus comprising:
a sentence encoder configured to, when document data is input, encode each of a plurality of sentences included in the document data to generate an encoding vector sequence including a plurality of encoding vectors;
a context embedder neural network configured to convert the generated encoding vectors into a plurality of context embedding vectors corresponding to each of the plurality of encoding vectors included in the encoding vector sequence to generate an embedding vector sequence;
a context anomaly detector neural network configured to calculate a result value indicating whether a contextually-anomalous sentence exists in the document data from the embedding vector sequence;
a detector learning unit configured to, if the document data is learning data, cause the context anomaly detector neural network to perform learning so that a difference between the result value calculated by the context anomaly detector neural network and an expected value indicating whether an anomalous sentence exists in the learning data falls within a predetermined range; and
a detection unit configured to, if the document data is suspected data, determine whether the anomalous sentence exists in the suspected data based on the result value calculated by the context anomaly detector neural network.

### Claim 2 (depends on 1)

The apparatus according to claim 1, further comprising:
a sentence sampling module configured to generate context learning data including pairs of encoding vectors corresponding to two or more sentences selected from the encoding vector sequence generated by the sentence encoder and a reference value indicating whether the contexts between the two or more sentences match with each other;
a distance learning neural network configured to, when generating the pairs of encoding vectors included in the context learning data as pairs of embedding vectors corresponding thereto by the context embedder neural network, calculate a distance value between the pairs of embedding vectors; and
an embedder learning unit configured to cause the context embedder neural network to perform learning so that a difference between the distance value between the pairs of embedding vectors calculated by the distance learning neural network and the reference value included in the context learning data falls within a predetermined range.

### Claim 3 (depends on 2)

The apparatus according to claim 2, wherein the reference value indicating whether the contexts between the two or more sentences match with each other is set based on whether the two or more sentences are arranged adjacent to each other in the document data.

### Claim 4 (depends on 2)

The apparatus according to claim 2, wherein the reference value indicating whether the contexts between the two or more sentences match with each other is set based on whether the two or more sentences belong to the same paragraph in the document data.

### Claim 5 (depends on 2)

The apparatus according to claim 2, wherein the reference value indicating whether the contexts between the two or more sentences match with each other is set based on whether the two or more sentences belong to the same category.

### Claim 6

A method for detecting a contextually-anomalous sentence in a document, the method comprising:
a first step of, when document data is input, encoding, by a sentence encoder, each of a plurality of sentences included in the document data to generate an encoding vector sequence including a plurality of encoding vectors;
a second step of converting, by a context embedder neural network, the generated encoding vectors into a plurality of context embedding vectors corresponding to each of the plurality of encoding vectors included in the encoding vector sequence to generate an embedding vector sequence;
a third step of calculating, by a context anomaly detector neural network, a result value indicating whether a contextually-anomalous sentence exists in the document data from the embedding vector sequence; and
a fourth step of, if the document data is suspected data, determining, by a detection unit, whether the anomalous sentence exists in the suspected data based on the result value calculated by the context anomaly detector neural network.

### Claim 7 (depends on 6)

The method according to claim 6, further comprising: before the fourth step, if the document data is learning data, causing, by a detector learning unit, the context anomaly detector neural network to perform learning so that a difference between the result value calculated by the context anomaly detector neural network and an expected value indicating whether an anomalous sentence exists in the learning data falls within a predetermined range.

### Claim 8 (depends on 6)

The method according to claim 6, further comprising: before the second step,
generating, by a sentence sampling module, context learning data including pairs of encoding vectors corresponding to two or more sentences selected from the encoding vector sequence generated by the sentence encoder and a reference value indicating whether the contexts between the two or more sentences match with each other;
when generating each of the pairs of encoding vectors included in the context learning data as pairs of embedding vectors corresponding thereto by the context embedder neural network,
calculating, by a distance learning neural network, a distance value between the pairs of embedding vectors; and
causing, by an embedder learning unit, the context embedder neural network to perform learning so that a difference between the distance value between the pairs of embedding vectors calculated by the distance learning neural network and the reference value included in the context learning data falls within a predetermined range.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the reference value indicating whether the contexts between the two or more sentences match with each other is set based on whether the two or more sentences are arranged adjacent to each other in the document data.

### Claim 10 (depends on 8)

The method according to claim 8, wherein the reference value indicating whether the contexts between the two or more sentences match with each other is set based on whether the two or more sentences belong to the same paragraph in the document data.

### Claim 11 (depends on 8)

The method according to claim 8, wherein the reference value indicating whether the contexts between the two or more sentences match with each other is set based on whether the two or more sentences belong to the same category.

### Claim 12 (depends on 6)

A computer-readable recording medium in which a program for performing the method for detecting a contextually-anomalous sentence in a document according to claim 6 is recorded.
