# US 10,311,149 B1 · Natural language translation device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,149 B1
- Document ID: US10311149B1
- Status: Granted
- Kind: B1
- Application number: 16/058,801
- Filed: 2018-08-08
- Granted: 2019-06-04
- Earliest claimed: 2018-08-08
- Assignees: Gyrfalcon Technology Inc.
- Applicants: Gyrfalcon Technology Inc.
- Inventors: Lin Yang; Patrick Z. Dong; Catherine Chi; Charles Jin Young; Jason Z Dong; Baohua Sun
- Web page: https://patentgrep.com/patent/US10311149B1

## CPC

- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G06F 40/126: Character encoding
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06F 40/44: Statistical methods, e.g. probability models
- G06F 40/51: Translation evaluation
- G06F 40/53: Processing of non-Latin text (kana-to-kanji conversion G06F40/129; vowelisation G06F40/232)
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Natural language translation device contains a bus, an input interface connecting to the bus for receiving a source sentence in a first natural language to be translated to a target sentence in second natural language one word at a time in sequential order. A two-dimensional (2-D) symbol containing a super-character characterizing the i-th word of the target sentence based on the received source sentence is formed in accordance with a set of 2-D symbol creation rules. The i-th word of the target sentence is obtained by classifying the 2-D symbol via a deep learning model that contains multiple ordered convolution layers in a Cellular Neural Networks or Cellular Nonlinear Networks (CNN) based integrated circuit.

## Claims

### Claim 1

A natural language translation device comprising:
a bus;
an input interface operatively connecting to the bus for receiving a source sentence in a first natural language to be translated to a target sentence in second natural language one word at a time in sequential order;
a processing unit operatively connecting to the bus for forming a two-dimensional (2-D) symbol in accordance with a set of 2-D symbol creation rules using a 2-D symbol creation module installed thereon, the 2-D symbol containing a super-character having a characteristic suggesting the i-th word of the target sentence based on the received source sentence; and
a Cellular Neural Networks or Cellular Nonlinear Networks (CNN) based integrated circuit operatively connecting to the bus for classifying the 2-D symbol via a deep learning model that contains a plurality of ordered convolutional layers.

### Claim 2 (depends on 1)

The device of claim 1, further comprises a display unit operatively connecting to the bus for displaying already-translated portion of the target sentence.

### Claim 3 (depends on 1)

The device of claim 1, wherein the 2-D symbol comprises a matrix of N×N pixels of K-bit data, and the matrix is divided into M×M sub-matrices with each of the sub-matrices containing (N/M)×(N/M) pixels, said each of the sub-matrices representing one ideogram, where K, N and M are positive integers or whole numbers, and N is a multiple of M.

### Claim 4 (depends on 1)

The device of claim 1, wherein the set of 2-D symbol creation rules comprises:
dividing the 2-D symbol into first and second parts;
converting the source sentence to a first set of ideograms;
converting the first word to the (i−1)-th word of the target sentence to a second set of ideograms; and
forming the 2-D symbol by including the first set of ideograms in the first part and including the second set of ideograms in the second part.

### Claim 5 (depends on 1)

The device of claim 1, wherein the set of 2-D symbol creation rules comprises:
converting the source sentence to a set of ideograms;
dividing the 2-D symbol into a predetermined number of sub-symbols with each sub-symbol for one ideogram; and
forming the 2-D symbol by including relevant portion of the set of ideograms in a scheme such that the ideogram corresponding to the i-th word is located in a predefined location, by filling each unoccupied sub-symbol with blank space if needed.

### Claim 6 (depends on 1)

The device of claim 1, wherein the set of 2-D symbol creation rules comprises:
dividing the 2-D symbol into a predetermined number of sub-symbols;
creating the predetermined number of groups of consecutive words from the source sentence with each group encompassing the i-th word of the source sentence and said each group corresponding to one of the sub-symbols; and
forming the 2-D symbol by including ideograms converted from respective groups of consecutive words for all sub-symbols.

### Claim 7 (depends on 1)

The device of claim 1, wherein the CNN based integrated circuit comprises a plurality of CNN processing engines operatively coupled to at least one input/output data bus, the plurality of CNN processing engines being connected in a loop with a clock-skew circuit, each CNN processing engine comprising:
a CNN processing block configured for simultaneously performing convolutional operations using the 2-D symbol and pre-trained filter coefficients of the plurality of ordered convolutional layers;
a first set of memory buffers operatively coupling to the CNN processing block for storing the 2-D symbol; and
a second set of memory buffers operatively coupling to the CNN processing block for storing the pre-trained filter coefficients.

### Claim 8 (depends on 7)

The device of claim 7, wherein the pre-trained filter coefficients comprises multiple bi-valued 3×3 filter kernels.

### Claim 9 (depends on 8)

The device of claim 8, wherein each of the bi-valued 3×3 filter kernels contains fixed point number with positive or negative of same numerical value.

### Claim 10 (depends on 7)

The device of claim 7, wherein the CNN based integrated circuit further performs pooling operations and activation operations.

### Claim 11 (depends on 1)

The device of claim 1, further comprises a memory operatively connected to the bus for providing data storage for the processing unit.

### Claim 12

A method of translating natural language using a Cellular Neural Networks or Cellular Nonlinear Networks (CNN) based integrated circuit, the method comprising:
receiving, in a natural language translation device having a two-dimensional (2-D) symbol creation module installed thereon, a source sentence in a first natural language to be translated to a target sentence in second natural language one word at a time in sequential order;
forming, with the 2-D symbol creation module, a multi-layer two-dimensional (2-D) symbol in accordance with a set of 2-D symbol creation rules, the 2-D symbol being a matrix of N×N pixels of K-bit data that contains a super-character, the matrix being divided into M×M sub-matrices with each of the sub-matrices containing (N/M)×(N/M) pixels, said each of the sub-matrices representing one ideogram, the super-character having a characteristic suggesting the i-th word of the target sentence based on the received source sentence, where K, N and M are positive integers or whole numbers, and N is a multiple of M; and
obtaining, with the 2-D symbol creation module, the i-th word of the target sentence by classifying the 2-D symbol via a deep learning model having a plurality of ordered convolution layers in a Cellular Neural Networks or Cellular Nonlinear Networks (CNN) based integrated circuit.

### Claim 13 (depends on 12)

The method of claim 12, further comprises displaying already-translated portion of the target sentence in a display unit of the natural language translation device.

### Claim 14 (depends on 12)

The method of claim 12, wherein the set of 2-D symbol creation rules comprises:
dividing the 2-D symbol into first and second parts;
converting the source sentence to a first set of ideograms;
converting the first word to the (i−1)-th word of the target sentence to a second set of ideograms; and
forming the 2-D symbol by including the first set of ideograms in the first part and including the second set of ideograms in the second part.

### Claim 15 (depends on 12)

The method of claim 12, wherein the set of 2-D symbol creation rules comprises:
converting the source sentence to a set of ideograms;
dividing the 2-D symbol into a predetermined number of sub-symbols with each sub-symbol for one ideogram; and
forming the 2-D symbol by including relevant portion of the set of ideograms in a scheme such that the ideogram corresponding to the i-th word is located in a predefined location, by filling each unoccupied sub-symbol with blank space if needed.

### Claim 16 (depends on 12)

The method of claim 12, wherein the set of 2-D symbol creation rules comprises:
dividing the 2-D symbol into a predetermined number of sub-symbols;
creating the predetermined number of groups of consecutive words from the source sentence with each group encompassing the i-th word of the source sentence and said each group corresponding to one of the sub-symbols; and
forming the 2-D symbol by including ideograms converted from respective groups of consecutive words for all sub-symbols.

### Claim 17 (depends on 12)

The method of claim 12, wherein the CNN based integrated circuit comprises a plurality of CNN processing engines operatively coupled to at least one input/output data bus, the plurality of CNN processing engines being connected in a loop with a clock-skew circuit, each CNN processing engine comprising:
a CNN processing block configured for simultaneously performing convolutional operations using the 2-D symbol and pre-trained filter coefficients of the plurality of ordered convolutional layers;
a first set of memory buffers operatively coupling to the CNN processing block for storing the 2-D symbol; and
a second set of memory buffers operatively coupling to the CNN processing block for storing the pre-trained filter coefficients.

### Claim 18 (depends on 17)

The method of claim 17, wherein the pre-trained filter coefficients comprises multiple bi-valued 3×3 filter kernels.

### Claim 19 (depends on 18)

The method of claim 18, wherein each of the bi-valued 3×3 filter kernels contains fixed point number with positive or negative of same numerical value.

### Claim 20 (depends on 17)

The method of claim 17, wherein the CNN based integrated circuit further performs pooling operations and activation operations.