# US 2021/0279569 A1 · Method and apparatus with vector conversion data processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0279569 A1 - Document ID: US20210279569A1 - Status: Published application - Kind: A1 - Application number: 17/019,688 - Filed: 2020-09-14 - Published: 2021-09-09 - Earliest claimed: 2020-03-09 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Minkyu KIM - Web page: https://patentgrep.com/patent/US20210279569A1 ## CPC - G06F 16/901: Indexing; Data structures therefor; Storage structures (for retrieval from the web G06F16/951) - G06N 3/08: Learning methods - G06F 16/2237: {Vectors, bitmaps or matrices} - G06F 16/258: {Data format conversion from or to a database} - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/045: Combinations of networks - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A data processing method includes: generating an input vector by embedding input data; converting a dimension of the input vector based on a pattern of the input vector; and performing attention on the dimension-converted input vector. ## Claims ### Claim 1 A data processing method, comprising: generating an input vector by embedding input data; converting a dimension of the input vector based on a pattern of the input vector; and performing attention on the dimension-converted input vector. ### Claim 2 (depends on 1) The method of claim 1, wherein the generating comprises: converting the input data into a dense vector; and generating the input vector by performing position embedding on the dense vector based on the position of the input data with respect to an entire input. ### Claim 3 (depends on 1) The method of claim 1, wherein the converting comprises: determining an embedding index with respect to the input vector based on the pattern of the input vector; and converting the dimension of the input vector based on the embedding index. ### Claim 4 (depends on 3) The method of claim 3, wherein the determining comprises determining, as the embedding index, an index corresponding to a boundary between a component to be used in the performing of the attention and a component not to be used in the performing of the attention, among components of the input vector. ### Claim 5 (depends on 3) The method of claim 3, wherein the component not to be used in the performing of the attention includes a value of “0”. ### Claim 6 (depends on 3) The method of claim 3, wherein the converting of the dimension of the input vector based on the embedding index comprises reducing the dimension of the input vector by removing a component corresponding to an index greater than the embedding index from the input vector. ### Claim 7 (depends on 4) The method of claim 4, wherein the input vector comprises a plurality of input vectors, and the embedding index is an index having a max position among indices corresponding to boundaries between components of the input vectors to be used in the performing of the attention and components of the input vectors not to be used in the performing of the attention. ### Claim 8 (depends on 1) The method of claim 1, further comprising: restoring the dimension of the input vector on which the attention is performed. ### Claim 9 (depends on 8) The method of claim 8, wherein the restoring comprises increasing the dimension of the input vector on which the attention is performed to the same dimension as the input vector based on an embedding index determined based on the pattern of the input vector. ### Claim 10 (depends on 9) The method of claim 9, wherein the increasing comprises performing zero padding on a component corresponding to an index greater than or equal to the embedding index with respect to the input vector on which the attention is performed. ### Claim 11 (depends on 1) The method of claim 1, further comprising: generating an output sentence as a translation of an input sentence, based on the input vector on which the attention is performed, wherein the input data corresponds to the input sentence. ### Claim 12 (depends on 1) A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the method of claim 1. ### Claim 13 A data processing apparatus, comprising: a processor configured to: generate an input vector by embedding input data, convert a dimension of the input vector based on a pattern of the input vector, and perform attention on the dimension-converted input vector. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein, for the generating, the processor is configured to: convert the input data into a dense vector, and generate the input vector by performing position embedding on the dense vector based on the position of the input data with respect to an entire input. ### Claim 15 (depends on 13) The apparatus of claim 13, wherein, for the converting, the processor is configured to: determine an embedding index with respect to the input vector based on the pattern of the input vector, and convert the dimension of the input vector based on the embedding index. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein, for the determining, the processor is configured to determine, as the embedding index, an index corresponding to a boundary between a component to be used in the performing of the attention and a component not to be used in the performing of the attention, among components of the input vector. ### Claim 17 (depends on 15) The apparatus of claim 15, wherein the component not to be used in the performing of the attention includes a value of “0”. ### Claim 18 (depends on 15) The apparatus of claim 15, wherein, for the converting, the processor is configured to reduce the dimension of the input vector by removing a component corresponding to an index greater than or equal to the embedding index from the input vector. ### Claim 19 (depends on 13) The apparatus of claim 13, wherein the processor is configured to restore the dimension of the input vector on which the attention is performed. ### Claim 20 (depends on 19) The apparatus of claim 19, wherein, for the restoring, the processor is configured to increase the dimension of the input vector on which the attention is performed to the same dimension as the input vector based on an embedding index determined based on the pattern of the input vector. ### Claim 21 (depends on 20) The apparatus of claim 20, wherein, for the increasing, the processor is configured to perform zero padding on a component corresponding to an index greater than the embedding index with respect to the input vector on which the attention is performed. ### Claim 22 (depends on 13) The apparatus of claim 13 further comprising a memory storing instructions that, when executed by the processor, configure the processor to perform the generating of the input vector, the converting of the dimension of the input vector, and the performing of the attention on the dimension-converted input vector.