# US 12,217,154 B2 · Method and apparatus with neural network operation and keyword spotting > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,154 B2 - Document ID: US12217154B2 - Status: Granted - Kind: B2 - Application number: 17/229,273 - Filed: 2021-04-13 - Published: 2022-06-02 - Granted: 2025-02-04 - Earliest claimed: 2020-12-02 - Assignees: Samsung Electronics Co., Ltd.; Seoul National University R&Db Foundation - Applicants: Samsung Electronics Co., Ltd.; Seoul National University R&DB Foundation - Inventors: Xue Qian; Jin Hwan Park; Wonyong Sung - Web page: https://patentgrep.com/patent/US12217154B2 ## CPC - G06N 3/048: Activation functions - G10L 15/16: using artificial neural networks - G06F 5/01: for shifting, e.g. justifying, scaling, normalising {(digital stores in which the information is moved stepwise, e.g. shift-registers G11C19/00; digital stores in which the information circulates G11C21/00)} - G06F 17/153: {Multidimensional correlation or convolution} - G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} - G06F 40/279: Recognition of textual entities - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0495: Quantised networks; Sparse networks; Compressed networks - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G10L 2015/088: {Word spotting} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A neural network operation method includes: receiving an input vector sequence including a plurality of channels; performing a first convolution operation on a first input vector of the input vector sequence; and performing a second convolution operation on a second input vector of the input vector sequence that is adjacent to the first input vector in a channel direction. ## Claims ### Claim 1 A neural network operation method, comprising: receiving an input vector sequence including a plurality of channels; performing a first convolution operation on a first input vector of the input vector sequence; and performing a second convolution operation on a second input vector of the input vector sequence that is adjacent to and separate from the first input vector as determined by a preset distance in a channel direction. ### Claim 2 (depends on 1) The method of claim 1, wherein the performing of the first convolution operation comprises: performing a temporal convolution on the first input vector with a filter having a preset length on a time axis. ### Claim 3 (depends on 1) The method of claim 1, wherein the first convolution operation and the second convolution operation respectively include a one-dimensional (1D) convolution operation. ### Claim 4 (depends on 1) The method of claim 1, wherein the performing of the second convolution operation comprises: performing the second convolution operation on one or more second input vectors, including the second input vector, that are separate from the first input vector by a distance less than or equal to the preset distance in the channel direction. ### Claim 5 (depends on 4) The method of claim 4, wherein the preset distance has a value less than a dimension of the first input vector. ### Claim 6 (depends on 1) The method of claim 1, further comprising: performing a batch normalization operation based on a result of the second convolution operation; and performing a first activation operation based on a result of the batch normalization operation. ### Claim 7 (depends on 6) The method of claim 6, wherein the performing of the first activation operation comprises: performing a rectified linear unit (ReLU) operation based on the result of the batch normalization operation. ### Claim 8 (depends on 6) The method of claim 6, further comprising: adding the first input vector and a result of the first activation operation; and performing a second activation operation based on a result of the adding. ### Claim 9 (depends on 8) The method of claim 8, further comprising: performing a pooling operation based on a result of the second activation operation; and performing a fully-connected operation based on a result of the pooling operation. ### Claim 10 (depends on 1) A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the method of claim 1. ### Claim 11 A neural network operation apparatus, comprising: a receiver configured to receive an input vector sequence including a plurality of channels; and a processor configured to: perform a first convolution operation on a first input vector of the input vector sequence; and perform a second convolution operation on a second input vector of the input vector sequence that is adjacent to and separate from the first input vector as determined by a preset distance in a channel direction. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein, for the performing of the first convolution operation, the processor is configured to: perform a temporal convolution on the first input vector with a filter having a preset length on a time axis. ### Claim 13 (depends on 11) The apparatus of claim 11, wherein the first convolution operation and the second convolution operation respectively include a one-dimensional (1D) convolution operation. ### Claim 14 (depends on 11) The apparatus of claim 11, wherein, for the performing of the second convolution operation, the processor is configured to: perform the second convolution operation on one or more second input vectors, including the second input vector, that are separate from the first input vector by a distance less than or equal to the preset distance in the channel direction. ### Claim 15 (depends on 14) The apparatus of claim 14, wherein the preset distance has a value less than a dimension of the first input vector. ### Claim 16 (depends on 11) The apparatus of claim 11, wherein the processor is configured to: perform a batch normalization operation based on a result of the second convolution operation; and perform a first activation operation based on a result of the batch normalization operation. ### Claim 17 (depends on 16) The apparatus of claim 16, wherein the processor is configured to: perform a rectified linear unit (ReLU) operation based on the result of the batch normalization operation. ### Claim 18 (depends on 16) The apparatus of claim 16, wherein the processor is configured to: add the first input vector and a result of the first activation operation; and perform a second activation operation based on a result of the adding. ### Claim 19 (depends on 18) The apparatus of claim 18, wherein the processor is configured to: perform a pooling operation based on a result of the second activation operation; and perform a fully-connected operation based on a result of the pooling operation. ### Claim 20 A keyword spotting method comprising: receiving a natural language; performing a first convolution operation on a first input vector of an input vector sequence including a plurality of channels based on the received natural language; performing a second convolution operation on a second input vector of the input vector sequence that is adjacent to and separate from the first input vector as determined by a preset distance in a channel direction; and detecting a keyword included in the natural language based on a result of the second convolution operation. ### Claim 21 (depends on 20) The method of claim 20, wherein the performing of the first convolution operation comprises: generating the input vector sequence by extracting a feature from the natural language. ### Claim 22 A neural network operation method, comprising: performing a first convolution and a second convolution respectively on a first input vector and second input vector of an input vector sequence, wherein the first and second input vectors are separated from each other as determined by a preset distance in a channel direction; performing a first activation based on the second convolution; adding the first input vector and a result of the first activation; performing a second activation based on the adding; and performing a fully-connected operation based on the second activation. ### Claim 23 (depends on 22) The method of claim 22, wherein the performing of the first activation comprises: performing a batch normalization based on a result of the second convolution; and performing the first activation based on a result of the batch normalization. ### Claim 24 (depends on 22) The method of claim 22, wherein the performing of the fully-connected operation comprises: performing a pooling based on a result of the second activation; and performing the fully-connected operation based on the result of the pooling.