# US 2019/0318727 A1 · Sub-matrix input for neural network layers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0318727 A1
- Document ID: US20190318727A1
- Status: Published application
- Kind: A1
- Application number: 16/452,959
- Filed: 2019-06-26
- Published: 2019-10-17
- Earliest claimed: 2015-01-27
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Ignacio Lopez Moreno; Yu-hsin Joyce Chen
- Web page: https://patentgrep.com/patent/US20190318727A1

## CPC

- G10L 15/16: using artificial neural networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/09: Supervised learning
- G10L 17/18: Artificial neural networks; Connectionist approaches
- G10L 2015/088: {Word spotting}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes generating, by a speech recognition system, a matrix from a predetermined quantity of vectors that each represent input for a layer of a neural network, generating a plurality of sub-matrices from the matrix, using, for each of the sub-matrices, the respective sub-matrix as input to a node in the layer of the neural network to determine whether an utterance encoded in an audio signal comprises a keyword for which the neural network is trained.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A method performed by one or more computing devices, the method comprising:
obtaining, by the one or more computing devices, a set of values indicating acoustic characteristics of an utterance;
providing, by the one or more computing devices, the set of values as input to a neural network of a keyword detection system, wherein the neural network comprises a layer having nodes configured to respectively receive input derived from different proper subsets of the values provided as input to the neural network;
obtaining, by the one more computing devices, output that the neural network provided in response to receiving the set of values; and
determining, by the one or more computing devices, whether the utterance includes a keyword based on the output of the neural network.

### Claim 22 (depends on 21)

The method of claim 21, wherein the neural network is configured so that outputs of at least some of the nodes in the layer are based on application of different sets of weights to their respective proper subsets of the values provided as input to the neural network.

### Claim 23 (depends on 21)

The method of claim 21, wherein at least some of the nodes of the layer have different weights used to generate output from the nodes.

### Claim 24 (depends on 21)

The method of claim 21, wherein the different proper subsets comprise partially overlapping subsets.

### Claim 25 (depends on 21)

The method of claim 21, wherein the different proper subsets comprise non-overlapping subsets.

### Claim 26 (depends on 21)

The method of claim 21, wherein the neural network is configured such that at least some of the nodes in the layer are configured to each receive input derived from a respective subset of values that are localized.

### Claim 27 (depends on 21)

The method of claim 21, wherein the neural network is configured such that at least some of the nodes in the layer are configured to each receive input derived from a respective subset of values that are localized in frequency.

### Claim 28 (depends on 21)

The method of claim 21, wherein to receive input derived from a proper subset of the values provided as input to the neural network, the nodes of the layer are each configured to receive a corresponding proper subset of the values provided as input to the neural network.

### Claim 29 (depends on 21)

The method of claim 21, wherein determining whether the utterance includes a keyword based on the output of the neural network comprises determining whether the utterance comprises utterance of a key word or key phrase from among a set of predetermined key words or key phrases that are each designated as a signal that a mobile device should activate.

### Claim 30 (depends on 21)

The method of claim 21, wherein determining whether the utterance includes a keyword based on the output of the neural network comprises determining, based on the output of the neural network, whether an utterance contains a word.

### Claim 31 (depends on 21)

The method of claim 21, wherein determining whether the utterance includes a keyword based on the output of the neural network comprises determining, based on the output of the neural network, whether an utterance a particular word or part of a phrase for which the keyword detection system has been trained to identify.

### Claim 32 (depends on 21)

The method of claim 21, wherein the neural network is configured to provide output comprising one or more vectors of posterior probability scores.

### Claim 33 (depends on 21)

The method of claim 21, wherein the set of values comprises audio features derived from audio data for the utterance.

### Claim 34 (depends on 21)

The method of claim 21, wherein the layer is a first hidden layer of the neural network.

### Claim 35 (depends on 21)

The method of claim 21, wherein the layer is a second layer of the neural network, and wherein the neural network comprises an input layer configured to receive the set of values, wherein the nodes of the second layer are configured to respectively receive input derived from different proper subsets of the values provided to the input layer.

### Claim 36

A device comprising:
one or more processing devices and one or more data storage devices, one or more processing devices and the one or more data storage devices being configured to implement a keyword detection function by causing the device to perform operations comprising:
obtaining a set of values indicating acoustic characteristics of an utterance;
providing the set of values as input to a neural network of a keyword detection system, wherein the neural network comprises a layer having nodes configured to respectively receive input derived from different proper subsets of the values provided as input to the neural network;
obtaining output that the neural network provided in response to receiving the set of values; and
determining whether the utterance includes a keyword based on the output of the neural network.

### Claim 37 (depends on 36)

The device of claim 36, wherein the neural network is configured so that outputs of at least some of the nodes in the layer are based on application of different sets of weights to their respective proper subsets of the values provided as input to the neural network.

### Claim 38 (depends on 36)

The device of claim 36, wherein at least some of the nodes of the layer have different weights used to generate output from the nodes.

### Claim 39 (depends on 36)

The device of claim 36, wherein the different proper subsets comprise non-overlapping subsets.

### Claim 40

One or more non-transitory data storage devices storing instructions that, when executed by one or more processing devices, cause the one or more processing devices to perform operations comprising:
obtaining, by the one or more processing devices, a set of values indicating acoustic characteristics of an utterance;
providing, by the one or more processing devices, the set of values as input to a neural network of a keyword detection system, wherein the neural network comprises a layer having nodes configured to respectively receive input derived from different proper subsets of the values provided as input to the neural network;
obtaining, by the one more processing devices, output that the neural network provided in response to receiving the set of values; and
determining, by the one or more processing devices, whether the utterance includes a keyword based on the output of the neural network.
