# US 2017/0255861 A1 · Union processing of sequences of patterns

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0255861 A1
- Document ID: US20170255861A1
- Status: Published application
- Kind: A1
- Application number: 15/060,119
- Filed: 2016-03-03
- Published: 2017-09-07
- Earliest claimed: 2016-03-03
- Assignees: Numenta, Inc.
- Applicants: Numenta, Inc.
- Inventors: Jeffrey C. Hawkins; Yuwei Cui
- Web page: https://patentgrep.com/patent/US20170255861A1

## CPC

- G06N 20/00: Machine learning
- G06N 3/045: Combinations of networks
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Embodiments relate to a processing node of a hierarchical temporal memory (HTM) system with a union processor that enables a more stable representation of sequences by unionizing or pooling patterns of a temporal sequence. The union processor biases the HTM system so a learned temporal sequence may be more quickly recognized. The union processor includes union elements that are associated with incoming spatial patterns or with cells that represent temporal relationships between the spatial patterns. A union element of the union processor may be activated if a persistence score of the union element satisfies a predetermined criterion. The persistence score of the detector is updated based on the activation states of the spatial patterns or cells associated with the detector. After activation, the union element remains active for a period longer than a time step for performing the spatial pooling.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving input data having a plurality of elements, activation of the elements in the input data changing over time;
performing spatial pooling on the input data by selecting, for each time step, elements in a series of vectors to be activated based on active elements of the input data mapped to the elements in the vectors, each of the vectors in sparse distributed representation; and
generating an output data having one or more elements activated based on the selected elements in each of the vectors, the activated elements in the output data remaining active for a duration longer than the time step.

### Claim 2 (depends on 1)

The method of claim 1, wherein each of the elements in the output data is mapped to a subset of elements in the vector, an element in the output data activated responsive to a predetermined number or portion of the mapped subset of elements being activated.

### Claim 3 (depends on 1)

The method of claim 1, wherein each of the one or more elements in the output data remain active for a predetermined number of time steps.

### Claim 4 (depends on 1)

The method of claim 1, wherein each of the elements in the output data is mapped to a plurality of elements in the vector.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
updating a state of a sequence processor responsive to receiving each of the vectors for each time step; and
generating state information representing the updated state of the sequence processor and indicative of detection or prediction of temporal sequences in the input data, the one or more elements in the output data activated based on activation of elements of the state information mapped to the one or more elements in the output data.

### Claim 6 (depends on 5)

The method of claim 5, wherein each of the elements of the state information is further mapped to activation states of one or more cells in the sequence processor, the activation states of the one or more cells indicative of detection or prediction of the temporal sequences in the input data.

### Claim 7 (depends on 5)

The method of claim 5, wherein an element in the output data is activated responsive to a predetermined number or portion of cells mapped to the element being activated.

### Claim 8 (depends on 7)

The method of claim 7, wherein the element in the output data is activated further based on a ratio of the activated cells that contributed to correctly predicting the temporal sequences in the input data relative to the activated cells that did not contribute to correctly predicting the temporal sequences.

### Claim 9 (depends on 5)

The method of claim 5, wherein each of the elements of the vector is mapped to a plurality of cells in the sequence processor.

### Claim 10 (depends on 1)

The method of claim 1, wherein each of the one or more activated elements of the output data has a persistence score that meets a predetermined criterion, each of the one or more activated elements remaining active based on a rate of change in the persistence score after a selected element corresponding to each of the one or more activated elements is deactivated.

### Claim 11 (depends on 10)

The method of claim 10, wherein generating the output data comprises:
increasing a persistence score of each of the elements in the output data responsive to at least predetermined number or portion of a subset of elements in the vector mapped to each of the elements in the output or a subset of cells in a sequence processor being activated; and
decreasing the persistence score of each of the elements in the output data responsive to less than the predetermined number or portion of the mapped subset of elements in the vector or the cells in the sequence processor being activated;
wherein the predetermined criterion is that the persistent score of each of the one or more activated elements of the output data is above a threshold.

### Claim 12 (depends on 1)

The method of claim 1, wherein the input data is generated by spatial pooling and temporal processing of another input data.

### Claim 13

A computer-implemented method, comprising:
receiving input data having a plurality of elements, activation of the elements in the input data changing over time;
updating a state of a sequence processor responsive to receiving the input data for each time step; and
generating state information representing the updated state of the sequence processor and indicative of detection or prediction of temporal sequences in the input data, the one or more elements in the output data activated based on activation of elements of the state information mapped to the one or more elements in the output data; and generating an output data having one or more elements activated based on the activation of elements of the state information mapped to the one or more elements in the output data, the activated elements in the output data remaining active for a duration longer than the time step.

### Claim 14 (depends on 13)

The method of claim 13, wherein each of the elements in the output data is mapped to a subset of elements in the state information, an element in the output data activated responsive to a predetermined number or portion of the mapped subset of elements being activated.

### Claim 15 (depends on 13)

The method of claim 13, wherein each of the one or more elements in the output data remain active for a predetermined number of time steps.

### Claim 16 (depends on 13)

The method of claim 13, wherein each of the elements in the output data is mapped to a plurality of elements in the state element.

### Claim 17 (depends on 13)

The method of claim 13, wherein each of the elements of the state information is further mapped to activation states of one or more cells in the sequence processor, the activation states of the one or more cells indicative of detection or prediction of the temporal sequences in the input data.

### Claim 18 (depends on 17)

The method of claim 17, wherein an element in the output data is activated responsive to a predetermined number or portion of cells mapped to the element being activated.

### Claim 19 (depends on 18)

The method of claim 18, wherein the element in the output data is activated further based on a ratio of the activated cells that contributed to correctly predicting the temporal sequences in the input data relative to the activated cells that did not contribute to correctly predicting the temporal sequences.

### Claim 20 (depends on 13)

The method of claim 13, wherein each of the one or more activated elements of the output data has a persistence score that meets a predetermined criterion, each of the one or more activated elements remaining active based on a rate of change in the persistence score after a selected element corresponding to each of the one or more activated elements is deactivated.

### Claim 21 (depends on 20)

The method of claim 20, wherein generating the output data comprises:
increasing a persistence score of each of the elements in the output data responsive to at least predetermined number or portion of a subset of elements in the vector mapped to each of the elements in the output or a subset of cells in a sequence processor being activated; and
decreasing the persistence score of each of the elements in the output data responsive to less than the predetermined number or portion of the mapped subset of elements in the vector or the cells in the sequence processor being activated;
wherein the predetermined criterion is that the persistent score of each of the one or more activated elements of the output data is above a threshold.

### Claim 22

A computing device, comprising:
a processor; and
a memory operably coupled to the processor, the memory comprising:
a spatial pooler configured to:
receive input data having a plurality of elements, activation of the elements in the input data changing over time, and
perform spatial pooling on the input data by selecting, for each time step, elements in a series of vectors to be activated based on active elements of the input data mapped to the elements in the vectors, each of the vectors in sparse distributed representation; and
a union processor configured to generate an output data having one or more elements activated based on the selected elements in each of the vectors, the activated elements in the output data remaining active for a duration longer than the time step.

### Claim 23 (depends on 22)

The computing device of claim 22, wherein each of the elements in the output data is mapped to a subset of elements in the vector, an element in the output data activated responsive to a predetermined number or portion of the mapped subset of elements being activated.

### Claim 24 (depends on 22)

The computing device of claim 22, wherein each of the one or more elements in the output data remain active for a predetermined number of time steps.

### Claim 25 (depends on 22)

The computing device of claim 22, wherein each of the elements in the output data is mapped to a plurality of elements in the vector.

### Claim 26 (depends on 22)

The computing device of claim 22, wherein the memory further comprises a sequence processor configured to:
receive each of the vectors for each time step, and
generate state information representing indicating detection or prediction of temporal sequences in the input data, the one or more elements in the output data activated based on activation of elements of the state information mapped to the one or more elements in the output data.

### Claim 27 (depends on 26)

The computing device of claim 26, wherein each of the elements of the state information is further mapped to activation states of one or more cells in the sequence processor, the activation states of the one or more cells indicative of detection or prediction of the temporal sequences in the input data.

### Claim 28 (depends on 26)

The computing device of claim 26, wherein the sequence processor is further configured to activate an element in the output data responsive to a predetermined number or portion of cells mapped to the element being activated.

### Claim 29 (depends on 28)

The computing device of claim 28, wherein the sequence processor is further configured to activate the element in the output data further based on a ratio of the activated cells that contributed to correctly predicting the temporal sequences in the input data relative to the activated cells that did not contribute to correctly predicting the temporal sequences.

### Claim 30 (depends on 26)

The computing device of claim 26, wherein each of the elements of the vector is mapped to a plurality of cells in the sequence processor.

### Claim 31 (depends on 22)

The computing device of claim 22, wherein each of the one or more activated elements of the output data are elements has a persistence score that meets a predetermined criterion, each of the one or more activated elements remaining active based on a rate of change in the persistence score after a selected element corresponding to each of the one or more activated elements is deactivated.

### Claim 32 (depends on 31)

The computing device of claim 31, wherein the union processor is further configured to:
increase a persistence score of each of the elements in the output data responsive to at least predetermined number or portion of a subset of elements in the vector mapped to each of the elements in the output or a subset of cells in a sequence processor being activated; and
decrease the persistence score of each of the elements in the output data responsive to less than the predetermined number or portion of the mapped subset of elements in the vector or the cells in the sequence processor being activated;
wherein the predetermined criterion is that the persistent score of each of the one or more activated elements of the output data is above a threshold.

### Claim 33

A computing device, comprising:
a processor; and
a memory operably coupled to the processor, the memory comprising:
a sequence processor configured to:
receive input data having a plurality of elements, activation of the elements in the input data changing over time,
update a state of a sequence processor responsive to receiving the input data for each time step, and
generate state information representing the updated state of the sequence processor and indicative of detection or prediction of temporal sequences in the input data, the one or more elements in the output data activated based on activation of elements of the state information mapped to the one or more elements in the output data; and
a union processor configured to generate an output data having one or more elements activated based on the activation of elements of the state information mapped to the one or more elements in the output data, the activated elements in the output data remaining active for a duration longer than the time step.

### Claim 34

A non-transitory computer readable storage medium comprising instructions configured to be executed by a processor, the instructions when executed by the processor cause the processor to:
receive input data having a plurality of elements, activation of the elements in the input data changing over time;
perform spatial pooling on the input data by selecting, for each time step, elements in a series of vectors to be activated based on active elements of the input data mapped to the elements in the vectors, each of the vectors in sparse distributed representation; and
generate an output data having one or more elements activated based on the selected elements in each of the vectors, the activated elements in the output data remaining active for a duration longer than the time step.

### Claim 35

A non-transitory computer readable storage medium comprising instructions configured to be executed by a processor, the instructions when executed by the processor cause the processor to:
receive input data having a plurality of elements, activation of the elements in the input data changing over time;
update a state of a sequence processor responsive to receiving the input data for each time step; and
generate state information representing the updated state of the sequence processor and indicative of detection or prediction of temporal sequences in the input data, the one or more elements in the output data activated based on activation of elements of the state information mapped to the one or more elements in the output data; and
generate an output data having one or more elements activated based on the activation of elements of the state information mapped to the one or more elements in the output data, the activated elements in the output data remaining active for a duration longer than the time step.
