# US 2019/0244112 A1 · Method and system for an intelligent artificial agent

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244112 A1
- Document ID: US20190244112A1
- Status: Published application
- Kind: A1
- Application number: 15/888,619
- Filed: 2018-02-05
- Published: 2019-08-08
- Earliest claimed: 2018-02-05
- Assignees: Cogitai, Inc.
- Applicants: Cogitai, Inc.
- Inventors: Michael BOWLING; Satinder BAVEJA; Peter WURMAN
- Web page: https://patentgrep.com/patent/US20190244112A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 5/02: Knowledge representation; Symbolic representation
- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods use deep, convolutional neural networks over exponentially long history windows to learn alphabets for context tree weighting (CTW) for prediction. Known issues of depth and breadth in conventional context tree weighting predictions are addressed by the systems and methods. To deal with depth, the history can be broken into time windows, permitting the ability to look exponentially far back while having less information the further one looks back. To deal with breadth, a deep neural network classifier can be used to learn to map arbitrary length histories to a small output symbol alphabet. The sequence of symbols produced by such a classifier over the history windows would then become the input sequence to CTW.

## Claims

### Claim 1

An artificial intelligence system, comprising: a computing device including at least one processor, one or more data storage devices, and a non-transitory data storage medium interfaced with the at least one processor, the non-transitory data storage medium containing instructions that, when executed cause the at least one processor to:
save observed sensory sequence information in a plurality of history windows, wherein a size of the plurality of history windows increase exponentially from a last observed time step;
apply a function to the observed sensory sequence information in each history window, wherein the function maps the observed sensory sequence information into a fixed set of discrete classes; and
apply a context tree weighting algorithm to an alphabet resulting from the fixed set of discrete classes for each of the plurality of history windows to predict a future discrete sequence.

### Claim 2 (depends on 1)

The artificial intelligence system of claim 1, wherein the instructions cause the at least one processor to choose at least one parameter of the exponentially increasing history window size as a hyperparameter to allow the system to trade off bias-variance.

### Claim 3 (depends on 1)

The artificial intelligence system of claim 1, wherein the function is a feature-wise maximum over time steps in one or more of the plurality of history windows.

### Claim 4 (depends on 3)

The artificial intelligence system of claim 3, wherein the observed sensory sequence information is a binary event.

### Claim 5 (depends on 1)

The artificial intelligence system of claim 1, wherein the instructions cause the at least one processor to use a deep neural network classifier to map arbitrary length histories to a minimal output symbol alphabet as an input sequence for the context tree weighting algorithm.

### Claim 6 (depends on 5)

The artificial intelligence system of claim 5, wherein a long short-term memory-based sequence to symbol method is used to map the arbitrary length histories to the minimal output symbol alphabet.

### Claim 7 (depends on 1)

The artificial intelligence system of claim 1, wherein the instructions cause the at least one processor to perform a temporal convolution in a deep neural network to map observed sensory sequence information from the plurality of history windows to symbols.

### Claim 8 (depends on 7)

The artificial intelligence system of claim 7, wherein the temporal convolution includes defining each of the plurality of history windows of events as a ($2{circumflex over ( )}k$)-by-n matrix, where $2{circumflex over ( )}k$ is a number of time steps in each of the plurality of history windows and n is a number of events at each of the time steps, and applying a convolution that is an l-by-n matrix, where l is a small number, wherein a set of the convolutions produces a new set of events.

### Claim 9 (depends on 8)

The artificial intelligence system of claim 8, wherein the convolution is applied to each of the plurality of history windows.

### Claim 10

An artificial intelligence system, comprising: a computing device including at least one processor, one or more data storage devices, and a non-transitory data storage medium interfaced with the at least one processor, the non-transitory data storage medium containing instructions that, when executed cause the at least one processor to:
save observed sensory sequence information in a plurality of history windows, wherein a size of the plurality of history windows increase exponentially from a last observed time step;
apply a function to the observed sensory sequence information in each history window, wherein the function maps the observed sensory sequence information into a fixed set of discrete classes, fixed for all of the plurality of history windows;
choose at least one parameter of the exponentially increasing history window size as a hyperparameter to allow the system to trade off bias-variance; and
apply a context tree weighting algorithm to an alphabet resulting from the fixed set of discrete classes for each of the plurality of history windows to predict a future discrete sequence; and
use a deep neural network classifier to map arbitrary length histories to a minimal output symbol alphabet as an input sequence for the context tree weighting algorithm.

### Claim 11 (depends on 10)

The artificial intelligence system of claim 10, wherein a long short-term memory-based sequence to symbol method is used to map the arbitrary length histories to the minimal output symbol alphabet.

### Claim 12 (depends on 10)

The artificial intelligence system of claim 10, wherein the observed sensory sequence information is a binary event.

### Claim 13 (depends on 10)

The artificial intelligence system of claim 10, wherein the instructions cause the at least one processor to perform a temporal convolution in a deep neural network to map observed sensory sequence information from the plurality of history windows to symbols.

### Claim 14 (depends on 13)

The artificial intelligence system of claim 13, wherein the temporal convolution includes defining each of the plurality of history windows of events as a ($2{circumflex over ( )}k$)-by-n matrix, where $2{circumflex over ( )}k$ is a number of time steps in each of the plurality of history windows and n is a number of events at each of the time steps, and applying a convolution that is an l-by-n matric, where l is a small number, wherein a set of the convolutions produces a new set of events.

### Claim 15

An artificial intelligence system, comprising: a computing device including at least one processor, one or more data storage devices, and a non-transitory data storage medium interfaced with the at least one processor, the non-transitory data storage medium containing instructions that, when executed cause the at least one processor to:
save observed sensory sequence information in a plurality of history windows, wherein a size of the plurality of history windows increase exponentially from a last observed time step;
apply a function to the observed sensory sequence information in each history window, wherein the function maps the observed sensory sequence information into a fixed set of discrete classes, fixed for all of the plurality of history windows;
choose at least one parameter of the exponentially increasing history window size as a hyperparameter to allow the system to trade off bias-variance;
apply a context tree weighting algorithm to an alphabet resulting from the fixed set of discrete classes for each of the plurality of history windows to predict a future discrete sequence; and
perform a temporal convolution in a deep neural network to map observed sensory sequence information from the plurality of history windows to symbols, wherein
the temporal convolution includes defining each of the plurality of history windows of events as a ($2{circumflex over ( )}k$)-by-n matrix, where $2{circumflex over ( )}k$ is a number of time steps in each of the plurality of history windows and n is a number of events at each of the time steps, and applying a convolution that is an l-by-n matric, where l is a small number, wherein a set of the convolutions produces a new set of events.

### Claim 16 (depends on 15)

The artificial intelligence system of claim 15, wherein the instructions cause the at least one processor to choose at least one parameter of the exponentially increasing history window size as a hyperparameter to allow the system to trade off bias-variance.
