# US 8,788,441 B1 · Intelligent control with hierarchical stacked neural networks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,788,441 B1 - Document ID: US8788441B1 - Status: Granted - Kind: B1 - Application number: 12/611,317 - Filed: 2009-11-03 - Granted: 2014-07-22 - Earliest claimed: 2002-09-30 - Applicants: Michael Lamport Commons; Mitzi Sturgeon White - Inventors: Michael Lamport Commons; Mitzi Sturgeon White - Web page: https://patentgrep.com/patent/US8788441B1 ## CPC - G06N 3/045: Combinations of networks - G06N 3/0499: Feedforward networks - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 3/096: Transfer learning - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract An intelligent control system based on an explicit model of cognitive development (Table 1) performs high-level functions. It comprises up to O hierarchically stacked neural networks, Nm, . . . , Nm+(O−1), where m denotes the stage/order tasks performed in the first neural network, Nm, and O denotes the highest stage/order tasks performed in the highest-level neural network. The type of processing actions performed in a network, Nm, corresponds to the complexity for stage/order m. Thus N1 performs tasks at the level corresponding to stage/order 1. N5 processes information at the level corresponding to stage/order 5. Stacked neural networks begin and end at any stage/order, but information must be processed by each stage in ascending order sequence. Stages/orders cannot be skipped. Each neural network in a stack may use different architectures, interconnections, algorithms, and training methods, depending on the stage/order of the neural network and the type of intelligent control system implemented. ## Claims ### Claim 1 A stacked neural network, comprising: at least three architecturally distinct, separately trainable, neural networks, each having an input layer, a hidden layer, and an output layer, the neural networks being organized hierarchically with ascending levels of cognitive complexity, and a constrained set of outputs representing predetermined non-arbitrary organizations; each neural network between the hierarchically lowest and the hierarchically highest receiving at least one an input representing a processed output from a next hierarchically inferior neural network which is processed to produce the respective output, and feedback from at least a next hierarchically superior neural network which alters a processing of subsequent inputs with respect to the predetermined non-arbitrary organizations; at least one input to a neural network representing a natural language communication, and at least one output of a neural network representing an identification of at least one object or concept associated with the natural language communication. ### Claim 2 (depends on 1) The stacked neural network of claim 1, wherein the at least three neural networks comprise at least five architecturally distinct neural networks. ### Claim 3 (depends on 1) The stacked neural network of claim 1, wherein at least one output of a neural network represents a relationship between a plurality of words or concepts. ### Claim 4 (depends on 1) The stacked neural network of claim 1, wherein at least one output of a neural network, and at least one output represents an appropriate response to the natural language communication. ### Claim 5 (depends on 4) The stacked neural network of claim 4, wherein at least one output of a neural network represents an action to be taken based on the relationship between the plurality of words or concepts. ### Claim 6 (depends on 1) The stacked neural network of claim 1, wherein at least one neural network is subject to supervised training. ### Claim 7 (depends on 6) The stacked neural network of claim 6, wherein at least one neural network is subject to unsupervised training. ### Claim 8 (depends on 1) The stacked neural network of claim 1, wherein neurons within at least one neural network receive both excitatory inputs and inhibitory inputs. ### Claim 9 (depends on 1) The stacked neural network of claim 1, wherein at least one neural network has a plurality of neurons organized into a plurality of clusters, which are respectively activated together to represent a respective predetermined non-arbitrary organization. ### Claim 10 (depends on 9) The stacked neural network of claim 9, wherein the at least one neural network makes a decision based on a relative strength of respective joint domain patterns of respective clusters, and wherein a strength of the respective joint domain patterns is dependent on the feedback received by the at least one neural network. ### Claim 11 A method of processing information, comprising: providing a stacked neural network comprising at least three architecturally distinct, separately trainable, neural networks, each having an input layer, an output layer, and a hidden layer, the neural networks being organized hierarchically with ascending levels of cognitive complexity, each having a constrained set of outputs representing non-arbitrary organizations, each neural network between the lowest and the highest each receiving at least one input representing a processed output from a next hierarchically inferior neural network which is processed to select a member of the set of outputs, and feedback from at least a next hierarchically superior neural network which alters a processing of subsequent inputs into the non-arbitrary organizations; receiving natural language as an input to at least one neural network; and producing as an output from at least one neural network an identification of at least one object or concept referred to by the natural language. ### Claim 12 (depends on 11) The method of claim 11, further comprising producing as an output a contextually appropriate response to the natural language. ### Claim 13 (depends on 11) The method of claim 11, further comprising adapting an output of a neural network to alter an input-to-output response based on feedback from a plurality of neural networks having respectively superior hierarchical positions. ### Claim 14 (depends on 11) The method of claim 11, further comprising representing a relationship between a plurality of words or concepts as an output of at least one neural network. ### Claim 15 (depends on 14) The method of claim 14, further comprising representing an action to be taken based a the relationship between the plurality of words or concepts as an output of at least one neural network. ### Claim 16 (depends on 11) The method of claim 11, further comprising training at least one neural network within the stacked neural network. ### Claim 17 (depends on 11) The method of claim 11, further comprising replacing one of the neural networks at a respective hierarchical level with another neural network trained within a different stacked neural network. ### Claim 18 (depends on 11) The method of claim 11, wherein neurons within at least one neural network receive excitatory inputs and inhibitory inputs. ### Claim 19 (depends on 11) The method of claim 11, wherein at least one neural network has a plurality of neurons organized into a plurality of clusters, which are respectively activated together based on an input pattern associated with a respective non-arbitrary organization. ### Claim 20 (depends on 19) The method of claim 19, further comprising making a decision by at least one neural network based on a relative strength of joint domain patterns of respective clusters, and wherein a strength of the respective joint domain patterns is dependent on the feedback.