Sparse Highly Connected Artificial Neural Network Architectures Involving Hybrid Scale Structure

A method of constructing a geometry-induced sparse hybrid highly connected artificial neural network architecture with a hierarchy of edge length scales, the method comprising:

• at least one processing unit, a computer readable memory and a computer non-transitory readable storage medium associated with a computing device;
• selecting a geometry defined in terms of a manifold;
• selecting a direction of data flow in the geometry;
• selecting a node set as a finite subset of the geometry, where nodes specify the locations of artificial neurons in the geometry-induced sparse hybrid highly connected artificial neural network architecture with a hierarchy of edge length scales;
• partitioning the node set into layers with respect to the geometry and the direction of data flow, wherein each node belongs to a unique layer of the layers, and wherein the layers are ordered, wherein the first layer of the layers is called the input layer, and the last layer of the layers is called the output layer;
• selecting edges between each node in each non-input layer of the layers and specified nodes in one or more preceding layers of the layers with respect to the geometry and the direction of data flow, wherein the set of edges is partitioned into several families defined by specified disjoint ranges of edge lengths as determined by the geometry, wherein each family is comprised of edges with lengths within the corresponding specified range, wherein these families are arranged in a hierarchy beginning with the family comprised of the shortest edges and ending with the family comprised of the longest edges, wherein the length ranges defining each family are related to each other by predetermined multiples, and wherein the edges specify locations of synaptic connections between pairs of artificial neurons in the geometry-induced sparse hybrid highly connected artificial neural network architecture with a hierarchy of edge length scales;
• modifying a predetermined percentage of the synaptic connections in the geometry-induced sparse hybrid highly connected artificial neural network architecture with a hierarchy of edge length scales to achieve a specific target connectivity or to accommodate boundary effects;
• implementing the geometry-induced sparse hybrid highly connected artificial neural network architecture with a hierarchy of edge length scales concretely by assigning one or more edge weights to each synaptic connection and one or more activation functions and bias parameters to each artificial neuron;
• and
• training the geometry-induced sparse hybrid highly connected artificial neural network architecture with a hierarchy of edge length scales using at least one dataset, wherein the training is accomplished by means of a specified algorithm, and wherein the trained geometry-induced sparse hybrid highly connected artificial neural network architecture with a hierarchy of edge length scales performs operations that solve specified problems and/or carry out specified tasks.

A method of constructing a subgraph family of geometry-induced sparse hybrid highly connected artificial neural network architectures with hierarchies of edge length scales, the method comprising:

• at least one processing unit, a computer readable memory and a computer non-transitory readable storage medium associated with a computing device;
• selecting a geometry defined in terms of a manifold;
• selecting a direction of data flow in the geometry;
• selecting an ambient node set as a finite subset of the geometry;
• partitioning the ambient node set into ambient layers with respect to the geometry and the direction of data flow, wherein each node in the ambient node set belongs to a unique layer of the ambient layers, and wherein the ambient layers are ordered, wherein the first layer of the ambient layers is called the ambient input layer, and wherein the last layer of the ambient layers is called the ambient output layer;
• selecting an ambient edge set by defining an edge between each node in each non-input layer of the ambient layers and each node in each preceding layer of the ambient layers;
• selecting an ambient digraph defined to consist of the ambient node set together with the ambient edge set;
• selecting a family of subgraphs of the ambient digraph by applying a sequence of subgraph selection processes to the ambient digraph, wherein each subgraph thereby chosen defines a member of a subgraph family of geometry-induced sparse hybrid highly connected artificial neural network architectures with hierarchies of edge length scales, and wherein each subgraph selection process consists of the following steps:
• selecting a subgraph node set consisting of a subset of the ambient node set, wherein the subgraph node set is partitioned into layers, called subgraph layers, uniquely determined by the ambient layers, and wherein the subgraph layers are thereby ordered, wherein the first layer of the subgraph layers is called the subgraph input layer, and wherein the last layer of the subgraph layers is called the subgraph output layer, and wherein the nodes in the subgraph node set specify the locations of artificial neurons in the corresponding member of the subgraph family of geometry-induced sparse hybrid highly connected artificial neural network architectures with hierarchies of edge length scales;
• selecting a subgraph edge set consisting of edges connecting each node in each non-input layer of the subgraph layers to specified nodes in one or more preceding layers of the subgraph layers, wherein the subgraph edge set is partitioned into several families defined by specified disjoint ranges of edge lengths as determined by the geometry, wherein each family of edges in the subgraph node sets is comprised of edges with lengths within the corresponding specified range, wherein these families are arranged in a hierarchy beginning with the family comprised of the shortest edges in the subgraph node set and ending with the family comprised of the longest edges in the subgraph node set, wherein the length ranges defining each family of edges in the subgraph node set are related to each other by specified multiples, and wherein the edges in the subgraph edge set specify locations of synaptic connections between pairs of artificial neurons in the corresponding specific member of the subgraph family of geometry-induced sparse hybrid highly connected artificial neural network architectures with hierarchies of edge length scales;
• modifying a predetermined percentage of the synaptic connections for each member of the subgraph family of geometry-induced sparse hybrid highly connected artificial neural network architectures with hierarchies of edge length scales to achieve specific target connectivities or to accommodate boundary effects;
• implementing the subgraph family of geometry-induced sparse hybrid highly connected artificial neural network architectures with hierarchies of edge length scales concretely by assigning one or more edge weights to each synaptic connection and one or more activation functions and bias parameters to each artificial neuron;
• and
• training each member of the subgraph family of geometry-induced sparse hybrid highly connected artificial neural network architectures with hierarchies of edge length scales using at least one dataset, wherein the training is accomplished by means of a specified algorithm, and wherein the trained members of the subgraph family of geometry-induced sparse hybrid highly connected artificial neural network architectures with hierarchies of edge length scales are combined in a specified manner to perform operations that solve specified problems and/or carry out specified tasks.