US 2019/0244093 A1Published application

Generalized operational perceptrons: new generation artificial neural networks

AssigneeQatar University

InventorsSerkan KIRANYAZ, Turker INCE, Moncef GABBOUJ, Alexandros IOSIFIDIS

Life of this patent

Estimating the term…

Loading patent…

Abstract

Certain embodiments may generally relate to various techniques for machine learning. Feed-forward, fully-connected Artificial Neural Networks (ANNs), or the so-called Multi-Layer Perceptrons (MLPs) are well-known universal approximators. However, their learning performance may vary significantly depending on the function or the solution space that they attempt to approximate for learning. This is because they are based on a loose and crude model of the biological neurons promising only a linear transformation followed by a nonlinear activation function. Therefore, while they learn very well those problems with a monotonous, relatively simple and linearly separable solution space, they may entirely fail to do so when the solution space is highly nonlinear and complex. In order to address this drawback and also to accomplish a more generalized model of biological neurons and learning systems, Generalized Operational Perceptrons (GOPs) may be formed and they may encapsulate many linear and nonlinear operators.

Claims15

Dependent claims

A method, comprising:

receiving data at an input neural node of an input layer, the received data corresponding to a learning objective that is to be accomplished;
initializing a final progressive operational perceptron (POP) by assigning the input layer as an input layer of a maximum POP configuration (POPmax);
forming a 3-layer, single hidden layer multi-layered progressive operational perceptron (1st GOPmin) using the configuration of the input layer, a first hidden layer and an output layer of the POPmax;
inserting the formed hidden layer of the 1st GOPmin as a first hidden layer of the final POP;
generating learning performance statistics of the 1st GOPmin;
determining if the learning objective can be achieved with the 1st GOPmin;
if the learning objective is achieved, terminate the formation process;
if the learning objective cannot be achieved with the 1st GOPmin, using a previous hidden layer's output as the input layer by forward propagating training data, forming a second 3-layer, single hidden layer multi-layered progressive operational perceptron (2nd GOPmin) using the configurations of a second hidden layer of the POPmax as the hidden layer and the output layer of the, POPmax, as the output layer;
forming the 2nd GOPmin and inserting the formed hidden layer of the 2nd GOPmin as a 2nd hidden layer of the final POP;
generating learning performance statistics of the 2nd GOPmin;
checking if a target performance is achieved or not with the 2nd GOPmin, and if not achieved, repeat:
forming, checking and inserting in the same order for a third, fourth, and additional GOPmin, until the target performance is achieved or all hidden layers of POPmax are formed;
and
forming the output layer of the final POP as the output layer of the last GOPmin formed.

11 more elements

An apparatus, comprising:

at least one memory comprising computer program code;
and
at least one processor;
wherein the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus at least to:
receive data at an input neural node of an input layer, the received data corresponding to a learning objective that is to be accomplished;
form a 3-layer, single hidden layer multi-layered progressive operational perceptron (1st GOPmin) using a first hidden layer and an output layer of a maximum progressive operational perceptron (POP) configuration, (POPmax);
determine if the learning objective can be achieved with the first hidden layer;
if the learning objective cannot be achieved with the first hidden layer, use a previous hidden layer's output as the input layer, forming a second 3-layer, single hidden layer multi-layered progressive operational perceptron (2nd GOPmin) using a second hidden layer as the only hidden layer and the output layer of the POPmax;
train the 2nd GOPmin and check if a target performance is achieved or not, and if not achieved, repeat the training and checking until the target performance is achieved or all hidden layers of POPmax are formed;
form an output layer corresponding to the first hidden layer and any additional hidden layers;
and
generate learning performance statistics based on the received data.

9 more elements

A computer program, embodied on a non-transitory computer readable medium, the computer program, when executed by a processor, causes the processor to:

receive data at an input neural node of an input layer, the received data corresponding to a learning objective that is to be accomplished;
form a 3-layer, single hidden layer multi-layered progressive operational perceptron (1st GOPmin) using a first hidden layer and an output layer of a maximum progressive operational perceptron (POP) configuration, (POPmax);
determine if the learning objective can be achieved with the first hidden layer;
if the learning objective cannot be achieved with the first hidden layer, use a previous hidden layer's output as the input layer, forming a second 3-layer, single hidden layer multi-layered progressive operational perceptron (2nd GOPmin) using a second hidden layer as the only hidden layer and the output layer of the POPmax;
train the 2nd GOPmin and check if a target performance is achieved or not, and if not achieved, repeat the training and checking until the target performance is achieved or all hidden layers of POPmax are formed;
form an output layer corresponding to the first hidden layer and any additional hidden layers;
and
generate learning performance statistics based on the received data.

5 more elements

Dependencies are inferred from the claim text.