# US 2018/0322385 A1 · Efficient learning and using of topologies of neural networks in machine learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322385 A1
- Document ID: US20180322385A1
- Status: Published application
- Kind: A1
- Application number: 15/659,853
- Filed: 2017-07-26
- Published: 2018-11-08
- Earliest claimed: 2017-05-05
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: RAANAN YONATAN YEHEZKEL ROHEKAR; Guy Koren; Shami Nisimov; Gal Novik
- Web page: https://patentgrep.com/patent/US20180322385A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0475: Generative networks
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/09: Supervised learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A mechanism is described for facilitating learning and application of neural network topologies in machine learning at autonomous machines. A method of embodiments, as described herein, includes monitoring and detecting structure learning of neural networks relating to machine learning operations at a computing device having a processor, and generating a recursive generative model based on one or more topologies of one or more of the neural networks. The method may further include converting the generative model into a discriminative model.

## Claims

### Claim 1

An apparatus comprising:
detection/observation logic, as facilitated by or at least partially incorporated into a processor, to monitor and detect structure learning of neural networks relating to machine learning operations at the apparatus having the processor;
generative model logic, as facilitated by or at least partially incorporated into the processor, to generate a recursive generative model based on one or more topologies of one or more of the neural networks; and
discriminative model logic, as facilitated by or at least partially incorporated into the processor, to convert the generative model into a discriminative model.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the generative model is unsupervised and based on unlabeled data, and wherein the discriminative mode is supervised and based on labeled data, wherein the discriminative model is learned from the generative model.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the discriminative model logic is further to inverse the generative model into multiple inverse models, wherein a bidirectional connection is added to connect latent variables having a common parent in each of the multiple inverse models to consolidate the multiple inverse models into a single inverse model.

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein the discriminative model logic is further to convert the inverse model into the discriminative model by removing the bidirectional connection and adding a class node serving as a child node to latent leaves.

### Claim 5 (depends on 1)

The apparatus of claim 1, further comprising:
dropout logic, as facilitated by or at least partially incorporated into the processor, to perform methodological dropout of neurons from one or more of the neural networks, wherein the methodological dropout is performed in accordance with a predictivity based on historical statistical data relating to the neurons;
decomposition logic, as facilitated by or at least partially incorporated into the processor, to generate parallel and sequential execution schedules for memory sharing at sub-network precision levels of the one or more of the neural networks; and
on-the-fly learning/update logic, as facilitated by or at least partially incorporated into the processor, to perform on-the-fly learning and updating of network topologies of the neural networks based on at least one of currently available data and historically available data relating to the topologies of the neural networks.

### Claim 6 (depends on 1)

The apparatus of claim 1, further comprising:
structure learning logic, as facilitated by or at least partially incorporated into the processor, to facilitate at least one of an end-to-end structure learning and a sub-network structure learning; and
training and feature logic, as facilitated by or at least partially incorporated into the processor, to facilitate feature bagging or coping with large scale data by training large training sets.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the processor comprises a graphics processor co-located with an application processor on a common semiconductor package.

### Claim 8

A method comprising:
monitoring and detecting structure learning of neural networks relating to machine learning operations at a computing device having a processor;
generating a recursive generative model based on one or more topologies of one or more of the neural networks; and
converting the generative model into a discriminative model.

### Claim 9 (depends on 8)

The method of claim 8, wherein the generative model is unsupervised and based on unlabeled data, and wherein the discriminative mode is supervised and based on labeled data, wherein the discriminative model is learned from the generative model.

### Claim 10 (depends on 8)

The method of claim 8, further comprising inversing the generative model into multiple inverse models, wherein a bidirectional connection is added to connect latent variables having a common parent in each of the multiple inverse models to consolidate the multiple inverse models into a single inverse model.

### Claim 11 (depends on 10)

The method of claim 10, further comprising converting the inverse model into the discriminative model by removing the bidirectional connection and adding a class node serving as a child node to latent leaves.

### Claim 12 (depends on 8)

The method of claim 8, further comprising:
performing methodological dropout of neurons from one or more of the neural networks, wherein the methodological dropout is performed in accordance with a predictivity based on historical statistical data relating to the neurons;
generating parallel and sequential execution schedules for memory sharing at sub-network precision levels of the one or more of the neural networks; and
performing on-the-fly learning and updating of network topologies of the neural networks based on at least one of currently available data and historically available data relating to the topologies of the neural networks.

### Claim 13 (depends on 8)

The method of claim 8, further comprising:
facilitating at least one of an end-to-end structure learning and a sub-network structure learning; and
facilitating feature bagging or coping with large scale data by training large training sets.

### Claim 14 (depends on 8)

The method of claim 8, wherein the processor comprises a graphics processor co-located with an application processor on a common semiconductor package.

### Claim 15

At least one machine-readable medium comprising instructions that when executed by a computing device, cause the computing device to perform operations comprising:
monitoring and detecting structure learning of neural networks relating to machine learning operations at a computing device having a processor;
generating a recursive generative model based on one or more topologies of one or more of the neural networks; and
converting the generative model into a discriminative model.

### Claim 16 (depends on 15)

The machine-readable medium of claim 15, wherein the generative model is unsupervised and based on unlabeled data, and wherein the discriminative mode is supervised and based on labeled data, wherein the discriminative model is learned from the generative model.

### Claim 17 (depends on 15)

The machine-readable medium of claim 15, wherein the operations further comprise inversing the generative model into multiple inverse models, wherein a bidirectional connection is added to connect latent variables having a common parent in each of the multiple inverse models to consolidate the multiple inverse models into a single inverse model.

### Claim 18 (depends on 17)

The machine-readable medium of claim 17, wherein the operations further comprise converting the inverse model into the discriminative model by removing the bidirectional connection and adding a class node serving as a child node to latent leaves.

### Claim 19 (depends on 15)

The machine-readable medium of claim 15, wherein the operations further comprise:
performing methodological dropout of neurons from one or more of the neural networks, wherein the methodological dropout is performed in accordance with a predictivity based on historical statistical data relating to the neurons;
generating parallel and sequential execution schedules for memory sharing at sub-network precision levels of the one or more of the neural networks; and
performing on-the-fly learning and updating of network topologies of the neural networks based on at least one of currently available data and historically available data relating to the topologies of the neural networks.

### Claim 20 (depends on 15)

The machine-readable medium of claim 15, wherein the operations further comprise:
facilitating at least one of an end-to-end structure learning and a sub-network structure learning; and
facilitating feature bagging or coping with large scale data by training large training sets, wherein the processor comprises a graphics processor co-located with an application processor on a common semiconductor package.
