Channel gating for conditional computation

A computing device, comprising:

• a processor configured with processor-executable instructions to:
• receive a layer-specific input data that is specific to a current layer of a neural network that includes two or more filters;
• generate statistics based on the received layer-specific input data, wherein generating the statistics comprises using global average pooling of the layer-specific input data to generate a global average pooling result;
• use the generated statistics to assign a relevance to each of the two or more filters, wherein each assigned relevance indicates a relevance of a corresponding filter to the received layer-specific input data, wherein determining the relevance comprises applying the global average pooling result to a gating functionality of at least one filter of the two or more filters;
• selectively activating or deactivating each of the two or more filters in the current layer based on the corresponding relevance by matching a distribution of an output of the gating functionality network using a cumulative distribution function (CDF) loss function;
• and
• apply the received layer-specific input data to activated filters in the two or more filters to generate an output activation for the current layer of the neural network.

A non-transitory processor-readable storage medium having stored thereon processor-executable instructions to cause a processor in a computing device to activate gating within a current layer of a neural network that includes two or more filters and perform operations comprising:

• receiving, by a processor in a computing device, a layer-specific input data that is specific to the current layer of the neural network;
• generating statistics based on the received layer-specific input data, wherein generating the statistics comprises using global average pooling of the layer-specific input data to generate a global average pooling result;
• using the generated statistics to assign a relevance to each of the two or more filters, wherein each assigned relevance indicates a relevance of a corresponding filter to the received layer-specific input data, wherein determining the relevance comprises applying the global average pooling result to a gating functionality of at least one filter of the two or more filters;
• selectively activating or deactivating each of the two or more filters in the current layer based on the corresponding relevance by matching a distribution of an output of the gating functionality using a cumulative distribution function (CDF) loss function;
• and
• applying the received layer-specific input data to activated filters in the two or more filters to generate an output activation for the current layer of the neural network.