US 12,608,588 B2Granted

Methods, systems, and media for contextual discriminative explanation of convolutional neural networks

AssigneeHuawei Cloud Computing Technologies Co., Ltd.

InventorsCho Ho Lam, Yong Zhang

Life of this patent

Estimating the term…

Loading patent…

Abstract

Methods, systems and media for explaining a basis for a binary inference task performed by a neural network. Binary relationships are identified between features detected by the trained neural network and data samples in which those features are detected. Sets of features providing an inference context are then identified in tandem with sets of features providing the inference basis within a given inference context. Each context, basis pair potentially provides an explanation for the inference behavior of the neural network. Human perceptible representations of such pairs are generated to explain the inference behavior of the neural network, including automatically identifying similar samples sharing both the context and basis, and automatically identifying likely alternative samples sharing the context but not the basis.

Claims16

Dependent claims

A method for explaining a basis for a binary inference task performed by a neural network trained to perform the binary inference task, comprising:

obtaining a data set comprising a plurality of data samples, the plurality of data samples comprising:
a negative data subset comprising a plurality of negative data samples associated with a negative result of the binary inference task performed by the trained neural network;
and
a positive data subset comprising a plurality of positive data samples associated with a positive result of the binary inference task performed by the trained neural network;
processing the data set, using the trained neural network, to generate a set of relationships between features and data samples,
the set of relationships identifying, for each feature of a plurality of features, one or more data samples of the data set associated with the feature;
and
processing the set of relationships to generate at least one contextual discriminative pattern (CDP), each CDP identifying:
an inference context comprising one or more contextual features of the plurality of features, the inference context being associated with a first plurality of negative data samples of the negative data subset and a second plurality of positive data samples of the positive data subset;
and
an inference basis comprising one or more discriminative features of the plurality of features,
wherein processing the set of relationships to generate at least one CDP comprises:
selecting, from the plurality of features, the one or more contextual features of a first CDP and the one or more discriminative features of the first CDP such that a discriminativeness value of the inference basis within the inference context is above a discriminativeness threshold,
wherein the discriminativeness value of the inference basis within the inference context is computed as a difference between:
a first ratio between:
a first number indicating how many data samples in the positive data subset are associated with both the one or more discriminative features and the one or more contextual features;
and
a second number indicating how many data samples in the positive data subset are associated with the one or more contextual features;
and
a second ratio between:
a third number indicating how many data samples in the negative data subset are associated with both the one or more discriminative features and the one or more contextual features;
and
a fourth number indicating how many data samples in the negative data subset are associated with the one or more contextual features,
wherein the inference basis is associated with the second plurality of positive data samples and the first plurality of negative data samples such that the first ratio is greater than the second ratio.

21 more elements

A system comprising:

a processing system comprising one or more processor devices;
and
a memory storing instructions which, when executed by the processing system, cause the system to explain a basis for a binary inference task performed by a neural network trained to perform the binary inference task, by:
obtaining a data set comprising a plurality of data samples, the plurality of data samples comprising:
a negative data subset comprising a plurality of negative data samples associated with a negative result of the binary inference task performed by the trained neural network;
and
a positive data subset comprising a plurality of positive data samples associated with a positive result of the binary inference task performed by the trained neural network;
processing the data set, using the trained neural network, to generate a set of relationships between features and data samples,
the set of relationships identifying, for each feature of a plurality of features, one or more data samples of the data set associated with the feature;
and
processing the set of relationships to generate at least one contextual discriminative pattern (CDP), each CDP identifying:
an inference context comprising one or more contextual features of the plurality of features, the inference context being associated with a first plurality of negative data samples of the negative data subset and a second plurality of positive data samples of the positive data subset;
and
an inference basis comprising one or more discriminative features of the plurality of features,
wherein processing the set of relationships to generate at least one CDP comprises:
selecting, from the plurality of features, the one or more contextual features of a first CDP and the one or more discriminative features of the first CDP such that a discriminativeness value of the inference basis within the inference context is above a discriminativeness threshold,
wherein the discriminativeness value of the inference basis within the inference context is computed as a difference between:
a first ratio between:
a first number indicating how many data samples in the positive data subset are associated with both the one or more discriminative features and the one or more contextual features;
and
a second number indicating how many data samples in the positive data subset are associated with the one or more contextual features;
and
a second ratio between:
a third number indicating how many data samples in the negative data subset are associated with both the one or more discriminative features and the one or more contextual features;
and
a fourth number indicating how many data samples in the negative data subset are associated with the one or more contextual features,
wherein the inference basis is associated with the second plurality of positive data samples and the first plurality of negative data samples such that the first ratio is greater than the second ratio.

24 more elements

A non-transitory computer-readable medium having instructions tangibly stored thereon that, when executed by a processing system, cause the processing system to explain a basis for a binary inference task performed by a neural network trained to perform the binary inference task, by:

obtaining a data set comprising a plurality of data samples, the plurality of data samples comprising:
a negative data subset comprising a plurality of negative data samples associated with a negative result of the binary inference task performed by the trained neural network;
and
a positive data subset comprising a plurality of positive data samples associated with a positive result of the binary inference task performed by the trained neural network;
processing the data set, using the trained neural network, to generate a set of relationships between features and data samples,
the set of relationships identifying, for each feature of a plurality of features, one or more data samples of the data set associated with the feature;
and
processing the set of relationships to generate at least one contextual discriminative pattern (CDP), each CDP identifying:
an inference context comprising one or more contextual features of the plurality of features, the inference context being associated with a first plurality of negative data samples of the negative data subset and a second plurality of positive data samples of the positive data subset;
and
an inference basis comprising one or more discriminative features of the plurality of features,
wherein processing the set of relationships to generate at least one CDP comprises:
selecting, from the plurality of features, the one or more contextual features of a first CDP and the one or more discriminative features of the first CDP such that a discriminativeness value of the inference basis within the inference context is above a discriminativeness threshold,
wherein the discriminativeness value of the inference basis within the inference context is computed as a difference between:
a first ratio between:
a first number indicating how many data samples in the positive data subset are associated with both the one or more discriminative features and the one or more contextual features;
and
a second number indicating how many data samples in the positive data subset are associated with the one or more contextual features;
and
a second ratio between:
a third number indicating how many data samples in the negative data subset are associated with both the one or more discriminative features and the one or more contextual features;
and
a fourth number indicating how many data samples in the negative data subset are associated with the one or more contextual features,
wherein the inference basis is associated with the second plurality of positive data samples and the first plurality of negative data samples such that the first ratio is greater than the second ratio.

21 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/747,672Filed 18 May 2022
Published: US 2023/0376724 A123 Nov 2023
Granted: 21 Apr 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification10 codes

G06N 3/04
Neural networks: architecture, e.g. interconnection topology
G06N 3/0464
Architecture, e.g. interconnection topology: convolutional networks [CNN, ConvNet]
G06N 3/09
Learning methods: supervised learning
G06N 5/045
Inference or reasoning models: explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence