# US 2023/0274137 A1 · Knowledge Transfer

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274137 A1
- Document ID: US20230274137A1
- Status: Published application
- Kind: A1
- Application number: 18/102,411
- Filed: 2023-01-27
- Published: 2023-08-31
- Earliest claimed: 2022-02-28
- Assignees: Fujitsu Limited
- Applicants: Fujitsu Limited
- Inventors: Savvas MAKARIOU; Theodoros KASIOUMIS; Joseph TOWNSEND
- Web page: https://patentgrep.com/patent/US20230274137A1

## CPC

- G06N 3/08: Learning methods
- G06N 5/045: Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/096: Transfer learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computer-implemented method comprising: obtaining, based on an input image, a first activation map of a labelled filter of a first convolutional neural network, the first convolutional neural network being configured to identify one or more first features in the input image; obtaining, based on the input image, a second activation map of a filter of a second convolutional neural network, the second convolutional neural network being configured to identify one or more second features in the input image; calculating a similarity measure between the first activation map and the second activation map; and labelling, when the similarity measure is equal to or above a threshold similarity, the filter of the second convolutional neural network with a label of the labelled filter of the first convolutional neural network.

## Claims

### Claim 1

A computer-implemented method comprising:
obtaining, based on an input image, a first activation map of a labelled filter of a first convolutional neural network, the first convolutional neural network being configured to identify one or more first features in the input image;
obtaining, based on the input image, a second activation map of a filter of a second convolutional neural network, the second convolutional neural network being configured to identify one or more second features in the input image;
calculating a similarity measure between the first activation map and the second activation map; and
labelling, when the similarity measure is equal to or above a threshold similarity, the filter of the second convolutional neural network with a label of the labelled filter of the first convolutional neural network.

### Claim 2 (depends on 1)

The computer-implemented method as claimed in claim 1, wherein the calculating of the similarity measure comprises converting the first activation map and the second activation map into first and second binary matrices, respectively, and calculating the similarity measure between the first and second binary matrices.

### Claim 3 (depends on 2)

The computer-implemented method as claimed in claim 2, further comprising, before the calculating of the similarity measure between the first and second binary matrices, scaling the first and second binary matrices to dimensions of the input image, optionally using nearest neighbors interpolation.

### Claim 4 (depends on 2)

The computer-implemented method as claimed in claim 2, wherein the converting of the first activation map and the second activation map into first and second binary matrices comprises:
setting each activation value which has an absolute value above a threshold value in the first activation map and the second activation map to a first value, and
setting each activation value which has an absolute value equal to or below the threshold value to a second value.

### Claim 5 (depends on 2)

The computer-implemented method as claimed in claim 2, wherein the calculating of the similarity measure comprises calculating an intersection-over-union (IoU) metric between the first and second binary matrices.

### Claim 6 (depends on 1)

The computer-implemented method as claimed in claim 1, wherein the calculating of the similarity measure comprises calculating a cosine distance metric between the first activation map and the second activation map.

### Claim 7 (depends on 1)

The computer-implemented method as claimed in claim 1, further comprising using the second convolutional neural network in control of an autonomous or semi-autonomous vehicle.

### Claim 8 (depends on 1)

The computer-implemented method as claimed in claim 1, further comprising:
re-training the first convolutional neural network to provide the second convolutional neural network.

### Claim 9 (depends on 1)

The computer-implemented method as claimed in claim 1, comprising:
obtaining, based on the input image, a plurality of first activation maps including the first activation map of a plurality of labelled filters of the first convolutional neural network;
calculating a similarity measure for each of a plurality of pairs each comprising the second activation map and one of the plurality of first activation maps; and
labelling the filter of the second convolutional neural network with a label of the labelled filter corresponding to a first activation map belonging to the pair with a highest similarity measure.

### Claim 10 (depends on 1)

The computer-implemented method as claimed in claim 1, comprising:
obtaining, based on the input image, a plurality of first activation maps including the first activation map of a plurality of labelled filters of the first convolutional neural network;
selecting at least one first activation map each having an activation score above a threshold activation score or having a highest activation score;
calculating a similarity measure for each of a plurality of pairs each comprising the second activation map and one of the at least one selected first activation maps; and
labelling the filter of the second convolutional neural network with a label of the labelled filter corresponding to a first activation map belonging to the pair with a highest similarity measure.

### Claim 11 (depends on 1)

The computer-implemented method as claimed in claim 1, wherein for each of a plurality of input images in which the input image is included:
obtaining, based on the input image, a plurality of first activation maps of a plurality of labelled filters of the first convolutional neural network,
obtaining, based on the input image, the second activation map of the filter of the second convolutional neural network, and
for each of a plurality of pairs each comprising the second activation map and one of the plurality of first activation maps, calculating a similarity measure between the first activation map and the second activation map,
wherein the computer-implemented method further comprises:
labelling the filter of the second convolutional neural network with a label of the labelled filter corresponding to the first activation map belonging to the pair having a highest similarity measure among the pairs; or
selecting a label of the labelled filter corresponding to the first activation map belonging to each of at least one pair having the highest similarity measure among the pairs for each of the plurality of images, and when one label has been selected, labelling the filter of the second convolutional neural network with the selected label, and when a plurality of labels have been selected, labelling the filter of the second convolutional neural network with the label appearing most frequently among the selected plurality of labels or label the filter of the second convolutional neural network with a label selected at random from a plurality of labels appearing the most frequently among the selected plurality of labels; or
selecting a label of the labelled filter corresponding to the first activation map belonging to the or each pair having a said similarity measure above or equal to a threshold similarity, and when one label has been selected, labelling the filter of the second convolutional neural network with the selected label, and when a plurality of labels have been selected, labelling the filter of the second convolutional neural network with the label appearing most frequently among the selected plurality of labels or label the filter of the second convolutional neural network with a label selected at random from a plurality of labels appearing the most frequently among the selected plurality of labels.

### Claim 12 (depends on 1)

A computer-implemented method comprising:
obtaining, based on a plurality of input images, a plurality of corresponding activation maps of a filter of a second convolutional neural network, each activation map comprising activation values;
for each input image, calculating an activation score as an aggregation of the activation values of the corresponding activation map and selecting at least one input image having an activation score above a threshold activation score or having a highest activation score among the plurality of input images; and
using the at least one selected input image, implementing the method as claimed claim 1.

### Claim 13 (depends on 1)

A computer-implemented method comprising implementing the computer-implemented method as claimed in claim 1 for a plurality of filters of the second convolutional neural network.

### Claim 14

A non-transitory computer readable medium storing a program which, when run on a computer, causes the computer to carry out a method comprising:
obtaining, based on an input image, a first activation map of a labelled filter of a first convolutional neural network, the first convolutional neural network being configured to identify one or more first features in the input image;
obtaining, based on the input image, a second activation map of a filter of a second convolutional neural network, the second convolutional neural network is configured to identify the one or more second features in the input image;
calculating a similarity measure between the first activation map and the second activation map; and
labelling, when the similarity measure is equal to or above a threshold similarity, the filter of the second convolutional neural network with a label of the labelled filter of the first convolutional neural network.

### Claim 15

An information processing apparatus comprising:
a memory, and
a processor connected to the memory, wherein the processor is configured to:
obtain, based on an input image, a first activation map of a labelled filter of a first convolutional neural network, the first convolutional neural network being configured to identify one or more first features in the input image;
obtain, based on the input image, a second activation map of a filter of a second convolutional neural network, the second convolutional neural network being configured to identify one or more second features in the input image;
calculate a similarity measure between the first activation map and the second activation map; and
label, when the similarity measure is equal to or above a threshold similarity, the filter of the second convolutional neural network with a label of the labelled filter of the first convolutional neural network.
