# US 2026/0065654 A1 · Information processing device, relearning system, and relearning method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065654 A1
- Document ID: US20260065654A1
- Status: Published application
- Kind: A1
- Application number: 19/381,942
- Filed: 2025-11-06
- Published: 2026-03-05
- Earliest claimed: 2023-07-05
- Assignees: Mitsubishi Electric Corporation
- Applicants: Mitsubishi Electric Corporation
- Inventors: Takafumi KOIKE
- Web page: https://patentgrep.com/patent/US20260065654A1

## CPC

- G06V 10/778: Active pattern-learning, e.g. online learning of image or video features
- G06T 7/00: Image analysis
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V 10/56: relating to colour
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/776: Validation; Performance evaluation
- G06V 10/82: using neural networks
- G06V 40/107: {Static hand or arm}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An information processing device includes an acquisition unit that acquires an inference object image and a learned model, an inference unit that makes an inference by using the inference object image and the learned model, a generation unit that generates a heat map by using an inference result, an extraction unit that extracts a plurality of features based on the heat map, and a relearning processing unit. The generation unit generates a plurality of modification images by using the inference object image. The inference unit makes the inference by using the modification images and the learned model. The generation unit generates heat maps by using a plurality of inference results. When an inference basis is erroneous, the relearning processing unit generates relearning data, in which the feature indicated by the inference basis has been modified, by using learning data and relearns the learned model.

## Claims

### Claim 1

An information processing device comprising:
acquiring circuitry to acquire an inference object image and a learned model;
inferring circuitry to make an inference by using the inference object image and the learned model;
generating circuitry to generate a heat map that indicates a basis of an inference result by using the inference result;
extracting circuitry to extract a plurality of features based on a region in the inference object image, the region being a region corresponding to a part as the basis of the inference result indicated by the heat map; and
relearning processing circuitry, wherein
when the plurality of features has been extracted, the generating circuitry generates a plurality of modification images by making a modification in regard to each of the features by using the inference object image,
when the plurality of modification images has been generated, the inferring circuitry makes the inference by using the plurality of modification images and the learned model,
when the inference has been made by using the plurality of modification images and the learned model, the generating circuitry generates a plurality of heat maps by using a plurality of inference results,
the acquiring circuitry acquires learning data when an inference basis is erroneous based on the plurality of modification images and the plurality of heat maps, or inference basis information generated based on the inference object image, the heat map, the plurality of modification images and the plurality of heat maps, and
the relearning processing circuitry generates relearning data, in which the feature indicated by the inference basis has been modified, by using the learning data and relearns the learned model by using the relearning data.

### Claim 2 (depends on 1)

The information processing device according to claim 1, further comprising outputting circuitry to output the plurality of modification images and the plurality of heat maps, or the inference basis information,
wherein the acquiring circuitry acquires the learning data when a modification command for modifying the feature indicated by the inference basis and a relearning command are acquired after the outputting of the plurality of modification images and the plurality of heat maps, or the inference basis information.

### Claim 3 (depends on 1)

The information processing device according to claim 1, further comprising identifying circuitry to identify a heat map having a greatest difference from the heat map corresponding to the inference object image out of the plurality of heat maps, wherein
the generating circuitry generates a feature, that was modified when generating the modification image corresponding to the identified heat map, as the inference basis information being textual information,
the acquiring circuitry acquires correct answer information indicating a correct inference basis upon the generation of the inference basis information,
the relearning processing circuitry judges whether the inference basis is erroneous or not based on the inference basis information and the correct answer information,
the acquiring circuitry acquires the learning data when the inference basis is erroneous, and
the relearning processing circuitry generates the relearning data, in which the feature indicated by the inference basis information has been modified, by using the learning data.

### Claim 4 (depends on 1)

The information processing device according to claim 1, further comprising identifying circuitry to identify a plurality of heat maps having a difference from the heat map corresponding to the inference object image as a plurality of identified heat maps out of the plurality of heat maps, wherein
the generating circuitry generates a plurality of features, that were modified when generating a plurality of modification images corresponding to the plurality of identified heat maps, as the inference basis information being a plurality of pieces of textual information,
the acquiring circuitry acquires correct answer information indicating a correct inference basis upon the generation of the inference basis information,
the relearning processing circuitry judges whether there is an error in inference bases or not based on the inference basis information and the correct answer information,
the acquiring circuitry acquires the learning data when there is an error in the inference bases, and
the relearning processing circuitry generates the relearning data, in which a feature having an error in the inference basis among the plurality of features indicated by the plurality of pieces of textual information has been modified, by using the learning data.

### Claim 5

A relearning system including a plurality of information processing devices, comprising:
acquiring circuitry to acquire an inference object image and a learned model;
inferring circuitry to make an inference by using the inference object image and the learned model;
generating circuitry to generate a heat map that indicates a basis of an inference result by using the inference result;
extracting circuitry to extract a plurality of features based on a region in the inference object image, the region being a region corresponding to a part as the basis of the inference result indicated by the heat map; and
relearning processing circuitry, wherein
when the plurality of features has been extracted, the generating circuitry generates a plurality of modification images by making a modification in regard to each of the features by using the inference object image,
when the plurality of modification images has been generated, the inferring circuitry makes the inference by using the plurality of modification images and the learned model,
when the inference has been made by using the plurality of modification images and the learned model, the generating circuitry generates a plurality of heat maps by using a plurality of inference results,
the acquiring circuitry acquires learning data when an inference basis is erroneous based on the plurality of modification images and the plurality of heat maps, or inference basis information generated based on the inference object image, the heat map, the plurality of modification images and the plurality of heat maps, and
the relearning processing circuitry generates relearning data, in which the feature indicated by the inference basis has been modified, by using the learning data and relearns the learned model by using the relearning data.

### Claim 6

A relearning method performed by an information processing device, the relearning method comprising:
acquiring an inference object image and a learned model;
making an inference by using the inference object image and the learned model;
generating a heat map that indicates a basis of an inference result by using the inference result;
extracting a plurality of features based on a region in the inference object image, the region being a region corresponding to a part as the basis of the inference result indicated by the heat map;
generating a plurality of modification images by making a modification in regard to each of the features by using the inference object image;
making the inference by using the plurality of modification images and the learned model;
generating a plurality of heat maps by using a plurality of inference results;
acquiring learning data when an inference basis is erroneous based on the plurality of modification images and the plurality of heat maps, or inference basis information generated based on the inference object image, the heat map, the plurality of modification images and the plurality of heat maps;
generating relearning data, in which the feature indicated by the inference basis has been modified, by using the learning data; and
relearning the learned model by using the relearning data.

### Claim 7

An information processing device comprising:
a processor to execute a program; and
a memory to store the program which, when executed by the processor, performs processes of,
acquiring an inference object image and a learned model,
making an inference by using the inference object image and the learned model,
generating a heat map that indicates a basis of an inference result by using the inference result,
extracting a plurality of features based on a region in the inference object image, the region being a region corresponding to a part as the basis of the inference result indicated by the heat map,
generating a plurality of modification images by making a modification in regard to each of the features by using the inference object image,
making the inference by using the plurality of modification images and the learned model,
generating a plurality of heat maps by using a plurality of inference results,
acquiring learning data when an inference basis is erroneous based on the plurality of modification images and the plurality of heat maps, or inference basis information generated based on the inference object image, the heat map, the plurality of modification images and the plurality of heat maps,
generating relearning data, in which the feature indicated by the inference basis has been modified, by using the learning data, and
relearning the learned model by using the relearning data.
