# US 12,224,063 B2 · Information processing device, information processing method, and information processing system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,224,063 B2
- Document ID: US12224063B2
- Status: Granted
- Kind: B2
- Application number: 17/432,165
- Filed: 2020-02-18
- Published: 2022-06-02
- Granted: 2025-02-11
- Earliest claimed: 2019-02-28
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: Kenji Yamane; Shinji Watanabe; Takeshi Kunihiro; Kazuki Aisaka
- Web page: https://patentgrep.com/patent/US12224063B2

## CPC

- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G01N 33/48: Biological material, e.g. blood, urine (G01N33/02, G01N33/26, G01N33/44, G01N33/46 take precedence); Haemocytometers (counting blood corpuscules distributed over a surface by scanning the surface G06M11/02)
- G06T 7/0014: {using an image reference approach}
- G16H 30/40: for processing medical images, e.g. editing
- G06T 2207/20081: Training; Learning
- G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro
- G06T 2207/30096: Tumor; Lesion
- G06T 2207/30204: Marker
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An information processing device is provided that includes an identification data acquisition unit that acquires image data for identification in which a first biological region is captured and first site information of the first biological region, and an identification unit that selects an identifier corresponding to the first site information from identifiers associated respectively with a plurality of pieces of site information, and performs an identification process on the basis of the identifier and the image data for identification.

## Claims

### Claim 1

A first information processing device, comprising:
circuitry configured to:
acquire first image data of an identification target, wherein the first image data includes a first biological region of the identification target;
acquire first site information of a plurality of pieces of site information, wherein
the first site information is associated with the first biological region, and
each piece of site information of the plurality of pieces of site information is associated with a respective identifier of a plurality of identifiers;
select, from the plurality of identifiers, a first identifier corresponding to the acquired first site information;
execute an identification process based on each of the selected first identifier and the acquired first image data;
acquire second image data of the identification target, wherein the second image data includes a second biological region of the identification target;
acquire second site information of the plurality of pieces of site information, wherein
the second site information is associated with the second biological region,
each piece of site information of the plurality of pieces of site information is associated with a respective piece of model data of a plurality of pieces of model data, and
each piece of site information of the plurality of pieces of site information is associated with a respective learner of a plurality of learners;
select, from the plurality of learners, a first learner corresponding to the acquired second site information; and
generate model data, of the plurality of pieces of model data, corresponding to the acquired second site information, wherein the model data is generated based on each of the selected first learner and the acquired second image data.

### Claim 2 (depends on 1)

The first information processing device according to claim 1, wherein the circuitry is further configured to output an identification result based on the execution of the identification process.

### Claim 3 (depends on 2)

The first information processing device according to claim 2, wherein the identification result includes at least one of a diagnostic result corresponding to the first biological region or an analysis result corresponding to the first biological region.

### Claim 4 (depends on 3)

The first information processing device according to claim 3, wherein
the diagnostic result includes information regarding whether a subject has a disease, and
the subject has a body including the first biological region.

### Claim 5 (depends on 3)

The first information processing device according to claim 3, wherein the analysis result includes information indicating at least one of:
one of presence of a lesion in the first biological region or absence of the lesion in the first biological region,
a probability that the first biological region includes the lesion,
a location of the lesion, or
a type of the lesion.

### Claim 6 (depends on 2)

The first information processing device according to claim 2, wherein the circuitry is further configured to output information regarding the selected first learner.

### Claim 7 (depends on 2)

The first information processing device according to claim 2, wherein the circuitry is further configured to:
acquire first staining information of a plurality of pieces of staining information;
select, from the plurality of identifiers, a second identifier corresponding to each of the first site information and the first staining information, wherein each identifier of the plurality of identifiers is associated with a respective combination of a plurality of combinations of the plurality of pieces of site information and the plurality of pieces of staining information; and
perform the identification process based on the second identifier and the first image data.

### Claim 8 (depends on 1)

The first information processing device according to claim 1, wherein the circuitry is further configured to:
acquire, from a reading device, scan data including the first image data; and
acquire the first image data based on the acquired scan data.

### Claim 9 (depends on 8)

The first information processing device according to claim 8, wherein
the acquired scan data further includes the first site information, and
the circuitry is further configured to recognize the first site information based on the acquired scan data.

### Claim 10 (depends on 8)

The first information processing device according to claim 8, wherein
the acquired scan data further includes first specimen information, and
the circuitry is further configured to:
recognize the first specimen information from the acquired scan data; and
acquire the first site information corresponding to the first specimen information from a second information processing device.

### Claim 11 (depends on 1)

The first information processing device according to claim 1, wherein implementation of the first identifier is based on at least one of a neural network, a random forest, a support vector machine, or AdaBoost.

### Claim 12 (depends on 1)

The first information processing device according to claim 1, wherein the first site information includes information indicating that the first biological region is classified into clinically meaningful units.

### Claim 13 (depends on 12)

The first information processing device according to claim 12, wherein each of the clinically meaningful units is at least one of an organ system, an organ, a tissue, or a disease code.

### Claim 14 (depends on 1)

The first information processing device according to claim 1, wherein the circuitry is further configured to:
acquire correct answer data corresponding to the second image data; and
generate the model data based on the correct answer data.

### Claim 15 (depends on 1)

The first information processing device according to claim 1, wherein the circuitry is further configured to:
acquire second staining information;
select, from the plurality of learners, a second learner corresponding to each of the second site information and the second staining information; and
generate the model data based on the second learner and the second image data.

### Claim 16 (depends on 1)

The first information processing device according to claim 1, wherein the circuitry is further configured to:
determine an existence of the model data corresponding to the second site information;
determine an accuracy of the model data based on the determination of the existence of the model data; and
generate new model data based on the accuracy of the model data.

### Claim 17

An information processing method, comprising:
acquiring, by circuitry, first image data of an identification target, wherein the first image data includes first biological region of the identification target;
acquiring, by the circuitry, first site information of a plurality of pieces of site information, wherein
the first site information is associated with the first biological region, and
each piece of site information of the plurality of pieces of site information is associated with a respective identifier of a plurality of identifiers;
selecting, by the circuitry, an identifier corresponding to the acquired first site information, wherein the identifier is selected from the plurality of identifiers;
executing, by the circuitry, an identification process based on each of the selected identifier and the acquired first image data;
acquiring, by the circuitry, second image data of the identification target, wherein the second image data includes a second biological region of the identification target;
acquiring, by the circuitry, second site information of the plurality of pieces of site information, wherein
the second site information is associated with the second biological region,
each piece of site information of the plurality of pieces of site information is associated with a respective piece of model data of a plurality of pieces of model data, and
each piece of site information of the plurality of pieces of site information is associated with a respective learner of a plurality of learners;
selecting, by the circuitry, a learner corresponding to the acquired second site information, wherein the learner is selected from the plurality of learners; and
generating, by the circuitry, model data, of the plurality of pieces of model data, corresponding to the acquired second site information, wherein the model data is generated based on each of the selected learner and the acquired second image data.

### Claim 18

An information processing system, comprising;
an information processing device that includes:
an image sensor configured to:
capture an image of an identification target; and
output first image data of the identification target based on the captured image; and
circuitry configured to:
acquire the first image data of the identification target, wherein the first image data includes a first biological region of the identification target;
acquire first site information of a plurality of pieces of site information, wherein
the first site information is associated with the first biological region, and
each piece of site information of the plurality of pieces of site information is associated with a respective identifier of a plurality of identifiers;
select, from the plurality of identifiers, an identifier corresponding to the acquired first site information;
execute an identification process based on each of the selected identifier and the acquired first image data;
acquire second image data of the identification target, wherein the second image data includes a second biological region of the identification target;
acquire second site information of the plurality of pieces of site information, wherein
the second site information is associated with the second biological region,
each piece of site information of the plurality of pieces of site information is associated with a respective piece of model data of a plurality of pieces of model data, and
each piece of site information of the plurality of pieces of site information is associated with a respective learner of a plurality of learners;
select, from the plurality of learners, a learner corresponding to the acquired second site information; and
generate model data, of the plurality of pieces of model data, corresponding to the acquired second site information, wherein the model data is generated based on each of the selected learner and the acquired second image data.

### Claim 19

An information processing system, comprising:
a medical image imaging device configured to:
capture an image of an identification target; and
output first image data of the identification target based on the captured image; and
a non-transitory computer-readable medium, having stored thereon, computer-executable instructions that, when executed by a computer, cause the computer to execute operations, the operations comprising:
acquiring the first image data of the identification target, wherein the first image data includes a first biological region of the identification target;
acquiring first site information of a plurality of pieces of site information, wherein
the first site information is associated with the first biological region, and
each piece of site information of the plurality of pieces of site information is associated with a respective identifier of a plurality of identifiers;
selecting, from the plurality of identifiers, an identifier corresponding to the acquired first site information;
executing an identification process based on each of the acquired first image data and the selected identifier;
acquiring second image data of the identification target, wherein the second image data includes a second biological region of the identification target;
acquiring second site information of the plurality of pieces of site information, wherein
the second site information is associated with the second biological region,
each piece of site information of the plurality of pieces of site information is associated with a respective piece of model data of a plurality of pieces of model data, and
each piece of site information of the plurality of pieces of site information is associated with a respective learner of a plurality of learners;
selecting, from the plurality of learners, a learner corresponding to the acquired second site information; and
generating model data, of the plurality of pieces of model data, corresponding to the acquired second site information, wherein the model data is generated based on each of the selected learner and the acquired second image data.