# US 2024/0389847 A1 · Image processing device, treatment system, and image processing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0389847 A1
- Document ID: US20240389847A1
- Status: Published application
- Kind: A1
- Application number: 18/790,340
- Filed: 2024-07-31
- Published: 2024-11-28
- Earliest claimed: 2022-03-04
- Assignees: Olympus Corporation
- Applicants: Olympus Corporation
- Inventors: Hiroshi Suzuki; Koichiro Watanabe
- Web page: https://patentgrep.com/patent/US20240389847A1

## CPC

- A61B 1/317: for bones or joints, e.g. osteoscopes or arthroscopes
- A61B 1/00006: {of control signals}
- A61B 1/000095: {for image enhancement}
- A61B 1/045: Control thereof
- A61B 17/320068: {using mechanical vibrations, e.g. ultrasonic (A61B17/22012 takes precedence; dental tooth drilling devices operated by vibration A61C3/03; removing intra-ocular material using mechanical vibrations A61F9/00745)}
- G06T 7/0012: {Biomedical image inspection}
- A61B 17/16: Instruments for performing osteoclasis; Drills or chisels for bones; Trepans {(arthroscopic bone cutters A61B17/320016)}
- A61B 2034/2048: {using an accelerometer or inertia sensor}
- A61B 2090/309: {using white LEDs}
- A61B 2090/3612: {with images taken automatically}
- A61B 2090/371: {with simultaneous use of two cameras}
- A61B 2217/005: with suction drainage system
- A61B 2217/007: with irrigation system
- G06T 2207/10024: Color image
- G06T 2207/10068: Endoscopic image
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image processing device includes a processor including hardware. The processor is configured to: acquire first image data partially including a region in which a living body is treated with at least an energy treatment instrument; detect a change in tone from at least a part of a region of a first image corresponding to the first image data to obtain a first detection result; perform tone correction on the first image based on the first detection result to generate first correction-image data; and generate a display image based on the first correction-image data.

## Claims

### Claim 1

An image processing device comprising a processor comprising hardware, the processor being configured to:
acquire first image data partially including a region in which a living body is treated with at least an energy treatment instrument;
detect a change in tone from at least a part of a region of a first image corresponding to the first image data to obtain a first detection result;
perform tone correction on the first image based on the first detection result to generate first correction-image data; and
generate a display image based on the first correction-image data.

### Claim 2 (depends on 1)

The image processing device according to claim 1, wherein
the processor is further configured to:
acquire second image data that has a different wavelength band from the first image, and that partially includes the region;
detect a change in tone from at least a part of a region of a second image corresponding to the second image data to obtain a second detection result; and
perform tone correction on the second image based on the second detection result to generate second correction-image data, and
the display image is generated based on at least one of the first correction-image data and the second correction-image data.

### Claim 3 (depends on 2)

The image processing device according to claim 2, wherein
the processor is further configured to combine the first correction-image data and the second correction-image data to generate composite image data, and
the display image is generated based on the composite image data.

### Claim 4 (depends on 2)

The image processing device according to claim 2, wherein
the processor is further configured to:
generate a first display image based on the first correction-image data,
generate a second display image based on the second correction-image data, and
output at least one of the first display image and the second display image to a display.

### Claim 5 (depends on 1)

The image processing device according to claim 1, wherein
the processor is further configured to generate the display image based on a synchronization signal that is synchronized with imaging drive of an imager.

### Claim 6 (depends on 1)

The image processing device according to claim 1, wherein
the processor is further configured to perform the tone correction on the first correction-image data based on a driving signal of the energy treatment instrument to treat the living body and the first detection result to generate the first correction-image data.

### Claim 7 (depends on 1)

The image processing device according to claim 1, wherein
the processor is further configured to detect a haze included in the first image as the change in tone.

### Claim 8 (depends on 7)

The image processing device according to claim 7, wherein
the processor is further configured to:
determine whether the detected haze is equal to or larger than a predetermined value;
generate the display image based on the first correction-image data when it is unit determined that the detected haze is equal to or larger than the predetermined value; and
generate the display image based on the first image data when it is determined that the detected haze is not equal to or larger than the predetermined value.

### Claim 9 (depends on 1)

The image processing device according to claim 1, wherein
the processor is further configured to:
estimate a haze component of each pixel in the first image;
generate a brightness histogram based on the first image and the estimated haze component;
calculate a representative brightness based on the generated brightness histogram;
calculate a correction coefficient to correct a contrast of the first image based on the estimated haze component and the calculated representative brightness; and
perform contrast correction of a reference pixel with respect to the first image based on the estimated haze component and the calculated correction coefficient to generate the first correction-image data.

### Claim 10 (depends on 2)

The image processing device according to claim 2, wherein
the processor is further configured to acquire image data generated by an imager that is capable of receiving invisible light at least including an infrared wavelength band as the second image data.

### Claim 11 (depends on 10)

The image processing device according to claim 10, wherein
the processor is further configured to perform edge enhancement processing with respect to the second image to generate the second correction-image data.

### Claim 12 (depends on 11)

The image processing device according to claim 11, wherein
the processor is further configured to:
estimate a haze component of each pixel in the first image;
generate a brightness histogram based on the first image and the estimated haze component;
calculate a representative brightness based on the generated brightness histogram;
calculate a correction coefficient to correct a contrast of the first image based on the estimated haze component and the calculated representative brightness; and
perform contrast correction of a reference pixel with respect to the first image based on the estimated haze component and the calculated correction coefficient to generate the first correction-image data.

### Claim 13 (depends on 12)

The image processing device according to claim 12, wherein
the processor is further configured to combine the first correction-image data and the second correction-image data to generate a composite image, and
the display image is generated based on the composite image data.

### Claim 14 (depends on 1)

The image processing device according to claim 1, wherein
the processor is further configured to acquire image data generated by an imager that is capable of receiving visible light having a wavelength band inside a visible spectrum and invisible light having a wavelength band outside the visible spectrum as the first image data.

### Claim 15 (depends on 2)

The image processing device according to claim 2, wherein
the processor is further configured to:
acquire image data generated by a first imager that is capable of receiving visible light having a wavelength band inside a visible spectrum as the first image data; and
acquire image data generated by a second imager that is capable of receiving invisible light having a wavelength band outside the visible spectrum as the second image data.

### Claim 16 (depends on 1)

The image processing device according to claim 1, wherein
the processor is further configured to:
acquire second image data having a different wavelength from the first image;
detect a change in tone from at least a part of a region of a second image corresponding to the second image data to obtain a second detection result; and
perform tone correction on the second image based on the second detection result to generate second correction-image data, and
the display image is generated based on at least one of the first correction-image data and the second correction-image data.

### Claim 17

An image processing device comprising a processor comprising hardware, the processor being configured to:
acquire a first image corresponding to first image data that includes a region in which a living body is treated with an energy treatment instrument;
acquire a second image corresponding to second image data having a different wavelength from the first image;
detect a change in tone from at least a part of a region of the first image to obtain a detection result;
perform tone correction on the second image based on the detection result to generate correction image data; and
generate a display image based on the correction image data.

### Claim 18

A treatment system comprising:
an energy treatment instrument that can be inserted into a subject, and that is capable of treating a treatment target site;
an endoscope that can be inserted into the subject, and that is capable of generating first image data by imaging at least the treatment target site; and
an image processing device that performs image processing with respect to the first image data to output the processed first image data to a display,
the image processing device comprising a processor comprising hardware, the processor being configured to:
acquire the first image data;
detect a change in tone from at least a part of a region of a first image corresponding to the first image data to obtain a first detection result;
perform tone correction on the first image based on the first detection result to generate first correction-image data; and
generate a display image based on the first correction-image data.

### Claim 19

An image processing method that is performed by an image processing device included in a processor comprising hardware, the method comprising:
acquiring first image data that includes a region in which a living body is treated with an energy treatment instrument;
detecting a change in tone from at least a part of a region of a first image corresponding to the first image data to obtain a first detection result;
performing tone correction on the first image based on the first detection result to generate first correction-image data; and
generating a display image based on the first correction-image data.
