# US 2023/0066301 A1 · Medical observation system, control device, and control method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0066301 A1
- Document ID: US20230066301A1
- Status: Published application
- Kind: A1
- Application number: 17/799,265
- Filed: 2020-12-04
- Published: 2023-03-02
- Earliest claimed: 2020-03-19
- Assignees: Sony Olympus Medical Solutions Inc.
- Applicants: Sony Olympus Medical Solutions Inc.
- Inventors: Hiroshi USHIRODA; Kazuhiro YAMAGUCHI
- Web page: https://patentgrep.com/patent/US20230066301A1

## CPC

- A61B 1/0655: {Control therefor}
- G02B 21/0012: {Surgical microscopes (counterbalanced structures for surgical microscopes G02B7/001)}
- A61B 1/043: {for fluorescence imaging}
- A61B 1/05: characterised by the image sensor, e.g. camera, being in the distal end portion
- A61B 1/0638: {providing two or more wavelengths}
- A61B 1/0646: {with illumination filters}
- G02B 21/06: Means for illuminating specimens
- G02B 21/361: {Optical details, e.g. image relay to the camera or image sensor (G02B21/364 takes precedence; illumination details G02B21/06 and subgroups)}
- G02B 23/2461: {Illumination}
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- A61B 1/045: Control thereof
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Provided are a medical observation system, a control device, and a control method that are configured to prevent flickering of a portion to be observed of an object to be observed and reduce the size of the device. The medical observation system 1 includes an image sensor 212, a second control unit 94 that causes a light source device 3 to simultaneously emit second visible light and excitation light, and an image processing unit 91 that generates a fluorescence image based on a first pixel value that is output from a pixel in which a first filter is arranged and that is contained in image data and a background image based on a second pixel value that is output from a pixel in which a second filter is arranged.

## Claims

### Claim 1

A medical observation system comprising:
a light source device configured to emit, to an object, first visible light and second visible light having different wavelength bands, and excitation light exciting a fluorescent substance and causing emission of fluorescence;
an image sensor including: a pixel portion including a plurality of pixels; a first filter configured to transmit the first visible light and the fluorescence; and a second filter configured to transmit the second visible light and the fluorescence, each of the first filter and the second filter being provided on a light receiving surface of each of the plurality of pixels, the image sensor being configured to capture at least one of reflected light of at least one of the first visible light and the second visible light reflected from the object and the fluorescence to generate image data;
a control unit configured to control the light source device to simultaneously emit the second visible light and the excitation light; and
an image processing unit configured to generate a fluorescence image based on a first pixel value included in the image data and output from a pixel in which the first filter is arranged, and a background image based on a second pixel value output from a pixel in which the second filter is arranged.

### Claim 2 (depends on 1)

The medical observation system according to claim 1, comprising a cut filter provided on an incident side of the image sensor and configured to transmit the reflected light and the fluorescence while blocking the excitation light.

### Claim 3 (depends on 1)

The medical observation system according to claim 1, wherein the image processing unit is configured to
change the second pixel value based on the first pixel value, and
generate the background image based on the changed second pixel value.

### Claim 4 (depends on 3)

The medical observation system according to claim 3, wherein the image processing unit is configured to
subtract the first pixel value from the second pixel value, and
generate the background image based on a result of subtraction.

### Claim 5 (depends on 1)

The medical observation system according to claim 1, wherein the image processing unit is configured to
subtract, from the second pixel value, a multiplication result obtained by multiplying a value obtained by dividing a spectral sensitivity of the second filter to a fluorescence wavelength by a spectral sensitivity of the first filter to the fluorescence wavelength by the first pixel value, and
generate the background image based on a result of subtraction.

### Claim 6 (depends on 1)

The medical observation system according to claim 1, wherein the control unit is configured to
control, in a first observation mode, the light source device to emit the first visible light and the second visible light, and
control, in a second observation mode, the light source device to emit the second visible light and the excitation light.

### Claim 7 (depends on 1)

The medical observation system according to claim 1, wherein
the light source device is configured to emit third visible light having a wavelength band different from those of the first visible light and the second visible light,
the image sensor includes a third filter configured to transmit the third visible light and the fluorescence, and
the image sensor is configured to generate the background image based on a third pixel value output from a pixel in which the third filter is arranged and the second pixel value, the third pixel value and the second pixel value being included in image data.

### Claim 8 (depends on 7)

The medical observation system according to claim 7, wherein
the first visible light is light in a red wavelength band,
the second visible light is one of light in a green wavelength band and light in a blue wavelength band,
the third visible light is the other of the light in a green wavelength band and the light in a blue wavelength band,
the first filter is a red filter configured to transmit the light in a red wavelength band and the fluorescence,
the second filter is one of a green filter and a blue filter configured to transmit one of the light in a green wavelength band and the light in a blue wavelength band, and further to transmit the fluorescence, and
the third filter is the other of the green filter and the blue filter configured to transmit other of the light in a green wavelength band and the light in a blue wavelength band, and further to transmit the fluorescence.

### Claim 9 (depends on 8)

The medical observation system according to claim 8, wherein
the light source device includes:
a first light source unit configured to emit the light in a red wavelength band;
a second light source unit configured to emit the light in a green wavelength band;
a third light source unit configured to emit the light in a blue wavelength band; and
a fourth light source unit configured to emit the excitation light, and
the control unit is configured to
control the first light source unit, the second light source unit, and the third light source unit to emit light, in a white light observation mode for observation with white light, and
control the second light source unit, the third light source unit, and the fourth light source unit to emit light, in a fluorescence observation mode for observation of the fluorescence.

### Claim 10 (depends on 1)

The medical observation system according to claim 1, wherein the image processing unit is configured to generate a composite image in which the background image and the fluorescence image are combined.

### Claim 11 (depends on 1)

The medical observation system according to claim 1, wherein the image processing unit is configured to perform grayscale processing on at least one of the background image and the fluorescence image.

### Claim 12 (depends on 1)

The medical observation system according to claim 1, wherein the image processing unit is configured to perform colorization on at least one of the background image and the fluorescence image.

### Claim 13 (depends on 1)

The medical observation system according to claim 1, wherein
the fluorescent substance is indocyanine green, and
the excitation light has a center wavelength of 740 nm.

### Claim 14

A medical observation system comprising:
a light source device configured to emit, to an object, first visible light and second visible light having different wavelength bands, and excitation light exciting a fluorescent substance and causing emission of fluorescence;
a dichroic prism configured to split reflected light of at least one of the first visible light and the second visible light reflected from the object and the fluorescence into a plurality of wavelength bands;
a plurality of image sensors configured to receive light beams of the plurality of wavelength bands split by the dichroic prism and generate a plurality of pieces of image data;
a control unit configured to control the light source device to simultaneously emit the second visible light and the excitation light; and
an image processing unit configured to generate a background image and a fluorescence image based on the plurality of pieces of image data.

### Claim 15 (depends on 1)

The medical observation system according to claim 1 further comprising:
a support unit configured to turnably support a medical imaging device including the image sensor; and
a base portion configured to turnably hold a base end portion of the support unit, the base portion being movable on a floor surface.

### Claim 16 (depends on 1)

The medical observation system according to claim 1 further comprising
an insertion section configured to be insertable into a subject and including an optical system focusing the reflected light and the fluorescence to form an object image on a light receiving surface of the image sensor.

### Claim 17 (depends on 16)

The medical observation system according to claim 16, wherein the insertion section is detachable from a medical imaging device including the image sensor.

### Claim 18

A control device for controlling a light source device and a medical imaging device,
the light source device being configured to emit, to an object, first visible light and second visible light having different wavelength bands, and excitation light exciting a fluorescent substance and causing emission of fluorescence,
the medical imaging device including a pixel portion having a plurality of pixels, a first filter configured to transmit the first visible light and the fluorescence, and a second filter configured to transmit the second visible light and the fluorescence, each of the first filter and the second filter being provided on a light receiving surface of each of the plurality of pixels, the medical imaging device being configured to capture at least one of reflected light of at least one of the first visible light and the second visible light reflected from the object and the fluorescence to generate image data, the control device comprising:
a control unit configured to control the light source device to simultaneously emit the second visible light and the excitation light; and
an image processing unit configured to generate a fluorescence image based on a first pixel value included in the image data and output from a pixel in which the first filter is arranged, and a background image based on a second pixel value output from a pixel in which the second filter is arranged.

### Claim 19

A control method executed by a control device for controlling a light source device and a medical imaging device,
the light source device being configured to emit, to an object, first visible light and second visible light having different wavelength bands, and excitation light exciting a fluorescent substance and causing emission of fluorescence,
the medical imaging device including a pixel portion having a plurality of pixels, a first filter configured to transmit the first visible light and the fluorescence, and a second filter configured to transmit the second visible light and the fluorescence, each of the first filter and the second filter being provided on a light receiving surface of each of the plurality of pixels, the medical imaging device being configured to capture at least one of reflected light of at least one of the first visible light and the second visible light reflected from the object and the fluorescence to generate image data, the control method comprising:
controlling the light source device to simultaneously emit the second visible light and the excitation light; and
generating a fluorescence image based on a first pixel value included in the image data and output from a pixel in which the first filter is arranged, and a background image based on a second pixel value output from a pixel in which the second filter is arranged.
