# US 9,517,109 B2 · Medical system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,517,109 B2
- Document ID: US9517109B2
- Status: Granted
- Kind: B2
- Application number: 15/202,967
- Filed: 2016-07-06
- Published: 2016-10-27
- Granted: 2016-12-13
- Earliest claimed: 2014-09-24
- Assignees: Olympus Corporation
- Applicants: Olympus Corporation
- Inventors: Yorito Maeda; Kuniaki Kami; Koichi Tashiro
- Web page: https://patentgrep.com/patent/US9517109B2

## CPC

- A61B 90/37: {Surgical systems with images on a monitor during operation}
- G02B 27/01: Head-up displays
- G02B 27/0103: {comprising holographic elements}
- G02B 30/56: by projecting aerial or floating images
- G03H 1/0248: {Volume holograms}
- G03H 1/08: Synthesising holograms, {i.e. holograms synthesized from objects or objects from holograms}(using electric digital computers G06F; G06T)
- G03H 1/0866: {Digital holographic imaging, i.e. synthesizing holobjects from holograms}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06T 7/0012: {Biomedical image inspection}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 40/107: {Static hand or arm}
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- H04N 7/183: {for receiving images from a single remote source}
- A61B 1/00: Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes (examination of body cavities or body tracts using ultrasonic, sonic or infrasonic waves A61B8/12; endoscopic instruments for taking cell samples or for biopsy A61B10/04); Illuminating arrangements therefor (for the eyes A61B3/00)
- A61B 34/25: {User interfaces for surgical systems}
- A61B 2017/00207: {with hand gesture control or hand gesture recognition}
- A61B 2090/373: {using light, e.g. by using optical scanners}
- G02B 2027/0154: {with movable elements}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G03H 2001/0038: {analogue or digital holobjects}
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- G06T 2207/30004: Biomedical image processing
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G03H: HOLOGRAPHIC PROCESSES OR APPARATUS (holograms, e.g. point holograms, used as ordinary optical elements G02B5/32; producing stereoscopic or other three-dimensional effects G02B30/00; diffraction-grating systems G02B27/44; systems using moiré fringes G02B27/60; optical logic elements G02F3/00; stereo-photography G03B35/00; photosensitive materials or processes for photographic purposes G03C; {stereo-photographic or similar processes G03C9/00}; apparatus for processing exposed photographic materials G03D; analogue computers performing mathematical operations with the aid of optical elements G06E3/00; authentication by radiation, of concealed information carried by holograms or diffraction gratings G06K19/16; holographic storage G11B7/0065, G11C13/04; {stereoscopic or other three dimensional effects in television systems H04N13/00})
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A medical system that is capable of displaying a virtual image, comprising: a sensor comprising hardware, wherein the sensor implements: a first detection section that detects information related to at least one of a position and an orientation of an operator; a second detection section that detects a position of an object in a surgery room; and a processor comprising hardware, wherein the processor implements: a calculation section that calculates an area where the virtual image is to be arranged in the air based on the information; a determination section that determines whether or not the object is present in the area based on a result of the detection by the second detection section; and a control section that causes the virtual image to be arranged in the area based on a determination result by the determination section.

## Claims

### Claim 1

A medical system that is capable of displaying a virtual image, comprising:
a sensor comprising hardware, wherein the sensor implements:
a first detection section that detects information related to at least one of a position and an orientation of an operator;
a second detection section that detects a position of an object in a surgery room; and
a processor comprising hardware, wherein the processor implements:
a calculation section that calculates an area where the virtual image is to be arranged in the air based on the information;
a determination section that determines whether or not the object is present in the area based on a result of the detection by the second detection section; and
a control section that causes the virtual image to be arranged in the area based on a determination result by the determination section.

### Claim 2 (depends on 1)

The medical system according to claim 1, wherein
the second detection section detects the object present in the area, and
the control section arranges the virtual image in the area when the determination section determines that the object is not present in the area.

### Claim 3 (depends on 1)

The medical system according to claim 1, wherein the information includes at least one of a standing position of the operator, a position of a face of the operator and an orientation of the face of the operator.

### Claim 4 (depends on 1)

The medical system according to claim 1, further comprising,
a third detection section that detects positions of a hand and fingers of the operator,
wherein the virtual image includes an operation screen, and
the control section adjusts a position and an orientation of the virtual image such that the positions of the hand and the fingers match with the operation screen.

### Claim 5 (depends on 4)

The medical system according to claim 4, wherein
the determination section determines whether or not the operator is within a visual field range of the virtual image based on the information,
the control section enables an operation on the operation screen when the determination section determines that the operator is within the visual field range of the virtual image, and
the control section disables the operation on the operation screen when the determination section determines that the operator is outside the visual field range of the virtual image.

### Claim 6 (depends on 5)

The medical system according to claim 5, further comprising,
a display section that displays information related to a surgery,
wherein the control section causes a notice for guiding the operator into the visual field range of the virtual image to be displayed on the display section, when the determination section determines that the operator is outside the visual field range of the virtual image.

### Claim 7 (depends on 4)

The medical system according to claim 4, wherein the first detection section, the second detection section, and the third detection section acquire the information, the position of the object, and the positions of the hand and fingers, respectively, based on an output from a gesture sensor.

### Claim 8 (depends on 1)

The medical system according to claim 1, further comprising,
a monitor, and a plate for changing a traveling direction of a light beam,
wherein the virtual image is displayed by image-forming the light beam emitted from the display surface of the monitor in a position symmetrical to a display surface of the monitor, with the plate as a plane of symmetry.

### Claim 9 (depends on 8)

The medical system according to claim 8, further comprising,
an arrangement section that arranges the monitor in a position so as to enable the virtual image to be displayed, based on an instruction from the control section.

### Claim 10 (depends on 9)

The medical system according to claim 9, wherein the arrangement section supports the monitor from a ceiling or a wall in a room such that the monitor is movable.