# US 10,371,933 B2 · Medical image processing apparatus, medical image processing method, and medical observation system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,371,933 B2 - Document ID: US10371933B2 - Status: Granted - Kind: B2 - Application number: 15/559,602 - Filed: 2016-06-03 - Published: 2018-03-08 - Granted: 2019-08-06 - Earliest claimed: 2015-06-19 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Yuki Sugie; Hisakazu Shiraki - Web page: https://patentgrep.com/patent/US10371933B2 ## CPC - G02B 21/367: {providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison} - A61B 1/000094: {extracting biological structures} - A61B 1/000095: {for image enhancement} - G02B 27/0075: {with means for altering, e.g. increasing, the depth of field or depth of focus} - H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes} - H04N 23/63: by using electronic viewfinders - H04N 23/67: Focus control based on electronic image sensor signals - H04N 23/673: {based on contrast or high frequency components of image signals, e.g. hill climbing method} - H04N 23/675: {comprising setting of focusing regions} - H04N 23/676: {Bracketing for image capture at varying focusing conditions} - H04N 23/6811: {based on the image signal} - H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory} - H04N 23/743: Bracketing, i.e. taking a series of images with varying exposure conditions - H04N 23/959: by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics - A61B 1/00188: {with focusing or zooming features} - A61B 1/04: combined with photographic or television appliances - A61B 1/0684: {using light emitting diodes [LED]} - G02B 21/0012: {Surgical microscopes (counterbalanced structures for surgical microscopes G02B7/001)} - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A medical observation system including a medical imaging device that captures a plurality of images of a living body while changing a focus position, and circuitry that generates a composite image by compositing the plurality of images captured by the medical imaging device, and switches output between the generated composite image and one of the plurality of images based on a result of analysis performed on at least one of the plurality of images. ## Claims ### Claim 1 A medical image processing apparatus comprising: circuitry configured to generate a composite image by compositing a plurality of images obtained by capturing with a medical imaging device an object of a living body while changing a focus position, select, based on a result of analysis performed on at least one of the plurality of images, an image of the plurality of images wherein the object of the living body at a center of the image is in focus, and switch output between the generated composite image and the selected image of the plurality of images based on the result of analysis performed on the at least one of the plurality of images. ### Claim 2 (depends on 1) The medical image processing apparatus according to claim 1, wherein the circuitry is further configured to switch output between the generated composite image and the selected image of the plurality of images based on a manual input operation. ### Claim 3 (depends on 1) The medical image processing apparatus according to claim 1, wherein the analysis performed on the at least one of the plurality of images includes analysis of an angle of view adjustment. ### Claim 4 (depends on 1) The medical image processing apparatus according to claim 1, wherein the analysis performed on the at least one of the plurality of images includes motion detection. ### Claim 5 (depends on 4) The medical image processing apparatus according to claim 4, wherein the analysis performed on the at least one of the plurality of images includes image recognition that recognizes a treatment tool in the at least one of the plurality of images. ### Claim 6 (depends on 1) The medical image processing apparatus according to claim 1, wherein the circuitry is configured to switch the output between the generated composite image and the selected image of the plurality of images based on the result of the analysis being lower than or equal to a predetermined threshold. ### Claim 7 (depends on 2) The medical image processing apparatus according to claim 2, wherein the manual input operation includes an input via an interface on a treatment tool. ### Claim 8 (depends on 1) The medical image processing apparatus according to claim 1, wherein the circuitry is configured to generate the composite image by selecting a pixel of a previous composite image or a pixel of a newly captured image that is in focus and compositing the previous composite image and the newly captured image. ### Claim 9 (depends on 1) The medical image processing apparatus according to claim 1, wherein the medical imaging device is configured to capture the plurality of images of the object of the living body at an imaging frame rate, and the circuitry is configured to output the generated composite image or the selected image of the plurality of images at a display frame rate, the display frame rate being equal to the imaging frame rate. ### Claim 10 (depends on 1) The medical image processing apparatus according to claim 1, wherein the circuitry is configured to output the generated composite image or the selected image of the plurality of images at a display frame rate, the display frame rate being at least 120 Hz. ### Claim 11 (depends on 1) The medical image processing apparatus according to claim 1, wherein the circuitry is further configured to perform alignment between the generated composite image and a newly captured image, and update the generated composite image by compositing the aligned generated composite image and the newly captured image. ### Claim 12 (depends on 11) The medical image processing apparatus according to claim 11, wherein the circuitry is configured to adjust an angle of view of each of the generated composite image and the newly captured image and perform alignment between the generated composite image and the newly captured image after adjusting the angle of view. ### Claim 13 (depends on 3) The medical image processing apparatus according to claim 3, wherein the circuitry is configured to switch output to the selected image of the plurality of images when the analysis determines that a reliability of adjustment of the angle of view performed on each of the generated composite image and a newly captured image is lower than or equal to a threshold. ### Claim 14 (depends on 11) The medical image processing apparatus according to claim 11, wherein the circuitry is configured to detect motion in each of the generated composite image and the newly captured image and perform the alignment on the basis of a result of the motion detection. ### Claim 15 (depends on 4) The medical image processing apparatus according to claim 4, wherein the circuitry is configured to switch output to the selected image of the plurality of images when the analysis determines that a reliability of the motion detection is lower than or equal to a threshold. ### Claim 16 (depends on 1) The medical image processing apparatus according to claim 1, wherein the plurality of images is obtained by capturing with the medical imaging device the object of the living body while changing the focus position within a range determined based on depth information of the object of the living body. ### Claim 17 (depends on 1) The medical image processing apparatus according to claim 1, wherein the circuitry is configured to generate the composite image by compositing the plurality of images obtained at each focus position in a predetermined range including, a peak position that is a focus position at which a peak of a score, used in Auto Focus (AF), is obtained. ### Claim 18 (depends on 1) The medical image processing apparatus according to claim 1, wherein the plurality of images includes a left eye image and a right eye image. ### Claim 19 A medical image processing method comprising: generating a composite image by compositing a plurality of images obtained by capturing with a medical imaging device an object of a living body while changing a focus position; selecting, based on a result of analysis performed on at least one of the plurality of images, an image from the plurality of images wherein the object of the living body at a center of the image is in focus; and switching output between the generated composite image and the selected image of the plurality of images based on the result of analysis performed on the at least one of the plurality of images. ### Claim 20 A medical observation system comprising: a medical imaging device configured to capture a plurality of images of an object of a living body while changing a focus position; and circuitry configured to generate a composite image by compositing the plurality of images captured by the medical imaging device, select, based on a result of analysis performed on at least one of the plurality of images, an image from the plurality of images wherein the object of the living body at a center of the image is in focus, and switch output between the generated composite image and the selected image of the plurality of images based on the result of analysis performed on the at least one of the plurality of images. ### Claim 21 (depends on 20) The medical image processing apparatus according to claim 20, wherein the medical imaging device is a surgical video microscope. ### Claim 22 (depends on 20) The medical image processing apparatus according to claim 20, wherein the medical imaging device is an endoscope. ### Claim 23 (depends on 20) The medical image processing apparatus according to claim 20, wherein the medical imaging device is configured to capture the plurality of images of the object of the living body during a medical procedure.