# US 11,451,698 B2 · Medical system and control unit > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,451,698 B2 - Document ID: US11451698B2 - Status: Granted - Kind: B2 - Application number: 16/617,528 - Filed: 2018-03-28 - Published: 2020-04-23 - Granted: 2022-09-20 - Earliest claimed: 2017-06-05 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Daisuke Kikuchi; Yuki Sugie; Yukihiro Nakamura; Kentaro Fukazawa; Kenji Ikeda - Web page: https://patentgrep.com/patent/US11451698B2 ## CPC - A61B 1/000095: {for image enhancement} - A61B 1/00006: {of control signals} - A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled} - A61B 1/045: Control thereof - A61B 90/20: Surgical microscopes characterised by non-optical aspects - G02B 23/26: using light guides {(for illumination G02B23/2469)} - H04N 23/66: Remote control of cameras or camera parts, e.g. by remote control devices - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - G02B 21/0012: {Surgical microscopes (counterbalanced structures for surgical microscopes G02B7/001)} - G02B 21/367: {providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison} - H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract In a case where a plurality of imaging apparatuses is used for a surgery, the appearance is regulated between images captured by the respective imaging apparatuses. According to the present disclosure, a medical system is provided, the medical system including a plurality of surgical imaging apparatuses and a control unit to which each of the surgical imaging apparatuses is connected, the control unit including a signal processing unit that links images captured by the respective surgical imaging apparatuses. With this configuration, in a case where a plurality of imaging apparatuses is arranged for a surgery, it becomes possible to regulate the appearance between images captured by the respective imaging apparatuses. ## Claims ### Claim 1 A medical system comprising: a plurality of surgical imaging apparatuses each having a different angle of view, the plurality of the surgical imaging apparatuses being physically separate apparatuses respectively; and processing circuitry, to which each of the surgical imaging apparatuses is coupled, configured to determine whether the plurality of the surgical imaging apparatuses image a same subject or different subjects concurrently, and link images captured by respective ones of the plurality of the surgical imaging apparatuses in response to determining that the plurality of the surgical imaging apparatuses image the same subject concurrently, and not link the images captured by the respective ones of the plurality of the surgical imaging apparatuses in response to determining that the plurality of the surgical imaging apparatuses image the different subjects concurrently, wherein the processing circuitry is further configured to, in response to determining that the plurality of the surgical imaging apparatuses image the same subject concurrently, perform a process of adapting a shake amount between the images captured by the respective ones of the plurality of the surgical imaging apparatuses, by tracking a first image of the images with a shake in accordance with a second image of the images without a shake or with a shake less than the shake of the first image, and correcting the shake in the first image by removing a shake component in the first image in accordance with the second image, and the first image is captured by a first surgical imaging apparatus of the plurality of the surgical imaging apparatuses having a first angle of view, and the second image is captured by a second surgical imaging apparatus of the plurality of the surgical imagine apparatuses having a second angle of view different from the first angle of view. ### Claim 2 (depends on 1) The medical system according to claim 1, wherein the plurality of the surgical imaging apparatuses include at least two different types of surgical imaging apparatuses of an endoscope, an extracorporeal scope, a microscope, or a surgical field camera. ### Claim 3 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to switch whether or not to perform the linking, according to occurrence of an adjustment trigger. ### Claim 4 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to perform a process for the linking according to occurrence of an adjustment trigger, and the adjustment trigger occurs by a user operation. ### Claim 5 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to: obtain information regarding a state of an operating surgeon who performs a surgery; and perform a process for the linking based on the obtained information. ### Claim 6 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to: obtain identification information for identifying the plurality of the surgical imaging apparatuses; and perform a process for the linking based on the obtained identification information. ### Claim 7 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to perform a process of adapting colors between the images captured by the respective ones of the plurality of the surgical imaging apparatuses. ### Claim 8 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to perform a process of adapting brightness between the images captured by the respective ones of the plurality of the surgical imaging apparatuses. ### Claim 9 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to perform a process of adapting contrast between the images captured by the respective ones of the plurality of the surgical imaging apparatuses. ### Claim 10 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to perform a process of adapting resolution between the images captured by the respective ones of the plurality of the surgical imaging apparatuses. ### Claim 11 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to perform a process of adapting noise between the images captured by the respective ones of the plurality of the surgical imaging apparatuses. ### Claim 12 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to perform a process of adapting a depth of field between the images captured by the respective ones of the plurality of the surgical imaging apparatuses. ### Claim 13 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to perform a process of adapting a depth between stereoscopic images captured by the respective ones of the plurality of the surgical imaging apparatuses. ### Claim 14 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to perform a process of adapting an angle of view between the images captured by the respective ones of the plurality of the surgical imaging apparatuses. ### Claim 15 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to link an image captured by one of the plurality of the surgical imaging apparatuses with an image captured by another one of the plurality of the surgical imaging apparatuses as a reference. ### Claim 16 (depends on 1) The medical system according to claim 1, wherein the processing circuitry is configured to link images captured by the plurality of the surgical imaging apparatuses with an arbitrary desired image as a reference. ### Claim 17 A control unit comprising: processing circuitry, to which each of a plurality of surgical imaging apparatuses is coupled, configured to determine whether the plurality of the surgical imaging apparatuses image a same subject or different subjects concurrently, link images captured by respective ones of the plurality of the surgical imaging apparatuses in response to determining that the plurality of the surgical imaging apparatuses image the same subject concurrently, and not link the images captured by the respective ones of the plurality of the surgical imaging apparatuses in response to determining that the plurality of the surgical imaging apparatuses image the different subjects concurrently, wherein each of the plurality of the surgical imaging apparatuses has a different angle of view, the plurality of the surgical imaging apparatuses being physically separate apparatuses respectively, the processing circuitry is further configured to, in response to determining that the plurality of the surgical imaging apparatuses image the same subject concurrently, perform a process of adapting a shake amount between the images captured by the respective ones of the plurality of the surgical imaging apparatuses, by tracking a first image of the images with a shake in accordance with a second image of the images without a shake or with a shake less than the shake of the first image, and correcting the shake in the first image by removing a shake component in the first image in accordance with the second image, and the first image is captured by a first surgical imaging apparatus of the plurality of the surgical imaging apparatuses having a first angle of view, and the second image is captured by a second surgical imaging apparatus of the plurality of the surgical imaging apparatuses having a second angle of view different from the first angle of view. ### Claim 18 A medical system comprising: a plurality of surgical imaging apparatuses each having a different angle of view, the plurality of the surgical imaging apparatuses being physically separate apparatuses respectively; a plurality of control units, each of the plurality of the surgical imaging apparatuses being coupled to one of the plurality of the control units; and an integrated apparatus to which each of the plurality of the control units is coupled, the integrated apparatus including processing circuitry configured to determine whether the plurality of the surgical imaging apparatuses image a same subject or different subjects concurrently, and link images captured by respective ones of the plurality of the surgical imaging apparatuses in response to determining that the plurality of the surgical imaging apparatuses image the same subject concurrently, and not link the images captured by the respective ones of the plurality of the surgical imaging apparatuses in response to determining that the plurality of the surgical imaging apparatuses image the different subjects concurrently, wherein the processing circuitry is further configured to, in response to determining that the plurality of the surgical imaging apparatuses image the same subject concurrently, perform a process of adapting a shake amount between the images captured by the respective ones of the plurality of the surgical imaging apparatuses, by tracking a first image of the images with a shake in accordance with a second image of the images without a shake or with a shake less than the shake of the first image, and correcting the shake in the first image by removing a shake component in the first image in accordance with the second image, and the first image is captured by a first surgical imaging, apparatus of the plurality of the surgical imaging, apparatuses having a first angle of view, and the second image is captured by a second surgical imaging apparatus of the plurality of the surgical imaging apparatuses having a second angle of view different from the first angle of view. ### Claim 19 (depends on 1) The medical system according to claim 1, wherein the first image is an endoscopic image captured by an endoscope, and the second image is an stationary exoscopic image captured by an exoscope.