# US 11,998,400 B2 · System and method for enhanced data analysis with specialized video enabled software tools for medical environments > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,998,400 B2 - Document ID: US11998400B2 - Status: Granted - Kind: B2 - Application number: 18/097,203 - Filed: 2023-01-14 - Published: 2023-10-05 - Granted: 2024-06-04 - Earliest claimed: 2009-05-29 - Applicants: Jack Wade - Inventors: Jack Wade - Web page: https://patentgrep.com/patent/US11998400B2 ## CPC - A61B 90/361: {Image-producing devices, e.g. surgical cameras} - A61B 1/00009: {of image signals during a use of endoscope} - A61B 1/000095: {for image enhancement} - A61B 1/00011: {characterised by signal transmission} - A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled} - A61B 1/3132: {for laparoscopy} - A61B 5/0084: {for introduction into the body, e.g. by catheters (A61B5/1459 takes precedence)} - A61B 5/0095: {by applying light and detecting acoustic waves, i.e. photoacoustic measurements} - A61B 5/1032: {Determining colour of tissue for diagnostic purposes} - A61B 34/25: {User interfaces for surgical systems} - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G16H 10/00: ICT specially adapted for the handling or processing of patient-related medical or healthcare data (for medical reports G16H15/00; for therapies or health-improving plans G16H20/00; for the handling or processing of medical images G16H30/00) - G16H 10/60: for patient-specific data, e.g. for electronic patient records - G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture - G16H 30/40: for processing medical images, e.g. editing - G16H 40/63: for local operation - G16H 40/67: for remote operation - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients - G16H 80/00: ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring - H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)} - A61B 5/0075: {by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy (A61B5/0071 takes precedence)} - A61B 5/024: Measuring pulse rate or heart rate - A61B 90/30: Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure - A61B 2017/00203: {with speech control or speech recognition} - A61B 2090/3612: {with images taken automatically} - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/30004: Biomedical image processing - H04L 67/125: involving control of end-device applications over a network - H04N 7/183: {for receiving images from a single remote source} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Medical software tools platforms utilize a surgical display to provide access to specific medical software tools, such as medically-oriented applications or widgets, that can assist surgeons or surgical team in performing various procedures. In particular, an endoscopic camera may register the momentary rise in the optical signature reflected from a tissue surface and in turn transmit it to a medical image processing system which can also receive patient heart rate data and display relevant anomalies. Changes in various spectral components and the speed at which they change in relation to a source of stimulus (heartbeat, breathing, light source modulation, etc.) may indicate the arrival of blood, contrast agents or oxygen absorption. Combinations of these may indicate various states of differing disease or margins of tumors, and so forth. Also, changes in temperatures, physical dimensions, pressures, photoacoustic pressures and the rate of change may indicate tissue anomalies in comparison to historic values. ## Claims ### Claim 1 A system for medical software tools, comprising: an image stream interface module configured to receive an image stream from a surgical camera via an image stream processing system; a user interface overlay module configured to provide a user interface overlay adapted for presentation over the image stream via an image stream interface module; an optical sensor corresponding with said surgical camera configured to register changes in spectral characteristics reflected from a tissue surface under inspection via a lens of said surgical camera; a medical software tools module configured to provide a medical software tool through the user interface, the medical software tool being configured to perform an operation with respect to the image stream and provide an output adapted to be presented over the image stream via a plurality of image processing modules; a medical image processing system for processing patient medical data and corresponding said patient medical data with momentary changes in spectral characteristics for generating optical signature data indicative of various patient conditions, wherein the medical software tool measures: changes in color intensity; rates at which said color intensities change in response to heartbeat pushed blood, breathing pushed oxygen or light intensity or modulation from a light source; said momentary changes in spectral characteristics and rates of changes are analyzed using a database of historical samples to delineate tissue areas of interest; and timers set by a user to facilitate a pace at which a medical procedure is performed. ### Claim 2 (depends on 1) The image stream interface of claim 1, wherein said image stream interface provides a stream of video footage or static images. ### Claim 3 (depends on 1) The system of claim 1, wherein said optical sensor is located within an endoscopic lens. ### Claim 4 (depends on 1) The optical sensor according to claim 1, further comprising of a heart rate monitoring device coupled to said optical sensor to transmit heart rate data. ### Claim 5 (depends on 4) The optical sensor according to claim 4, wherein said optical sensor provides an optical signature analysis engine and user interface. ### Claim 6 (depends on 1) The medical image processing system of claim 1, wherein said medical image processing system processes and transmits image data from said optical sensor and heart rate data to said user interface. ### Claim 7 (depends on 1) The medical image processing system according to claim 1, further comprising of a margin guide tool for margin guide outlines to mark unique identifiable tissue structures. ### Claim 8 (depends on 1) The medical software tools module according to claim 1, wherein said modules include a medical device control module, an image similarity search module, an image stream processing control module, a measurement module, an image stream tagging and tracking module, a stereoscopic image stream module, an optical signature module, a timer module, an image enhancement module, an embedded object tracking module, a grid overlay module, and a checklist module. ### Claim 9 A method for an image stream interface module configured to receive an image stream from a surgical camera, the method comprising; enclosing an optical sensor, an image processing unit, a display unit, a display interface, a power unit, a user interface, and a wireless communication unit within a sterile environment; overlaying an image stream on a user interface module for presentation; registering changes in spectral characteristics reflected from a tissue surface under inspection from said optical sensor located in a corresponding position to a surgical camera; documenting changes in said spectral characteristics reflected from a tissue surface under inspection from said optical sensor located in said corresponding position to said surgical camera; overlaying a plurality of medical software tools through said user interface; performing an operation captured using said image stream and said optical sensor; providing an output of said operation in a form of said image stream with said overlay interface; processing patient medical data corresponding to said patient medical data using a medical image processing system that documents momentary changes in said spectral characteristics; generating optical signature data indicative of various patient conditions, wherein at least one of the said medical software tool measures: changes in color intensity; rates at which said color intensities change in response to heartbeat pushed blood, breathing pushed oxygen or light intensity or modulation from a light source; analyzing said optical signature data for potential anomalies and diseased areas by cross-referencing a database of historical samples; and setting timers to facilitate a pace at which a medical procedure is performed. ### Claim 10 (depends on 9) The image processing unit according to claim 9, wherein said image processing unit is a module with a field programmable gate array that utilizes image processing functions. ### Claim 11 (depends on 9) The image processing unit according to claim 9, wherein a processed image and video stream is enhanced for clarity. ### Claim 12 (depends on 10) The image processing unit according to claim 10, wherein unprocessed image and video stream only processes and enhances certain marked parts of an image and video stream. ### Claim 13 (depends on 9) The optical signature data according to claim 9, wherein said optical signature data is analyzed by user input devices, including a visual display, keyboard, touchscreen or voice activated device. ### Claim 14 (depends on 13) The voice activated device according to claim 13, wherein said voice activated device is used to control said visual display. ### Claim 15 (depends on 13) The visual display according to claim 13, wherein said visual display includes a color map tool in a presentation format with color translation tables. ### Claim 16 An image stream processing system for use in a sterile and medical environment, the system comprising: image data, collected by way of an imaging device, configured to receive video and image data captured by a surgical camera during a surgical procedure; a camera head video pre-processing, wherein said camera head and video pre-processing is unit communicatively coupled to a video input module, configured to enhance the received video data in real-time; a module for image registration and segmentation to optimize the video data and communicate via a system bus with a central processing unit; a video transport unit for image post-processing, configured to format an initial sensor resolution to a display resolution; an input module, configured to receive input commands from surgical and medical professionals; wherein said input commands are image markers used to delineate tissue areas of interest; a real-time video enhancement module with a plurality of editing functionalities, enabling a medical professional to apply one or more editing operations to an anomalous segment or area of interest; a digital zoom feature activated by delineated tissue areas of interest used to indicate an area wherein said image stream is processed digitally for presentation of a zoomed in image for annotation by a user; and wherein a visual display presents annotation data input by said medical professional. ### Claim 17 (depends on 16) The image data of claim 16, wherein raw video and image data contains both pre-processed and processed image and video data from an optical sensor. ### Claim 18 (depends on 16) The video transport unit of claim 16, further comprising of a conversion module for video cross conversion, scaling, and graphic overlays. ### Claim 19 (depends on 16) The digital video enhancement functionality in claim 16, wherein the editing operations include at least one action of selecting, zooming, cropping, rotating, scaling, adjusting brightness or contrast, applying filters, or implementing annotations. ### Claim 20 (depends on 16) The visual display according to claim 16, further comprising: a checklist module, integrated within the system and configured to provide a list of previous, current, suggested, and next steps of a surgical procedure.