# US 10,869,645 B2 · Methods and systems for adaptive imaging for low light signal enhancement in medical visualization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,869,645 B2 - Document ID: US10869645B2 - Status: Granted - Kind: B2 - Application number: 15/623,100 - Filed: 2017-06-14 - Published: 2017-12-14 - Granted: 2020-12-22 - Earliest claimed: 2016-06-14 - Assignees: Stryker European Operations Limited - Applicants: Stryker European Operations Limited - Inventors: John J. P. Fengler; Paul Roald Westwick; Frederick Allen Moore - Web page: https://patentgrep.com/patent/US10869645B2 ## CPC - G16H 30/40: for processing medical images, e.g. editing - A61B 1/000095: {for image enhancement} - A61B 1/043: {for fluorescence imaging} - A61B 1/045: Control thereof - A61B 1/05: characterised by the image sensor, e.g. camera, being in the distal end portion - A61B 1/0638: {providing two or more wavelengths} - A61B 1/0655: {Control therefor} - A61B 1/0669: {at proximal end of an endoscope} - A61B 1/0684: {using light emitting diodes [LED]} - A61B 1/07: using light-conductive means, e.g. optical fibres - A61B 5/0071: {by measuring fluorescence emission} - A61B 6/481: {involving the use of contrast agents} - A61B 6/5205: {involving processing of raw data to produce diagnostic data} - G02B 23/12: with means for image conversion or intensification - G06T 5/50: using two or more images, e.g. averaging or subtraction - H04N 23/11: for generating image signals from visible and infrared light wavelengths - H04N 23/70: Circuitry for compensating brightness variation in the scene - H04N 23/74: by influencing the scene brightness using illuminating means - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - A61B 1/00186: {with imaging filters} - A61B 1/0646: {with illumination filters} - A61B 1/3132: {for laparoscopy} - A61B 5/1121: {Determining geometric values, e.g. centre of rotation or angular range of movement} - A61B 5/6869: {Heart} - A61B 5/6875: {Uterus} - A61B 5/7221: {Determining signal validity, reliability or quality (preventing, reducing or removing noise induced by motion artefacts A61B5/7207; noise originating from a therapeutic or surgical apparatus A61B5/7217)} - A61B 5/7264: {Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems} - A61B 2505/05: Surgical care - A61B 2562/0219: Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches - A61B 2562/0233: Special features of optical sensors or probes classified in A61B5/00 - A61B 2576/00: Medical imaging apparatus involving image processing or analysis (A61B1/00009, A61B6/52 and A61B8/52 take precedence) - G06T 2207/10064: Fluorescence image - G06T 2207/10068: Endoscopic image - G06T 2207/20008: Globally adaptive - G06T 2207/30004: Biomedical image processing - H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes} - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Adaptive imaging methods and systems for generating enhanced low light video of an object for medical visualization are disclosed and include acquiring, with an image acquisition assembly, a sequence of reference frames and/or a sequence of low light video frames depicting the object, assessing relative movement between the image acquisition assembly and the object based on at least a portion of the acquired sequence of reference video frames or the acquired sequence of low light video frames, adjusting a level of image processing of the low light video frames based at least in part on the relative movement between the image acquisition assembly and the object, and generating a characteristic low light video output from a quantity of the low light video frames, wherein the quantity of the low light video frames is based on the adjusted level of image processing of the low light video frames. ## Claims ### Claim 1 An adaptive imaging method for generating enhanced low-intensity light video of an object for medical visualization, comprising: acquiring, with an image acquisition assembly, a sequence of low light video frames depicting the object; receiving a sequence of reference video frames; assessing relative movement between the image acquisition assembly and the object based on the reference video frames; adjusting a level of image processing of the low light video frames based at least in part on the relative movement between the image acquisition assembly and the object; and generating a characteristic low light video output from a quantity of the low light video frames, wherein the quantity of the low light video frames is based on the adjusted level of image processing of the low light video frames; wherein adjusting the level of image processing of the low light video frames comprises adjusting the quantity of low light video frames from which the characteristic low light video output is generated, wherein adjusting the quantity of low light video frames comprises setting the quantity of low light video frames to a first predetermined value if the relative movement between the image acquisition assembly and the object is below a first motion threshold, wherein setting the quantity of low light video frames to the first predetermined value comprises gradually increasing or decreasing the quantity of low light video frames to the first predetermined value over a series of frames of the characteristic low light video output, and wherein adjusting the quantity of low light video frames comprises setting the quantity of low light video frames to a second predetermined value if the relative movement between the image acquisition assembly and the object is above the first motion threshold, the second predetermined value being lower than the first predetermined value. ### Claim 2 (depends on 1) The method of claim 1, wherein the sequence of reference video frames comprises frames from the sequence of low light video frames. ### Claim 3 (depends on 1) The method of claim 1, further comprising acquiring, with the image acquisition assembly, a sequence of higher intensity light video frames generally having a higher light intensity than the low light video frames, wherein the sequence of higher intensity light video frames are acquired substantially simultaneously with the sequence of low light video frames. ### Claim 4 (depends on 3) The method of claim 3, wherein the sequence of reference video frames comprises frames from the sequence of higher intensity light video frames. ### Claim 5 (depends on 1) The method of claim 1, wherein assessing relative movement between the image acquisition assembly and the object comprises measuring changes in pixel intensities in a plurality of the reference video frames. ### Claim 6 (depends on 1) The method of claim 1, wherein assessing relative movement between the image acquisition assembly and the object comprises determining a representative pixel intensity for each of a plurality of subregions in the plurality of reference video frames, and characterizing the changes in representative pixel intensity for the subregions in the plurality of the reference video frames. ### Claim 7 (depends on 1) The method of claim 1, wherein setting the quantity of low light video frames to the second predetermined value comprises gradually increasing or decreasing the quantity of low light video frames to the second predetermined value over a series of frames of the characteristic low light video output. ### Claim 8 (depends on 7) The method of claim 7, wherein adjusting the quantity of low light video frames comprises: setting the quantity of low light video frames to the second predetermined value if the relative movement between the image acquisition assembly and the object is above the first motion threshold and below a second motion threshold higher than the first motion threshold; and setting the quantity of low light video frames to a third predetermined value, if the relative movement between the image acquisition assembly and the object is above the first and second motion thresholds, the third predetermined value being lower than the first and second predetermined values. ### Claim 9 (depends on 8) The method of claim 8, wherein setting the quantity of low light video frames to the third predetermined value comprises gradually increasing or decreasing the quantity of low light video frames to the third predetermined value over a series of frames of the characteristic low light video output. ### Claim 10 (depends on 9) The method of claim 9, wherein adjusting the quantity of low light video frames comprises gradually increasing or decreasing the quantity of low light video frames toward a predetermined value over a series of frames of the characteristic low light video output. ### Claim 11 (depends on 1) The method of claim 1, wherein generating the characteristic low light video output comprises determining a sum of pixel intensities of the quantity of the low light video frames on a region-by-region basis. ### Claim 12 (depends on 11) The method of claim 11, wherein generating the characteristic low light video output further comprises dividing the sum of pixel intensities by the square root of the quantity of the low light video frames. ### Claim 13 (depends on 1) The method of claim 1, wherein generating the characteristic low light video output from the quantity of the low light video frames comprises averaging pixel intensities of the quantity of the low light video frames on a region-by-region basis. ### Claim 14 (depends on 13) The method of claim 13, further comprising adjusting a low light video frame exposure period based at least in part on the relative movement between the image acquisition assembly and the object. ### Claim 15 (depends on 1) The method of claim 1, further comprising displaying at least one of the characteristic low light video output and the reference video frames on a display. ### Claim 16 (depends on 1) The method of claim 1, further comprising controlling a timing scheme of a visible light source illuminating the object, an excitation light source illuminating the object, and the image acquisition assembly based at least in part on the relative movement between the image acquisition assembly and the object. ### Claim 17 (depends on 1) The method of claim 1, wherein the method is performed continuously. ### Claim 18 (depends on 1) The method of claim 1, wherein the sequence of low light video frames comprises a sequence of fluorescence video frames. ### Claim 19 (depends on 1) The method of claim 1, wherein the sequence of low light video frames comprises a sequence of reflected light video frames. ### Claim 20 (depends on 1) The method of claim 1, wherein the sequence of reference video frames comprises a sequence of reflected light video frames. ### Claim 21 An adaptive imaging system for generating enhanced low-intensity light video of an object for medical visualization, comprising: an image acquisition assembly configured to acquire a sequence of low light video frames depicting the object; one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: assessing relative movement between the image acquisition assembly and the object based on reference video frames; adjusting a level of image processing of the low light video frames based at least in part on the relative movement between the image acquisition assembly and the object; and generating a characteristic low light video output from a quantity of the low light video frames, wherein the quantity of the low light video frames is based on the adjusted level of image processing of the low light video frames, wherein adjusting the level of image processing of the low light video frames comprises adjusting the quantity of low light video frames from which the characteristic low light video output is generated, wherein adjusting the quantity of low light video frames comprises setting the quantity of low light video frames to a first predetermined value if the relative movement between the image acquisition assembly and the object is below a first motion threshold, wherein setting the quantity of low light video frames to the first predetermined value comprises gradually increasing or decreasing the quantity of low light video frames to the first predetermined value over a series of frames of the characteristic low light video output, and wherein adjusting the quantity of low light video frames comprises setting the quantity of low light video frames to a second predetermined value if the relative movement between the image acquisition assembly and the object is above the first motion threshold, the second predetermined value being lower than the first predetermined value. ### Claim 22 (depends on 1) A fluorescence imaging agent for use with the method of claim 1 for imaging an object during blood flow imaging, tissue perfusion imaging, lymphatic imaging, or a combination thereof.