# US 12,216,112 B2 · Method to detect white blood cells and/or white blood cell subtypes from non-invasive capillary videos > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,216,112 B2 - Document ID: US12216112B2 - Status: Granted - Kind: B2 - Application number: 18/508,899 - Filed: 2023-11-14 - Published: 2024-03-14 - Granted: 2025-02-04 - Earliest claimed: 2020-05-28 - Assignees: Leuko Labs, Inc. - Applicants: Leuko Labs, Inc. - Inventors: Carlos Castro Gonzalez; Ian Butterworth; Aurelien Bourquard; Alvaro Sanchez Ferro - Web page: https://patentgrep.com/patent/US12216112B2 ## CPC - G01N 33/49: Blood {(chemical methods for determining blood cell populations G01N33/5094; chemical analysis of blood groups or blood types G01N33/80)} - G01N 15/1429: Signal processing - G01N 15/1433: using image recognition - G01N 15/147: {the analysis being performed on a sample stream} - G01N 21/31: Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry {(G01N21/72 takes precedence)} - G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade} - G06T 7/0016: {involving temporal comparison} - G06T 7/11: Region-based segmentation - G06T 7/38: Registration of image sequences - G06V 10/56: relating to colour - G06V 10/764: using classification, e.g. of video objects - G06V 10/82: using neural networks - G06V 20/693: {Acquisition} - G06V 20/695: {Preprocessing, e.g. image segmentation} - G06V 20/698: {Matching; Classification} - G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS - G16H 30/40: for processing medical images, e.g. editing - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients - G01N 2015/012: {Red blood cells} - G01N 2015/016: {White blood cells} - G01N 2015/1006: {for cytology} - G01N 2015/1443: {Auxiliary imaging} - G01N 2015/1486: {Counting the particles} - G06T 2207/10016: Video; Image sequence - G06T 2207/10056: Microscopic image - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30101: Blood vessel; Artery; Vein; Vascular - G06T 2207/30242: Counting objects in image - G06V 2201/03: Recognition of patterns in medical or anatomical images - G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract In one aspect, a method to detect white blood cells and/or white blood cell subtypes from non-invasive capillary videos is featured. The method includes acquiring a first plurality of images of a region of interest including one or more capillaries of a predetermined area of a human subject from non-invasive capillary videos captured with an optical device, processing the first plurality of images to determine one or more optical absorption gaps located in said capillary, and annotating the first plurality of images with an indication of any optical absorption gap detected in the first plurality of images. The method also includes acquiring a second plurality of images of the same region of interest of the same capillary with an advanced optical device capable of resolving cellular structure of white blood cells and white blood cell subtypes and spatiotemporally annotating the second plurality of images with an indication of any white blood cell detected and/or a subtype of any white blood cell detected in the second plurality of images. The method also includes inputting the first plurality of images and annotated information from the first plurality of images and annotated information from the spatiotemporally annotated second plurality of images into a machine learning subsystem configured to determine a presence of white blood cells and/or the subtype of any white blood cells present in the one or more optical absorption gaps in the first plurality of images. ## Claims ### Claim 1 A method to detect white blood cells and/or white blood cell subtypes from non-invasive capillary videos, the method comprising: acquiring a first plurality of images of a region of interest including one or more capillaries of a predetermined area of a human subject from non-invasive capillary videos captured with an optical imaging process; processing the first plurality of images to determine one or more optical absorption gaps located in said capillary; annotating the first plurality of images with an indication of any optical absorption gap detected in the first plurality of images; acquiring a second plurality of images of the same region of interest of the same capillary with an advanced optical imaging process capable of resolving cellular structure of white blood cells and white blood cell subtypes; spatiotemporally annotating the second plurality of images with an indication of any white blood cell detected and/or a subtype of any white blood cell detected in the second plurality of images; and determining a presence of white blood cells and/or the subtype of any white blood cells present in the one or more optical absorption gaps using a machine learning subsystem. ### Claim 2 (depends on 1) The method of claim 1 in which the machine learning subsystem is configured to determine a white blood cell subtype for any optical absorption gap detected in the first plurality of images. ### Claim 3 (depends on 2) The method of claim 2 in which the machine learning subsystem is configured to determine full white blood cell differential measurements and/or partial white blood cell differential measurements. ### Claim 4 (depends on 1) The method of claim 1 including temporally aligning the first plurality of images to the spatiotemporally annotated second plurality of images. ### Claim 5 (depends on 4) The method of claim 4 in which the temporally aligning includes creating said region of interest and said same region of interest by using a same objective lens for the optical process and the advanced optical process. ### Claim 6 (depends on 4) The method of claim 4 in which the temporally aligning includes creating said region of interest and said same region of interest by focusing the optical process and the advanced optical process at a same location in the capillary. ### Claim 7 (depends on 4) The method of claim 4 including generating optical absorption gap reference data including a frame identifier and indication of any optical absorption gap detected in the first plurality of images. ### Claim 8 (depends on 7) The method of claim 7 including generating spatiotemporally annotated lookup data including a frame identifier and indication of the subtype of any white blood cell present. ### Claim 9 (depends on 8) The method of claim 8 in which the temporally aligning the first plurality of images to the spatiotemporally annotated second plurality of images includes temporally aligning the frame identifier of the first plurality of images to the frame identifier of the visually spatiotemporally annotated second plurality of images. ### Claim 10 (depends on 9) The method of claim 9 including inputting the first plurality of images, the optical absorption gap reference data, and the spatiotemporally annotated lookup data into the machine learning subsystem, the machine learning subsystem configured to output results data of the any white blood cells detected and/or the subtype of any white blood cells detected and compare the results table to ground truth data. ### Claim 11 (depends on 10) The method of claim 10 in which machine learning subsystem is configured to output results data of the any white blood cells detected and/or a subtype of any white blood cells detected for each optical absorption gap in the first plurality images and compare the results data to a ground truth data. ### Claim 12 (depends on 1) The method of claim 1 in which said spatiotemporally annotating the second plurality of images includes indicating one or more of: a size, a granularity, a brightness, a speed, an elongation, and/or a margination of the white blood cells and/or a change of density of red blood cells located upstream or downstream from a location of white blood cells detected. ### Claim 13 (depends on 1) The method of claim 1 in which the subtype of the white blood cell includes a granulocyte, a neutrophil, a lymphocyte, a monocyte, an eosinophil or a basophil. ### Claim 14 (depends on 1) The method of claim 1 in which the predetermined area of the human subject includes one or more of: a finger, a nailfold, a toe, a tongue, a gum, a lip, a retina, and/or an earlobe. ### Claim 15 (depends on 1) The method of claim 1 in which the optical process is configured to output at least one optical absorption gap signal. ### Claim 16 (depends on 1) The method of claim 1 in which the spatiotemporally annotating the second plurality of images is performed by a human. ### Claim 17 (depends on 1) The method of claim 1 in which the spatiotemporally annotating the second plurality of images is performed by a processing subsystem. ### Claim 18 (depends on 1) The method of claim 1 including determining the presence of white blood cells and/or the subtype of any white blood cells present in the one or more optical absorption gaps using the first plurality of images and annotated information from the first plurality of images and information from the machine learning subsystem which has learned and determined the presence of white blood cells and/or the subtype of white blood cells present in one or more optical absorption gaps using the annotated information from the second plurality of images acquired with the advanced optical process. ### Claim 19 A method to detect white blood cells and/or white blood cell subtypes from non-invasive capillary videos, the method comprising: acquiring a first plurality of images of a region of interest including one or more capillaries of a predetermined area of a human subject from non-invasive capillary videos captured with an optical process; processing the first plurality of images to determine one or more optical absorption gaps located in said capillary; annotating the first plurality of images with an indication of any optical absorption gap detected in the first plurality of images; and determining a presence of white blood cells and/or the subtype of any white blood cells present in the one or more optical absorption gaps using the first plurality of images and annotated information from the first plurality of images and information from a machine learning subsystem which has learned and determined the presence of white blood cells and/or the subtype of white blood cells present in one or more optical absorption gaps using annotated information from a second plurality of images acquired with an advanced optical process. ### Claim 20 A method to determine a density of red blood cells from non-invasive capillary videos, the method comprising: acquiring a first plurality of images of a region of interest including one or more capillaries of a predetermined area of a human subject from non-invasive capillary videos captured with an optical process; processing the first plurality of images to determine one or more areas of hemoglobin optical absorption gaps located in said capillary; annotating the first plurality of images with an indication of any areas of hemoglobin optical absorption gaps detected in the first plurality of images; acquiring a second plurality of images of the same region of interest of the same capillary with an advanced optical process capable of resolving cellular structure of red blood cells; spatiotemporally annotating the second plurality of images with an indication of a density of any red blood cells detected in the second plurality of images; and determining the density of any red blood cells present in the one or more optical absorption gaps using a machine learning subsystem. ### Claim 21 (depends on 20) The method of claim 20 in which a red blood cell count is determined from the density of red blood cells.