# US 12,322,512 B2 · Using a set of machine learning diagnostic models to determine a diagnosis based on a skin tone of a patient > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,322,512 B2 - Document ID: US12322512B2 - Status: Granted - Kind: B2 - Application number: 18/662,798 - Filed: 2024-05-13 - Published: 2024-09-05 - Granted: 2025-06-03 - Earliest claimed: 2019-06-18 - Assignees: Digital Diagnostics Inc. - Applicants: Digital Diagnostics Inc. - Inventors: Elliot Swart; Elektra Efstratiou Alivisatos; Joseph Ferrante; Elizabeth Asai - Web page: https://patentgrep.com/patent/US12322512B2 ## CPC - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens} - A61B 5/1032: {Determining colour of tissue for diagnostic purposes} - A61B 5/441: {Skin evaluation, e.g. for skin disorder diagnosis} - A61B 5/444: {Evaluating skin marks, e.g. mole, nevi, tumour, scar} - A61B 5/6898: {Portable consumer electronic devices, e.g. music players, telephones, tablet computers} - A61B 5/7267: {involving training the classification device} - A61B 5/7275: {Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor} - G06F 18/24765: {Rule-based classification} - G06N 20/00: Machine learning - G06T 7/0012: {Biomedical image inspection} - G06T 7/0014: {using an image reference approach} - G06T 7/90: Determination of colour characteristics - G06V 10/56: relating to colour - G06V 10/72: Data preparation, e.g. statistical preprocessing of image or video features - G06V 10/764: using classification, e.g. of video objects - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - G16H 30/40: for processing medical images, e.g. editing - G16H 40/67: for remote operation - A61B 2576/02: specially adapted for a particular organ or body part - G06T 2207/10024: Color image - G06T 2207/20008: Globally adaptive - G06T 2207/20076: Probabilistic image processing - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30088: Skin; Dermal - G06T 2207/30168: Image quality inspection - G06V 2201/03: Recognition of patterns in medical or anatomical images - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Systems and methods are disclosed herein for determining a diagnosis based on a base skin tone of a patient. In an embodiment, the system receives a base skin tone image of a patient, generates a calibrated base skin tone image by calibrating the base skin tone image using a reference calibration profile, and determines a base skin tone of the patient based on the calibrated base skin tone image. The system receives a concern image of a portion of the patient's skin, and selects a set of machine learning diagnostic models from a plurality of sets of candidate machine learning diagnostic models based on the base skin tone of the patient, each of the sets of candidate machine learning diagnostic models trained to receive the concern image and output a diagnosis of a condition of the patient. ## Claims ### Claim 1 A method for determining a diagnosis based on a base skin tone of a patient, the method comprising: determining a base skin tone of a patient by: generating a calibrated base skin tone image using an image of the patient and a reference calibration profile; determining a numerical representation of each pixel of the calibrated base skin tone image; generating an aggregate representation by performing a statistical operation based on the numerical representation of each pixel; identifying a point in a color space corresponding to the aggregate representation; and mapping the point in the color space into a discrete value in a classification system; receiving a concern image of a portion of the patient's skin; inputting at least the concern image into at least one machine learning model; and receiving, as output from the at least one machine learning model, information from which the diagnosis is derived, wherein the diagnosis is at least partially derived based on the base skin tone, wherein determining the diagnosis further comprises: modifying the information from which the diagnosis is derived based on the base skin tone; and determining the diagnosis based on the modified information. ### Claim 2 (depends on 1) The method of claim 1, wherein the base skin tone is calibrated responsive to determining that the image of the patient satisfies a quality criterion. ### Claim 3 (depends on 1) The method of claim 1, wherein the method further comprises: generating an adapted imaging profile based on the reference calibration profile and the base skin tone; and calibrating the concern image using the adapted imaging profile. ### Claim 4 (depends on 1) The method of claim 1, wherein the method further comprises selecting a set of machine learning diagnostic models from a plurality of sets of candidate machine learning diagnostic models based on the base skin tone of the patient, each of the sets of candidate machine learning diagnostic models trained to receive the concern image and output a diagnosis of a condition of the patient, and wherein the at least one machine learning model is a part of the selected set. ### Claim 5 (depends on 4) The method of claim 4, wherein the method further comprises determining a classification that corresponds to the base skin tone, wherein selecting the set of machine learning diagnostic models comprises selecting a diagnosis classifier that corresponds to the classification, and wherein the method further comprises: inputting the concern image into the selected diagnosis classifier; and receiving, as output from the selected diagnosis classifier, information from which the diagnosis is derived. ### Claim 6 (depends on 1) The method of claim 1, wherein the diagnosis is at least partially derived from the base skin tone image by inputting the base skin tone along with the concern image into the at least one machine learning model. ### Claim 7 (depends on 1) The method of claim 1, wherein the method further comprises: determining whether the concern image is diagnosable based on the base skin tone; and responsive to determining that the concern image is not diagnosable, outputting an indication that the concern image is not diagnosable. ### Claim 8 (depends on 1) The method of claim 1, wherein the diagnosis comprises a probability that the patient has a condition. ### Claim 9 A computer program product for determining a diagnosis based on a base skin tone of a patient, the computer program product comprising a non-transitory computer-readable storage medium containing computer program code for: determining a base skin tone of a patient by: generating a calibrated base skin tone image using an image of the patient and a reference calibration profile; determining a numerical representation of each pixel of the calibrated base skin tone image; generating an aggregate representation by performing a statistical operation based on the numerical representation of each pixel; identifying a point in a color space corresponding to the aggregate representation; and mapping the point in the color space into a discrete value in a classification system; receiving a concern image of a portion of the patient's skin; inputting at least the concern image into at least one machine learning model; and receiving, as output from the at least one machine learning model, information from which the diagnosis is derived, wherein the diagnosis is at least partially derived based on the base skin tone, wherein determining the diagnosis further comprises: modifying the information from which the diagnosis is derived based on the base skin tone; and determining the diagnosis based on the modified information. ### Claim 10 (depends on 9) The computer program product of claim 9, wherein the base skin tone is calibrated responsive to determining that the image of the patient satisfies a quality criterion. ### Claim 11 (depends on 9) The computer program product of claim 9, wherein the computer program code is further for: generating an adapted imaging profile based on the reference calibration profile and the base skin tone; and calibrating the concern image using the adapted imaging profile. ### Claim 12 (depends on 9) The computer program product of claim 9, wherein the computer program code is further for selecting a set of machine learning diagnostic models from a plurality of sets of candidate machine learning diagnostic models based on the base skin tone of the patient, each of the sets of candidate machine learning diagnostic models trained to receive the concern image and output a diagnosis of a condition of the patient, and wherein the at least one machine learning model is a part of the selected set. ### Claim 13 (depends on 12) The computer program product of claim 12, wherein the computer program code is further for determining a classification that corresponds to the base skin tone, wherein selecting the set of machine learning diagnostic models comprises selecting a diagnosis classifier that corresponds to the classification, and wherein the computer program code is further for: inputting the concern image into the selected diagnosis classifier; and receiving, as output from the selected diagnosis classifier, information from which the diagnosis is derived. ### Claim 14 (depends on 9) The computer program product of claim 9, wherein the diagnosis is at least partially derived from the base skin tone image by inputting the base skin tone along with the concern image into the at least one machine learning model. ### Claim 15 (depends on 9) The computer program product of claim 9, wherein the computer program code is further for: determining whether the concern image is diagnosable based on the base skin tone; and responsive to determining that the concern image is not diagnosable, outputting an indication that the concern image is not diagnosable. ### Claim 16 (depends on 9) The computer program product of claim 9, wherein the diagnosis comprises a probability that the patient has a condition.