Smartphone Software-Based Measurement of Cyanosis and Capillary Refill Time

A method for measuring blood oxygenation levels and capillary refill time (CRT) of a user using a mobile computing device, the method comprising:

• capturing a video stream of a user, wherein the video stream includes a series of images representing a process of first squeezing a fingernail of the user to a blanched state, and then releasing the fingernail to a resting state;
• receiving one or more input images by an input layer of a Convolutional Neural Network (CNN);
• processing an input image to create a two-dimensional representation of one or more features;
• removing spatial relationships in the two-dimensional representation and generate non-spatial data;
• generating non-spatial metadata for the user based on a metadata vector;
• concatenating non-spatial metadata with the non-spatial metadata;
• identifying the fingernail in the resting state, and one or more regions within a static image of the fingernail, and correlating one or more image features with a value in a range of blood oxygenation values;
• generating a blood oxygenation value indicating blood oxygenation levels in the user, and a measurement confidence level indicating an extent of success of measurement;
• generating a CRT value of the user by applying the blood oxygenation value, and the measurement confidence level to a series of images captured by the camera over time, wherein the CRT corresponds to time lapsed from the blanched state to the resting state of the fingernail;
• and
• displaying the blood oxygenation and CRT values to the user, and generating one or more pre-defined alerts corresponding to the displayed blood oxygenation and CRT values.

A non-transitory computer readable medium configured to store a program causing a computer to measure blood oxygenation levels and capillary refill time (CRT) of a user using a mobile computing device, said program configured to:

• capture a video stream of a user, wherein the video stream includes a series of images representing a process of first squeezing a fingernail of the user to a blanched state, and then releasing the fingernail to a resting state;
• receive one or more input images by an input layer of a Convolutional Neural Network (CNN);
• process an input image to create a two-dimensional representation of one or more features;
• remove spatial relationships in the two-dimensional representation and generate non-spatial data;
• generate non-spatial metadata for the user based on a metadata vector;
• concatenate non-spatial metadata with the non-spatial metadata;
• identify the fingernail in the resting state, and one or more regions within a static image of the fingernail, and correlate one or more non-spatial features with a range of blood oxygenation values;
• generate a blood oxygenation value indicating blood oxygenation levels in the user, and a measurement confidence level indicating an extent of success of measurement;
• generate a CRT value of the user by applying the blood oxygenation value, and the measurement confidence level to a series of images captured by the camera over time, wherein the CRT corresponds to time lapsed from the blanched state to the resting state of the fingernail;
• and
• display the blood oxygenation and CRT values to the user, and generate one or more pre-defined alerts corresponding to the displayed blood oxygenation and CRT values.