Camera-based vital feature estimation

A system for estimating at least one vital feature of a patient from at least a video segment showing at least one skin-exposed region of the patient, the system comprising at least one memory for storing parameters of a neural network, the neural network comprising:

• an encoder trained to accept at least one video patch of the video segment as input and to generate a first vectorial embedding as output;
• at least one fully connected block trained to accept at least the first vectorial embedding as input and to generate at least one second vectorial embedding as output corresponding to a representation of a time series of predicted values of a physiological signal of the patient;
• and
• at least one fully connected block trained to accept an aggregation of the at least one second vectorial embedding and to generate the at least one estimated vital feature for the patient.

A non-transitory computer-readable medium having instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform a method comprising:

• receiving a video segment showing at least one skin-exposed region of a patient;
• cropping the video segment to create at least one video patch;
• and
• generating at least one estimated vital feature of the patient by providing the at least one video patch as input to a neural network trained on a training dataset comprising a plurality of training video segments of a plurality of training subjects, each training video segment showing at least one skin-exposed region of a training subject of the plurality of training subjects and being associated with at least one first time series corresponding to values of a measured physiological signal of the training subject and at least one second time series corresponding to measured values of a vital feature of the training subject,
• wherein the neural network is configured to generate a representation of at least one-time series of predicted values of a predicted physiological signal of the patient and to generate the at least one estimated vital feature based on the representation, and
• wherein the patient and the plurality of training subjects belong to different populations.