Method and system for heart rate extraction from RGB images

A method for determining one or more heart beat features, the method comprising:

• capturing a series of images of a user in a car by a camera in the car, wherein each image comprises one or more colour signals;
• for each of the images:
• locating a face of the user in the image;
• determining a boundary in the image, wherein the face is located within the boundary;
• detecting one or more facial features within the boundary;
• and
• based on the detected facial features, determining one or more regions in the image representing skin patches in the face;
• analysing the colour signals in the regions representing the skin patches;
• and
• determining one or more heart beat features based on the analysis;
• wherein analysing the colour signals further comprises:
• determining colour changes in the series of images of one or more skin patches;
• creating a pulse wave for each skin patch based on the colour changes;
• splitting the pulse waves from each skin patch into a plurality of overlapping time windows, wherein each of the plurality of overlapping time windows overlaps in time with an adjacent time window of the plurality of time windows;
• transforming each of the windows into the frequency space to generate frequency spectra;
• summing up said spectra into one accumulated spectrum;
• comparing each frequency spectrum with a predetermined pulse wave pattern and determining a level of similarity;
• discarding all pulse waves for which the level of similarity does not reach a first predetermined threshold;
• and
• determining a weighted average of all pulse waves for which the level of similarity does reach the first predetermined threshold, wherein the level of similarity defines weights for the weighted average.