US 2024/0379206 A1 · MONITORING AND OPTIMIZATION OF HUMAN-DEVICE INTERACTIONS

Abstract

A real time system for monitoring and optimizing patient adherence to digital therapy. The system can measure at least one of head stability, eye stability, gaze direction, head pose, facial expression, reading related eye movements, eye alignment stability, eye blink rate, attentive engagement, general engagement, emotional state, total eye stability, yawning and distance between the user and the camera. This information is provided as input to computer vision algorithms to generate a multi-dimensional representation of user attention and engagement. The system can alert the caregiver or health care professional if the patient disengages from the treatment and if the patient is sitting at the wrong distance from the treatment device. The system includes a camera, a processor, computer vision algorithms executed by the processor (a GPU processor), at least one digital display device, a loud speaker and an optional internet connection that can enable communication with other electronic devices.

Claims37

Dependent claims

A real-time attention monitoring system, comprising:

a screen at which a person's eyes are directed at;
an image capture device positioned to capture video frames of the person's head and face;
an audio output device controlled by the computing device to provide audio signals to the person;
and
a computing device configured to receive the captured video frames and extract visual cues of eyes and mouth of the person's head and face from each of the captured video frames, the computing device further configured to
increment a frame counter when
an average of an aspect ratio of a left eye and a right eye of the person exceeds a predetermined threshold in a current video frame, or
a yawn is detected from the shape of the mouth in the current video frame and the average of the aspect ratio of the left eye and the right eye of the person is less than or equal to the predetermined threshold,
reset the frame counter when no yawn is detected in the current video frame,
provide an indication of a drowsy state when the frame counter reaches a predetermined number of frames and resetting the frame counter, and
generate at least one feedback indicating the person is disengaged with the screen when at least the indication of the drowsy state is provided, wherein the at least one feedback is output as a sound through the audio output device.

8 more elements

Dependencies are inferred from the claim text.

Monitoring and optimization of human-device interactions

Life of this patent

Monitoring and optimization of human-device interactions

Life of this patent

Abstract

Claims37