Remote physical therapy and assessment of patients

A system for remote therapy and assessment of patients, comprising:

• one or more sensors to capture and transmit visual data;
• an at least one on-location client device or console to display personalized instructions, having or connected to the one or more sensors, the at least one on-location client device or console having an at least one processor and a memory communicatively coupled to the at least one processor, the memory storing instructions executable by the processor to:
• notify a patient of a scheduled prescribed activity via the on-location at least one client device or console;
• identify the patient with the one or more sensors;
• confirm, via the at least one client device or console, the patient's acknowledgment of the notification;
• demonstrate, via a graphical interactive avatar displayed on the at least one client device or console, the prescribed activity to be carried out by the patient;
• confirm, via the at least one client device or console, that the patient is undertaking or will be undertaking the prescribed activity;
• capture frames of the patient undertaking the prescribed activity via the one or more sensors;
• and
• process the frames of the patient undertaking the prescribed activity;
• an interactive graphical user interface to display the personalized instructions on the at least one on-location client device;
• and
• an AI virtual game engine, to analyze the captured frame data and produce updated personalized instructions;
• the capturing of the frames being undertaken by a left sensor and a right sensor;
• and
• the processing of the frames, comprising:
• transmitting the captured frames to an internal depth image processor, wherein the captured frames consist of pixel data of one or more captured scenes;
• calculating, by the internal depth image processor, a depth of each pixel, to produce depth pixel values;
• processing, by the internal depth image processor, of the depth pixel values to create a depth frame;
• combining, by the internal depth image processor, the depth frames into a depth video stream;
• applying a point cloud over the depth video stream;
• and
• estimating position and orientation of the patient's body joint centers.

A non-transitory media having instructions for executing a method by a processor, the method comprising:

• notifying a patient of a scheduled prescribed activity via an on-location at least one client device or console;
• identifying the patient with one or more sensors connected to or part of the at least one client device or console;
• confirming, via the at least one client device or console, the patient's acknowledgment of the notification;
• demonstrating, via a graphical interactive avatar displayed on the at least one client device or console, the prescribed activity to be carried out by the patient;
• instructing the patient to get into a specific position or orientation;
• confirming, via the at least one client device or console, that the patient is undertaking or will be undertaking the prescribed activity;
• capturing, via the one or more sensors, frames of the patient undertaking the prescribed activity;
• and
• processing the frames of the patient undertaking the prescribed activity, wherein the processing comprises:
• transmitting the captured frames to an internal depth image processor, wherein the captured frames consist of pixel data of one or more captured scenes;
• calculating, by the internal depth image processor, a depth of each pixel, to produce depth pixel values;
• processing, by the internal depth image processor, of the depth pixel values to create a depth frame;
• combining, by the internal depth image processor, the depth frames into a depth video stream;
• applying a point cloud over the depth video stream;
• and
• estimating position and orientation of the patient's body joint centers;
• transmitting the results of the processing of the frames to an AI game engine or a backend system, wherein the AI game engine or the backend system can analyze trends or changes in a movement of the patient;
• and
• presenting the analyzed data to the patient via a graphical user interface on the at least one client device or console;
• the capturing of the frames is undertaken by a left sensor and a right sensor.