# US 12,009,083 B2 · Remote physical therapy and assessment of patients > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,009,083 B2 - Document ID: US12009083B2 - Status: Granted - Kind: B2 - Application number: 17/526,839 - Filed: 2021-11-15 - Published: 2022-05-19 - Granted: 2024-06-11 - Earliest claimed: 2020-11-16 - Assignees: Electronic Caregiver, Inc. - Applicants: Electronic Caregiver, Inc. - Inventors: David W. Keeley; Anthony Dohrmann; Robert Wood - Web page: https://patentgrep.com/patent/US12009083B2 ## CPC - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising - A61B 5/0013: {Medical image data (A61B1/00011, A61B6/56, A61B8/56 take precedence)} - A61B 5/744: {Displaying an avatar, e.g. an animated cartoon character} - G06F 3/048: Interaction techniques based on graphical user interfaces [GUI] - G06N 20/00: Machine learning - G16H 10/60: for patient-specific data, e.g. for electronic patient records - G16H 30/00: ICT specially adapted for the handling or processing of medical images (computed tomography A61B6/03) - G16H 30/40: for processing medical images, e.g. editing - G16H 40/67: for remote operation - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - A61B 5/1118: {Determining activity level} - A61B 5/4082: {Diagnosing or monitoring movement diseases, e.g. Parkinson, Huntington or Tourette} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Systems and methods for physical therapy and training delivery are presented herein. These technologies may comprise notifying a patient of a scheduled prescribed activity via an on-location at least one client device or console; identifying the patient with one or more sensors connected to or part of the at least one client device or console; confirming, via the at least one client device or console, the patient's acknowledgment of the notification; demonstrating, via a graphical interactive avatar displayed on the at least one client device or console, the prescribed activity to be carried out by the patient; confirming, via the at least one client device or console, that the patient is undertaking or will be undertaking the prescribed activity; capturing, via the one or more sensors, frames of the patient undertaking the prescribed activity; and processing frames of the patient undertaking the prescribed activity. ## Claims ### Claim 1 A method for remote physical therapy and assessment, the method executed by an at least one processor, comprising: notifying a patient of a scheduled prescribed activity via an on-location at least one client device or console; identifying the patient with one or more sensors connected to or part of the at least one client device or console; confirming, via the at least one client device or console, the patient's acknowledgment of the notification; demonstrating, via a graphical interactive avatar displayed on the at least one client device or console, the prescribed activity to be carried out by the patient; confirming, via the at least one client device or console, that the patient is undertaking or will be undertaking the prescribed activity; capturing, via the one or more sensors, frames of the patient undertaking the prescribed activity; processing the frames of the patient undertaking the prescribed activity; the capturing of the frames being undertaken by a left sensor and a right sensor; and the processing of the frames comprising: transmitting the captured frames to an internal depth image processor, wherein the captured frames consist of pixel data of one or more captured scenes; calculating, by the internal depth image processor, a depth of each pixel, to produce depth pixel values; processing, by the internal depth image processor, of the depth pixel values to create a depth frame; combining, by the internal depth image processor, the depth frames into a depth video stream; applying a point cloud over the depth video stream; and estimating position and orientation of the patient's body joint centers. ### Claim 2 (depends on 1) The method of claim 1, wherein the frames are depth scene frames. ### Claim 3 (depends on 1) The method of claim 1, wherein the one or more sensors are cameras, imagers, image capture devices, or depth cameras. ### Claim 4 (depends on 1) The method of claim 1, wherein the processing occurs on the client device or console. ### Claim 5 (depends on 1) The method of claim 1, further comprising: instructing the patient to get into a specific position or orientation. ### Claim 6 (depends on 1) The method of claim 1, further comprising: transmitting the results of the processing of the frames to an AI game engine or a backend system, wherein the AI game engine or the backend system can analyze trends or changes in a movement of the patient; and presenting the analyzed results to the patient via a graphical user interface on the client device or console. ### Claim 7 (depends on 1) The method of claim 1, wherein failure to confirm patient's acknowledgment, or that the patient is undertaking or will be undertaking the prescribed activity, causes a notification to be sent to a care circle. ### Claim 8 A system for remote therapy and assessment of patients, comprising: one or more sensors to capture and transmit visual data; an at least one on-location client device or console to display personalized instructions, having or connected to the one or more sensors, the at least one on-location client device or console having an at least one processor and a memory communicatively coupled to the at least one processor, the memory storing instructions executable by the processor to: notify a patient of a scheduled prescribed activity via the on-location at least one client device or console; identify the patient with the one or more sensors; confirm, via the at least one client device or console, the patient's acknowledgment of the notification; demonstrate, via a graphical interactive avatar displayed on the at least one client device or console, the prescribed activity to be carried out by the patient; confirm, via the at least one client device or console, that the patient is undertaking or will be undertaking the prescribed activity; capture frames of the patient undertaking the prescribed activity via the one or more sensors; and process the frames of the patient undertaking the prescribed activity; an interactive graphical user interface to display the personalized instructions on the at least one on-location client device; and an AI virtual game engine, to analyze the captured frame data and produce updated personalized instructions; the capturing of the frames being undertaken by a left sensor and a right sensor; and the processing of the frames, comprising: transmitting the captured frames to an internal depth image processor, wherein the captured frames consist of pixel data of one or more captured scenes; calculating, by the internal depth image processor, a depth of each pixel, to produce depth pixel values; processing, by the internal depth image processor, of the depth pixel values to create a depth frame; combining, by the internal depth image processor, the depth frames into a depth video stream; applying a point cloud over the depth video stream; and estimating position and orientation of the patient's body joint centers. ### Claim 9 (depends on 8) The system of claim 8, further comprising: a network, whereby the network is connected to at least one of the one or more sensors, the at least one on-location client device or console, and the AI virtual game engine. ### Claim 10 (depends on 8) The system of claim 8, wherein the frames are depth scene frames. ### Claim 11 (depends on 8) The system of claim 8, wherein the one or more sensors are cameras, imagers, image capture devices, or depth cameras. ### Claim 12 (depends on 8) The system of claim 8, wherein the processing occurs on the client device or console. ### Claim 13 (depends on 8) The system of claim 8, further comprising: instructing the patient to get into a specific position or orientation. ### Claim 14 (depends on 8) The system of claim 8, further comprising: transmitting the results of the processing of the frames to the AI virtual game engine or a backend system, wherein the AI virtual game engine or the backend system can analyze trends or changes in a movement of the patient; and presenting the analyzed results to the patient via a graphical user interface on the at least one client device or console. ### Claim 15 (depends on 8) The system of claim 8, wherein failure to confirm patient's acknowledgment, or that the patient is undertaking or will be undertaking the prescribed activity, causes a notification or alert to be sent to a care circle. ### Claim 16 (depends on 8) The system of claim 8, further comprising: instructing the patient to get into a specific position or orientation. ### Claim 17 A non-transitory media having instructions for executing a method by a processor, the method comprising: notifying a patient of a scheduled prescribed activity via an on-location at least one client device or console; identifying the patient with one or more sensors connected to or part of the at least one client device or console; confirming, via the at least one client device or console, the patient's acknowledgment of the notification; demonstrating, via a graphical interactive avatar displayed on the at least one client device or console, the prescribed activity to be carried out by the patient; instructing the patient to get into a specific position or orientation; confirming, via the at least one client device or console, that the patient is undertaking or will be undertaking the prescribed activity; capturing, via the one or more sensors, frames of the patient undertaking the prescribed activity; and processing the frames of the patient undertaking the prescribed activity, wherein the processing comprises: transmitting the captured frames to an internal depth image processor, wherein the captured frames consist of pixel data of one or more captured scenes; calculating, by the internal depth image processor, a depth of each pixel, to produce depth pixel values; processing, by the internal depth image processor, of the depth pixel values to create a depth frame; combining, by the internal depth image processor, the depth frames into a depth video stream; applying a point cloud over the depth video stream; and estimating position and orientation of the patient's body joint centers; transmitting the results of the processing of the frames to an AI game engine or a backend system, wherein the AI game engine or the backend system can analyze trends or changes in a movement of the patient; and presenting the analyzed data to the patient via a graphical user interface on the at least one client device or console; the capturing of the frames is undertaken by a left sensor and a right sensor.