Systems and methods for assessing a patient and predicting patient outcomes

A computer-implemented method comprising:

• receiving, from a user device, video data and audio data of a patient assessment, wherein the patient assessment comprises at least one gait or balance assessment activity;
• processing the audio data to obtain speech information in the form of transcribed text, wherein processing the audio data further comprises identifying one or more speech cues in the speech information and assigning a respective timestamp to each of the one or more speech cues;
• assigning one or more timepoints to the video data based on the respective timestamps of the one or more speech cues to define at least one segment;
• processing the video data of the at least one segment to obtain pose information, wherein the pose information comprises a plurality of joint points representing the patient;
• extracting at least one gait or balance measurement from the pose information;
• and
• generating, via a machine learning model, a prediction of a patient outcome using the at least one gait or balance measurement as input, wherein the machine learning model has been trained using training data comprising patient outcomes in association with a plurality of previous gait or balance assessments.