Estimating uncertainty in predictions generated by machine learning models

A system comprising:

• one or more computers;
• and
• one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
• receiving multi-modal data characterizing a patient, wherein the multi-modal data comprises a respective feature representation for each of a plurality of modalities;
• processing the multi-modal data characterizing the patient using a machine learning model, in accordance with values of a set of machine learning model parameters, to generate a patient classification that classifies the patient as being included in a patient category from a set of patient categories, comprising:
• processing the multi-modal data characterizing the patient using an encoder neural network of the machine learning model to generate an embedding of the multi-modal data in a latent space;
• and
• generating the patient classification based on the embedding of the multi-modal data in the latent space;
• wherein the encoder neural network has been trained on a plurality of training examples by a machine learning training technique, the training comprising, for each training example:
• processing training multi-modal data included in the training example using the encoder neural network to generate an embedding of the training multi-modal data;
• processing the embedding of the training multi-modal data using a decoder neural network to generate a reconstruction of the training multi-modal data;
• and
• updating current values of a set of encoder neural network parameters using gradients of an objective function that measures an error in the reconstruction of the training multi-modal data;
• determining an uncertainty measure that characterizes an uncertainty of the patient classification generated by the machine learning model;
• and
• generating a clinical recommendation for medical treatment of the patient based on:
• (i) the patient classification, and (ii) the uncertainty measure that characterizes the uncertainty of the patient classification.

One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• receiving multi-modal data characterizing a patient, wherein the multi-modal data comprises a respective feature representation for each of a plurality of modalities;
• processing the multi-modal data characterizing the patient using a machine learning model, in accordance with values of a set of machine learning model parameters, to generate a patient classification that classifies the patient as being included in a patient category from a set of patient categories, comprising:
• processing the multi-modal data characterizing the patient using an encoder neural network of the machine learning model to generate an embedding of the multi-modal data in a latent space;
• and
• generating the patient classification based on the embedding of the multi-modal data in the latent space;
• wherein the encoder neural network has been trained on a plurality of training examples by a machine learning training technique, the training comprising, for each training example:
• processing training multi-modal data included in the training example using the encoder neural network to generate an embedding of the training multi-modal data;
• processing the embedding of the training multi-modal data using a decoder neural network to generate a reconstruction of the training multi-modal data;
• and
• updating current values of a set of encoder neural network parameters using gradients of an objective function that measures an error in the reconstruction of the training multi-modal data;
• determining an uncertainty measure that characterizes an uncertainty of the patient classification generated by the machine learning model;
• and
• generating a clinical recommendation for medical treatment of the patient based on:
• (i) the patient classification, and (ii) the uncertainty measure that characterizes the uncertainty of the patient classification.