Detecting Depression Using Machine Learning Models on Human Speech Samples

A computing device comprising:

• one or more processors;
• memory;
• and
• program instructions, stored in the memory, that upon execution by the one or more processors cause the computing device to perform operations comprising:
• receiving an input audio signal
• determining, by the computing device and from the input audio signal, (i) one or more acoustic features of the input audio signal, and (ii) a visual representation of the input audio signal
• providing the one or more acoustic features to a first trained neural network, wherein the first trained neural network was trained, with a first training data set comprising audio signals and psychological conditions respectively associated with the audio signals, to form predictions of the psychological conditions from the audio signals
• receiving, from the first trained neural network, a first prediction of a psychological condition associated with the input audio signal
• providing the visual representation to a second trained neural network, wherein the second trained neural network was trained, with a second training data set comprising visual images and psychological conditions respectively associated with the visual images, to form predictions of the psychological conditions from the visual images
• receiving, from the second trained neural network, a second prediction of the psychological condition associated with the input audio signal;
• determining a composite prediction comprising a weighted combination of the first and second prediction;
• and
• providing the composite prediction.

A computer-implemented method comprising:

• obtaining, by a computing device, a training data set comprising audio signals and psychological conditions respectively associated with the audio signals;
• determining, by the computing device and from the training data set, (i) one or more acoustic features of the audio signals, and (ii) visual representations of the audio signals;
• using the one or more acoustic features, training, by the computing device, a first neural network to form predictions of the psychological conditions associated with the audio signals;
• using the visual representations, training, by the computing device, a second neural network to form predictions of the psychological conditions associated with the audio signals;
• based at least in part on the training of the first and second neural networks, determining, at the computing device, weightings for combining outputs of the first and second neural networks;
• and
• providing, by the computing device, the first neural network, the second neural network, and the weightings to a second computing device, wherein the second computing device is configured to (i) apply the first neural network on an input audio signal to generate a first prediction, (ii) apply the second neural network on the input audio signal to generate a second prediction, and (iii) apply the weightings on the first and second predictions to produce a weighted combination of the first and second predictions.