Computer apparatus and method implementing sound detection to recognize an activity

A computer implemented method of recognising an activity of a user, the method comprising:

• receiving audio information relating to one or more non-verbal sounds captured by a microphone in an environment of the user, wherein the audio information comprises, for each frame of a sequence of frames of audio data captured by the microphone, a score for each of a set of sound classes that the frame represents the sound class;
• receiving motion information that is based on motion sensor data captured by a motion sensor, said motion information relating to motion of said user in the environment, wherein the motion information comprises, for each frame of a sequence of frames of motion data captured by the motion sensor, a score for each of a set of activity classes that the frame represents the activity class;
• processing the audio information and the motion information to recognise an activity of said user by:
• inputting the audio information and the motion information into a model configured to receive the scores for the set of sound classes and the scores for the set of activity classes, and return an activity recognition notification indicating said activity;
• and
• receiving, from the model, an activity recognition notification indicating said activity;
• and
• outputting the activity recognition notification indicating said activity.

A computing device, the computing device comprising a processor configured to:

• receive audio information relating to one or more non-verbal sounds captured by a microphone in an environment of a user, wherein the audio information comprises multiple acoustic features for each frame of a sequence of frames of audio data captured by the microphone;
• and
• receive motion information that is based on motion sensor data captured by a motion sensor, said motion information relating to motion of said user in the environment, wherein the motion information comprises multiple motion features for each frame of a sequence of frames of motion data captured by the motion sensor;
• process the audio information and the motion information to recognise an activity of said user by:
• inputting the audio information and the motion information into a model configured to:
• receive the audio features and the motion features and return an activity recognition notification indicating said activity;
• and
• receiving, from the model, the activity recognition notification indicating said activity;
• output the activity recognition notification indicating said activity.

A computer implemented method of recognising an activity of a user, the method comprising:

• receiving audio information relating to one or more non-verbal sounds captured by a microphone in an environment of a user, wherein the audio information comprises multiple acoustic features for each frame of a sequence of frames of audio data captured by the microphone;
• receiving motion information that is based on motion sensor data captured by a motion sensor, said motion information relating to motion of said user in the environment, wherein the motion information comprises multiple motion features for each frame of a sequence of frames of motion data captured by the motion sensor;
• processing the audio information and the motion information to recognise an activity of said user by:
• inputting the audio information and the motion information into a model configured to:
• receive the audio features and the motion features and return an activity recognition notification indicating said activity;
• and
• receiving, from the model, the activity recognition notification indicating said activity;
• and
• outputting the activity recognition notification indicating said activity.