Computer-implemented event detection using sonification

A system for event detection, the system comprising:

• one or more processors;
• and
• one or more memory devices coupled to the one or more processors, the one or more memory devices storing instructions that are executable by the one or more processors to perform operations including:
• obtaining multimedia data including multiple frames of video data and audio data corresponding to the video data;
• processing the multiple frames of video data to detect at least one object depicted in the video data and to track the at least one object between two or more frames of the multiple frames;
• generating sonification audio data representing a position of the at least one object in the two or more frames, movement of the at least one object in the two or more frames, or both the position and the movement of the at least one object in the two or more frames;
• generating combined audio data including the audio data and the sonification audio data;
• and
• providing input based on the combined audio data to a trained event classifier to detect an event represented in the multimedia data.

A computer program product for event detection, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processors to perform operations including:

• obtaining multimedia data including multiple frames of video data and audio data corresponding to the video data;
• processing the multiple frames of video data to detect at least one object depicted in the video data and to track the at least one object between two or more frames of the multiple frames;
• generating sonification audio data representing a position of the at least one object in the two or more frames, movement of the at least one object in the two or more frames, or both the position and the movement of the at least one object in the two or more frames;
• generating combined audio data including the audio data and the sonification audio data;
• and
• providing input based on the combined audio data to a trained event classifier to detect an event represented in the multimedia data.