System and method for multi-modal audio mining of telephone conversations
AssigneeGlobal Tel*Link Corporation
InventorsAndreas M. Olligschlaeger, James P. Rokosky
Life of this patent
Estimating the term…
Abstract
A system and method for the automated monitoring of inmate telephone calls as well as multi-modal search, retrieval and playback capabilities for said calls. A general term for such capabilities is multi-modal audio mining. The invention is designed to provide an efficient means for organizations such as correctional facilities to identify and monitor the contents of telephone conversations and to provide evidence of possible inappropriate conduct and/or criminal activity of inmates by analyzing monitored telephone conversations for events, including, but not limited to, the addition of third parties, the discussion of particular topics, and the mention of certain entities.
Claims20
A system for multi-modal audio mining of telephone data, the system comprising:
- one or more circuits and/or processors configured to:
- retrieve a data record from a call database, the data record including an audio inmate communication and metadata associated with the audio inmate communication;
- perform speech recognition of the audio inmate communication, the speech recognition detecting non-verbal characteristics of the audio inmate communication;
- generate a transcript of the audio inmate communication based on the speech recognition that includes the detected non-verbal characteristics;
- identify a plurality of related data records in the call database based on the metadata of the retrieved data record, the plurality of related data records being identified based in part on a similarity of the non-verbal characteristics of the audio inmate communication to non-verbal characteristics of communications included in the plurality of related data records;
- generate logical links between the retrieved data record and the plurality of related data records;
- generate a visual representation of data record relationships based on the logical links;
- and
- store the logical links and the transcript in association with each other in a second database.
A method for multi-modal audio mining of telephone data, the method comprising:
- retrieving a data record from a call database, the data record including an audio inmate communication and metadata associated with the audio inmate communication;
- performing speech recognition of the audio inmate communication, the speech recognition detecting non-verbal characteristics of the audio inmate communication;
- generating a transcript of the audio inmate communication based on the speech recognition that includes the detected non-verbal characteristics;
- identifying a plurality of related data records in the call database based on the metadata of the retrieved data record, the plurality of related data records being identified based in part on a similarity of the non-verbal characteristics of the audio inmate communication to non-verbal characteristics of communications included in the plurality of related data records;
- generating logical links between the retrieved data record and the plurality of related data records;
- generating a visual representation of data record relationships based on the logical links;
- and
- storing the logical links and the transcript in association with each other in a second database.
A system for visually representing relationships among inmate communications, the system comprising:
- a storage system that stores previous inmate communications;
- a retrieval subsystem configured to:
- receive a current audio inmate communication, the current audio inmate communication including a content portion and a metadata portion, the metadata portion including a plurality of metadata elements;
- perform speech recognition of the current audio inmate communication, the speech recognition detecting non-verbal characteristics of the current audio inmate communication;
- generate a transcript of the current audio inmate communication based on the speech recognition that includes the detected non-verbal characteristics;
- access the storage system;
- and
- retrieve a related inmate communication from among the stored previous inmate communications, the related inmate communication having a predetermined number of metadata elements that match corresponding ones of the plurality of metadata elements of the current inmate communication and having non-verbal characteristics of a communication that are substantially similar to non-verbal characteristics of the audio inmate communication;
- and
- a linking subsystem configured to:
- identify a logical link between the current inmate communication and the related inmate communication based on the matching metadata elements;
- generate a visual representation of a relationship between the current inmate communication and the related inmate communication that illustrates the logical link;
- and
- store the logical links and the transcript in association with each other in a second database.
Dependencies are inferred from the claim text.