Back to results
US 10,757,061 B1Granted
Cross-modal event summarization system based on network of networks representations
AssigneeHRL Laboratories, LLC
InventorsJiejun Xu, Samuel D. Johnson, Kang-Yu Ni
Life of this patent
Estimating the term…
Abstract
Described is a system for automated event summarization. A multi-layer network representing a multimodal data set is generated, where nodes within a given layer represent information tokens in a given modality. A topically diverse set of nodes is ranked and selected from each layer to represent temporal event highlights. Temporal event highlights are linked into storylines. Using the storylines, the system monitors a progression of an event or opinions regarding a topic. A temporal summary of the progression of the event or the opinions regarding the topic is generated.
Claims22
A system for automated event summarization, the system comprising:
- one or more processors and a non-transitory computer-readable medium having executable instructions encoded thereon such that when executed, the one or more processors perform operations of:
- connecting electronically with a plurality of media networks and automatically acquiring temporal event data related to an event from a plurality of multimodal data sources;
- using the temporal event data, generating a multi-layer network comprising:
- a plurality of layers, a plurality of nodes within each layer, at least one inter-layer edge connecting nodes between layers, and at least one intra-layer edge within each layer connecting nodes within the layer, wherein each layer comprises temporal event data from a distinct data source, wherein each node represents an information token, and wherein the nodes within each layer represent information tokens obtained from a distinct data source;
- forming, from the plurality of nodes, a plurality of tuples that maximize an objective function, wherein the nodes that form each tuple are connected via inter-layer edges;
- assigning a fitness score to each tuple according to the objective function;
- ranking the plurality of tuples according to their fitness scores;
- based on the ranking, selecting tuples in different modalities from each layer to represent temporal event highlights;
- linking the temporal event highlights into a temporal event progression of temporal event highlights;
- and
- electronically communicating the temporal event progression.
8 more elements
A computer implemented method for automated event summarization, the method comprising an act of:
- causing one or more processers to execute instructions encoded on a non-transitory computer-readable medium, such that upon execution, the one or more processors perform operations of:
- connecting electronically with a plurality of media networks and automatically acquiring temporal event data related to an event from a plurality of multimodal data sources;
- using the temporal event data, generating a multi-layer network comprising:
- a plurality of layers, a plurality of nodes within each layer, at least one inter-layer edge connecting nodes between layers, and at least one intra-layer edge within each layer connecting nodes within the layer, wherein each layer comprises temporal event data from a distinct data source, wherein each node represents an information token, and wherein the nodes within each layer represent information tokens obtained from a distinct data source;
- forming, from the plurality of nodes, a plurality of tuples that maximize an objective function, wherein the nodes that form each tuple are connected via inter-layer edges;
- assigning a fitness score to each tuple according to the objective function;
- ranking the plurality of tuples according to their fitness scores;
- based on the ranking, selecting tuples in different modalities from each layer to represent temporal event highlights;
- linking the temporal event highlights into a temporal event progression of temporal event highlights;
- and
- electronically communicating the temporal event progression.
8 more elements
A computer program product for automated event summarization, the computer program product comprising:
- computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having one or more processors for causing the processor to perform operations of:
- connecting electronically with a plurality of media networks and automatically acquiring temporal event data related to an event from a plurality of multimodal data sources;
- using the temporal event data, generating a multi-layer network comprising:
- a plurality of layers, a plurality of nodes within each layer, at least one inter-layer edge connecting nodes between layers, and at least one intra-layer edge within each layer connecting nodes within the layer, wherein each layer comprises temporal event data from a distinct data source, wherein each node represents an information token, and wherein the nodes within each layer represent information tokens obtained from a distinct data source;
- forming, from the plurality of nodes, a plurality of tuples that maximize an objective function, wherein the nodes that form each tuple are connected via inter-layer edges;
- assigning a fitness score to each tuple according to the objective function;
- ranking the plurality of tuples according to their fitness scores;
- based on the ranking, selecting tuples in different modalities from each layer to represent temporal event highlights;
- linking the temporal event highlights into a temporal event progression of temporal event highlights;
- and
- electronically communicating the temporal event progression.
8 more elements
Dependencies are inferred from the claim text.