# US 2026/0094438 A1 · Method and system for real-time event summarization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0094438 A1
- Document ID: US20260094438A1
- Status: Published application
- Kind: A1
- Application number: 18/903,669
- Filed: 2024-10-01
- Published: 2026-04-02
- Earliest claimed: 2024-10-01
- Assignees: Accenture Global Solutons Limited
- Applicants: Accenture Global Solutons Limited
- Inventors: Kamal MANNAR; Nita WANG; Hayley Skye DARUKHANAVALA; Lan GUAN; Elizabeth Ann KEATING
- Web page: https://patentgrep.com/patent/US20260094438A1

## CPC

- G06V 20/44: {Event detection}
- G06V 10/82: using neural networks
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems and methods for summarizing a real-time event are disclosed herein. A system obtains a set of multimedia data feeds from image capturing devices, wherein the set of multimedia data feeds correspond to a real-time event. The system processes the obtained set of multimedia data feeds using model hyperparameters and sequences the processed set of multimedia data feeds in a predetermined order. The system also obtains one or more input prompts corresponding to the set of multimedia data feeds. The system generates an output representation of the real-time event by encoding the sequenced set of multimedia data feeds and the obtained one or more input prompts using a trained vision encoder model, wherein the generated output representation corresponds to a multi-resolution summary image of the real-time event at a time instance. The system also predicts one or more actions performed in the generated output representation using an action prediction model.

## Claims

### Claim 1

A system comprising:
a processor; and
a memory communicably coupled to the processor, wherein the memory comprises processor-executable instructions which, when executed by the processor, cause the processor to:
obtain a set of multimedia data feeds from a plurality of image capturing devices, wherein the set of multimedia data feeds correspond to a real-time event, and wherein the set of multimedia data feeds correspond to a time-series data captured at a plurality of time intervals and a multi-resolution data captured from the plurality of image capturing devices;
process the obtained set of multimedia data feeds using a plurality of model hyperparameters, wherein the plurality of hyperparameters comprise a frame rate, a domain specific semantic compression, a multimedia segment length, and overlap between multimedia segments;
sequence the processed set of multimedia data feeds in a predetermined order based on the time-series data and the multi-resolution data;
obtain at least one input prompt corresponding to the set of multimedia data feeds from at least one input source;
generate an output representation of the real-time event by encoding the sequenced set of multimedia data feeds and the obtained at least one input prompt using a trained vision encoder model, wherein the generated output representation corresponds to a multi-resolution summary image of the real-time event at a time instance;
predict at least one action performed in the generated output representation using an action prediction model, wherein the at least one action comprises at least one of an activity, a function, and a movement corresponding to the real-time event; and
output the predicted at least one action on a user interface of a user device.

### Claim 2 (depends on 1)

The system of claim 1, wherein the processor is configured to:
validate a model performance of the action prediction model based on key performance factors, wherein the key performance factors comprise a data sensitivity factor, a data specificity factor, and a ground truth level of the action prediction model; and
tune the action prediction model to generate an updated action based on results of validation.

### Claim 3 (depends on 1)

The system of claim 1, wherein to process the obtained set of multimedia data feeds using a plurality of model hyperparameters, the processor is configured to:
identify a type of multimedia data obtained by analyzing a file format, a data size, and contents of multimedia data;
identify at least one of a type of objects, a position of objects, gestures performed within the obtained set of multimedia data feeds, a text data, and an audio data using a computer vision model;
select at least one appropriate processing model for processing the obtained set of multimedia data feeds based on the identified type of multimedia data, and the identified type of objects, the position of objects, the gestures performed within the obtained set of multimedia data feeds, the text data, and the audio data; and
process the obtained set of multimedia data feeds using the selected at least one appropriate processing model.

### Claim 4 (depends on 3)

The system of claim 3, wherein the processor is configured to:
tune the plurality of model hyperparameters based on the selected at least one appropriate processing model, wherein the selected at least one appropriate processing model comprises a computer vision model and an audio model, wherein the computer vision model comprises at least one of an object detection, an object tracking, a person detection, a person tracking, a semantic segmentation, a semantic compression, a multi-camera person recognition, and a multi-camera object recognition, wherein the audio model comprises a noise reduction, a speech detection, and a speech diarization; and
retrain a vision encoder model based on the tuned plurality of model hyperparameters.

### Claim 5 (depends on 1)

The system of claim 1, wherein to obtain the at least one input prompt corresponding to the set of multimedia data feeds from at least one input source the processor is configured to:
obtain text prompts from at least one input source at real-time based on a type of the set of multimedia data feeds, wherein the at least one input source comprises one of a user input, and a model input.

### Claim 6 (depends on 1)

The system of claim 1, wherein to process the obtained set of multimedia data feeds using the plurality of model hyperparameters, the processor is configured to:
identify an event of interest within the set of multimedia data feeds captured from the plurality of image capturing devices;
determine a plurality of patterns corresponding to the identified event of interest with respect to a plurality of time instances by correlating each media frame with a subsequent media frame of the set of multimedia data feeds; and
process the obtained set of multimedia data feeds based on the determined plurality of patterns corresponding to the identified event of interest.

### Claim 7 (depends on 1)

The system of claim 1, wherein to generate the output representation of the real-time event by encoding the sequenced set of multimedia data feeds and the obtained at least one input prompt using the trained vision encoder model, the processor is configured to:
encode the sequenced set of multimedia data feeds using a computer vision encoder layer of the trained vision encoder model;
encode the obtained at least one input prompt using a word embedding layer of the trained vision encoder model;
correlate the encoded set of multimedia data feeds with the obtained at least one input prompt to identify an action of interest; and
generate the output representation of the real-time event based on the correlation, wherein the output representation indicates the action of interest.

### Claim 8 (depends on 1)

The system of claim 1, wherein to predict the at least one action performed in the generated output representation using the action prediction model, the processor is configured to:
identify at least one of a type of objects, a position of objects, gestures performed within the obtained set of multimedia data feeds using a computer vision model;
classify the set of multimedia data feeds into domain specific events based on the at least one of the type of the objects, the position of the objects, and the gestures performed;
generate a confidence score for each of the classified set of multimedia data feeds using the action prediction model; and
predict the at least one action performed in the generated output representation using the generated confidence score.

### Claim 9 (depends on 1)

The system of claim 1, wherein the processor is configured to:
determine at one pattern with an object within the obtained set of multimedia data feeds using the trained vision encoder model; and
detect a state of the object based on the determined at least one pattern, wherein the state of the object comprises one of a mental state and a physical state of the object.

### Claim 10

A method comprising:
obtaining, by a processor, a set of multimedia data feeds from a plurality of image capturing devices, wherein the set of multimedia data feeds correspond to a real-time event, and wherein the set of multimedia data feeds correspond to a time-series data captured at a plurality of time intervals and a multi-resolution data captured from the plurality of image capturing devices;
processing, by the processor, the obtained set of multimedia data feeds using a plurality of model hyperparameters, wherein the plurality of hyperparameters comprise a frame rate, a domain specific semantic compression, a multimedia segment length, and overlap between multimedia segments;
sequencing, by the processor, the processed set of multimedia data feeds in a predetermined order based on the time-series data and the multi-resolution data;
obtaining, by the processor, at least one input prompt corresponding to the set of multimedia data feeds from at least one input source;
generating, by the processor, an output representation of the real-time event by encoding the sequenced set of multimedia data feeds and the obtained at least one input prompt using a trained vision encoder model, wherein the generated output representation corresponds to a multi-resolution summary image of the real-time event at a time instance;
predicting, by the processor, at least one action performed in the generated output representation using an action prediction model, wherein the at least one action comprises at least one of an activity, a function, and a movement corresponding to the real-time event; and
outputting, by the processor, the predicted at least one action on a user interface of a user device.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
validating, by the processor, a model performance of the action prediction model based on key performance factors, wherein the key performance factors comprise a data sensitivity factor, a data specificity factor, and a ground truth level of the action prediction model; and
tuning, by the processor, the action prediction model to generate an updated action based on results of validation.

### Claim 12 (depends on 10)

The method of claim 10, wherein processing the obtained set of multimedia data feeds using a plurality of model hyperparameters comprises:
identifying, by the processor, a type of multimedia data obtained by analyzing a file format, a data size, and contents of multimedia data;
identifying, by the processor, at least one of a type of objects, a position of objects, gestures performed within the obtained set of multimedia data feeds, a text data, and an audio data using a computer vision model;
selecting, by the processor, at least one appropriate processing model for processing the obtained set of multimedia data feeds based on the identified type of multimedia data, and the identified type of objects, the position of objects, the gestures performed within the obtained set of multimedia data feeds, the text data, and the audio data; and
processing, by the processor, the obtained set of multimedia data feeds using the selected at least one appropriate processing model.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
tuning, by the processor, the plurality of model hyperparameters based on the selected at least one appropriate processing model, wherein the selected at least one appropriate processing model comprises a computer vision model and an audio model, wherein the appropriate computer vision model comprises at least one of an object detection, an object tracking, a person detection, a person tracking, a semantic segmentation, a semantic compression, a multi-camera person recognition, and a multi-camera object recognition, wherein the audio model comprises a noise reduction, a speech detection, and a speech diarization; and
retraining, by the processor, a vision encoder model based on the tuned plurality of model hyperparameters.

### Claim 14 (depends on 10)

The method of claim 10, wherein obtaining the at least one input prompt corresponding to the set of multimedia data feeds from at least one input source comprises:
obtaining, by the processor, text prompts from at least one input source at real-time based on a type of the set of multimedia data feeds, wherein the at least one input source comprises one of a user input, and a model input.

### Claim 15 (depends on 10)

The method of claim 10, wherein processing the obtained set of multimedia data feeds using the plurality of model hyperparameters comprises:
identifying, by the processor, an event of interest within the set of multimedia data feeds captured from the plurality of image capturing devices;
determining, by the processor, a plurality of patterns corresponding to the identified event of interest with respect to a plurality of time instances by correlating each media frame with a subsequent media frame of the set of multimedia data feeds; and
processing, by the processor, the obtained set of multimedia data feeds based on the determined plurality of patterns corresponding to the identified event of interest.

### Claim 16 (depends on 10)

The method of claim 10, wherein generating the output representation of the real-time event by encoding the sequenced set of multimedia data feeds and the obtained at least one input prompt using the trained vision encoder model comprises:
encoding, by the processor, the sequenced set of multimedia data feeds using a computer vision encoder layer of the trained vision encoder model;
encoding, by the processor, the obtained at least one input prompt using a word embedding layer of the trained vision encoder model;
correlating, by the processor, the encoded set of multimedia data feeds with the obtained at least one input prompt to identify an action of interest; and
generating, by the processor, the output representation of the real-time event based on the correlation, wherein the output representation indicates the action of interest.

### Claim 17 (depends on 10)

The method of claim 10, wherein predicting the at least one action performed in the generated output representation using the action prediction model comprises:
identifying, by the processor, at least one of a type of objects, a position of objects, gestures performed within the obtained set of multimedia data feeds using a computer vision model;
classifying, by the processor, the set of multimedia data feeds into domain specific events based on the at least one of the type of the objects, the position of the objects, and the gestures performed;
generating, by the processor, a confidence score for each of the classified set of multimedia data feeds using the action prediction model; and
predicting, by the processor, the at least one action performed in the generated output representation using the generated confidence score.

### Claim 18 (depends on 10)

The method of claim 10, further comprising:
determining, by the processor, at one pattern with an object within the obtained set of multimedia data feeds using the trained vision encoder model; and
detecting, by the processor, a state of the object based on the determined at least one pattern, wherein the state of the object comprises one of a mental state and a physical state of the object.

### Claim 19

A non-transitory computer readable medium comprising a processor-executable instructions that cause a processor to:
obtain a set of multimedia data feeds from a plurality of image capturing devices, wherein the set of multimedia data feeds correspond to a real-time event, and wherein the set of multimedia data feeds correspond to a time-series data captured at a plurality of time intervals and a multi-resolution data captured from the plurality of image capturing devices;
process the obtained set of multimedia data feeds using a plurality of model hyperparameters, wherein the plurality of hyperparameters comprise a frame rate, a domain specific semantic compression, a multimedia segment length, and overlap between multimedia segments;
sequence the processed set of multimedia data feeds in a predetermined order based on the time-series data and the multi-resolution data;
obtain at least one input prompt corresponding to the set of multimedia data feeds from at least one input source;
generate an output representation of the real-time event by encoding the sequenced set of multimedia data feeds and the obtained at least one input prompt using a trained vision encoder model, wherein the generated output representation corresponds to a multi-resolution summary image of the real-time event at a time instance;
predict at least one action performed in the generated output representation using an action prediction model, wherein the at least one action comprises at least one of an activity, a function, and a movement corresponding to the real-time event; and
output the predicted at least one action on a user interface of a user device.

### Claim 20 (depends on 19)

The non-transitory computer readable medium of claim 19, wherein the processor-executable instructions cause the processor to:
tune the plurality of model hyperparameters based on selected at least one appropriate processing model, wherein the selected at least one appropriate processing model comprises a computer vision model and an audio model, wherein the computer vision model comprises at least one of an object detection, an object tracking, a person detection, a person tracking, a semantic segmentation, a semantic compression, a multi-camera person recognition, and a multi-camera object recognition, wherein the audio model comprises a noise reduction, a speech detection, and a speech diarization; and
retrain a vision encoder model based on the tuned plurality of model hyperparameters.
