System

A system comprising:

• circuitry configured to:
• acquire, via a communication interface operating over a packet-switched network supporting a transport-layer protocol with acknowledgment and retransmission control, time-synchronized audio information sampled at a predetermined sampling rate and video information captured at a predetermined frame rate from an environment, and store the audio information and the video information in a time-indexed data structure;
• perform compression encoding on the audio information using a transform-based audio codec and on the video information using a motion-compensated prediction algorithm, segment compressed streams into data units of a predetermined duration, and generate a data set comprising a header region storing identification information and time range and a payload region storing the compressed streams;
• execute speech recognition processing on the audio information by extracting acoustic features comprising Mel-frequency cepstral coefficients and log-Mel spectrograms from overlapping windows of audio samples, and inputting the acoustic features to a neural network model of an encoder-decoder architecture with attention mechanisms trained using a cross-entropy loss function to generate character information as a sequence of tokens each associated with a speaker label, a start time, and an end time;
• select a prompt template from a plurality of stored prompt templates on the basis of session attribute information comprising at least one of a session type, a target attribute, and attendance information, fill template variables in the selected prompt template, and construct input data comprising the prompt sentence and the character information;
• input the input data to a generative AI model implemented as a transformer-based sequence-to-sequence model comprising a multi-layer self-attention mechanism, feedforward layers, and layer normalization, execute inference using a decoding strategy comprising at least one of beam search and top-k sampling, and obtain a generation result parsed into structured information comprising at least one of key point data, utterance organization data, and task data;
• execute optical character recognition on regions of interest in the video information to detect equations, labels, and text, and execute figure recognition processing comprising at least one of edge detection, contour extraction, and shape classification to identify graphical elements, and associate the detected display content with the character information and the generation result on the basis of the time information;
• integrate the structured information and visual analysis results into integrated information formatted as a structured document comprising text sections, inline image references, and navigational links encoding the time information;
• and
• generate notification information including the integrated information on the basis of user identification information and an attendance status, and transmit the notification information to a user terminal via the packet-switched network.

A method performed by circuitry of a system, the method comprising:

• acquiring, via a communication interface coupled to a packet-switched network, time-synchronized audio information and video information captured from an environment;
• performing compression encoding on the audio information and the video information and generating a data set associated with identification information and time information;
• executing speech recognition processing on the audio information to generate character information stored in association with the time information;
• generating a prompt sentence for a generative AI model on the basis of the character information, the time information, and session attribute information;
• constructing input data comprising the prompt sentence and the character information, inputting the input data to the generative AI model, and obtaining a generation result including structured information comprising at least one of key point data, utterance organization data, and task data;
• executing figure recognition processing and character recognition processing on the video information and associating detected display content with the character information and the generation result on the basis of the time information;
• integrating the structured information and visual analysis results into integrated information;
• and
• transmitting notification information including the integrated information to a user terminal via the packet-switched network on the basis of user identification information.