Back to results
US 2026/0253616 A1Published application
System
AssigneeSoftBank Group Corp.
InventorAtsushi NANSEN
Life of this patent
Estimating the term…
Abstract
The system according to the embodiment comprises a receiving unit, an analysis unit, a generation unit, a playback unit, and a formatting unit. The receiving unit loads a video. The analysis unit analyzes the video loaded by the receiving unit. The generation unit generates minutes based on the video analyzed by the analysis unit. The playback unit plays back the video based on a search word. The formatting unit generates minutes in a specified format.
Claims20
A system comprising:
- a communication interface configured to communicate with a client terminal via a packet-switched network;
- a memory storing a video analysis model and a text generation model, each obtained by machine learning;
- and
- circuitry configured to:
- receive, from the client terminal via the communication interface, video data representing a video captured by a camera of the client terminal;
- analyze the video data by inputting the video data into the video analysis model to extract, for each segment of the video data, a feature vector and an importance score;
- generate minutes data by inputting the feature vector and the importance score into the text generation model to produce at least one of a highlight timestamp sequence or a summary text;
- receive, from the client terminal via the communication interface, query data representing a search word;
- identify a playback position in the video data by comparing the query data with the feature vector;
- and
- transmit the minutes data and the playback position to the client terminal via the communication interface and the packet-switched network.
8 more elements
A system comprising:
- a communication interface configured to communicate with a client terminal via a packet-switched network;
- a processor;
- a random-access memory;
- a memory storing a video analysis model comprising a convolutional neural network and a recurrent neural network, a text generation model comprising a Transformer-based summarization model, and an emotion identification model;
- and
- circuitry configured to:
- receive, from the client terminal via the communication interface, video data representing a video;
- extract audio data from the video data and convert the audio data into speech text by inputting the audio data into the recurrent neural network;
- analyze the video data by inputting frame data into the convolutional neural network to detect faces and scene transitions, and inputting the speech text into the text generation model to extract, for each segment, a feature vector and an importance score;
- estimate an emotion of a user by applying the emotion identification model to sensor data received from the client terminal;
- generate minutes data comprising at least one of a highlight timestamp sequence or a summary text based on the importance score and the estimated emotion;
- and
- transmit the minutes data to the client terminal via the communication interface.
10 more elements
A method comprising:
- receiving, from a client terminal via a communication interface, video data representing a video captured by a camera of the client terminal;
- analyzing the video data by inputting the video data into a video analysis model obtained by machine learning to extract, for each segment of the video data, a feature vector and an importance score;
- generating minutes data by inputting the feature vector and the importance score into a text generation model obtained by machine learning to produce at least one of a highlight timestamp sequence or a summary text;
- receiving, from the client terminal via the communication interface, query data representing a search word;
- identifying a playback position in the video data by comparing the query data with the feature vector;
- and
- transmitting the minutes data and the playback position to the client terminal via the communication interface and a packet-switched network.
4 more elements
Dependencies are inferred from the claim text.