Back to results
US 2022/0067385 A1Published application
Multimodal game video summarization with metadata
AssigneeSony Interactive Entertainment Inc.
InventorsLakshmish Kaushik, Saket Kumar, Jaekwon Yoo, Kevin Zhang, Soheil Khorram, Sharath Rao, Ravi Sundaram
Life of this patent
Estimating the term…
Abstract
Video and audio from a computer simulation are processed by a machine learning engine to identify candidate segments of the simulation for use in a video summary of the simulation. Text input is then used to reinforce whether a candidate segment should be included in the video summary. Metadata can be added to the summary showing game summary information.
Claims20
An apparatus comprising:
- at least one processor programmed with instructions to:
- receive audio-video (AV) data;
- provide a video summary of the AV data that is shorter than the AV data at least in part by:
- input to a machine learning (ML) engine first modality data;
- input to the ML engine second modality data;
- receive the video summary of the AV data from the ML engine responsive to the inputting of the first and second modality data;
- and
- present in the video data metadata aligned in time with the first and second modality data such that the metadata is perceptible in the video summary.
5 more elements
A method comprising:
- identifying an audio-video (AV) entity;
- using audio from the AV entity, identifying plural first candidate segments of the AV entity for establishing a summary of the entity;
- using video from the AV entity, identifying plural second candidate segments of the AV entity for establishing a summary of the entity;
- using the at least some of the plural first and second candidate segments, generating a video summary of the AV entity that is shorter than the AV entity;
- and
- presenting in the video summary, in human-perceptible form, metadata related to the video summary.
3 more elements
An assembly, comprising:
- at least one display apparatus configured to present an audio-video (AV) computer game;
- at least one processor associated with the display apparatus and configured with instructions to execute a machine learning (ML) engine to generate a video summary of the computer game that is shorter than the computer game;
- and
- the instructions being executable to present in the video data metadata derived from game play and/or from the ML engine.
1 more element
Dependencies are inferred from the claim text.