US 2026/0196040 A1 · MULTIMODAL CLASSIFICATION OF VIDEO DATA

Abstract

A computing system for multimodal classification of video data includes processing circuitry that implements a video data classification program configured to perform classification tasks in a plurality of classification categories. In an inference phase, the processing circuitry receives video data to be classified according to a selected classification category of the plurality of classification categories and extracts a plurality of frames and corresponding text features from the video data. A classification model selection engine is implemented to select a multimodal classification model from a library of multimodal classification models based on the selected classification category. The selected multimodal classification model is queried with a prompt to classify the plurality of frames and the corresponding text features according to the selected classification category. The processor receives a prediction score from the selected multimodal classification model, indicating whether the video data includes content in the selected classification category.

Claims20

Dependent claims

A computing system for multimodal classification of video data, the computing system comprising:

a computing device including processing circuitry configured to execute instructions using portions of associated memory to implement a video data classification program configured to perform classification tasks in a plurality of classification categories, wherein the processing circuitry is configured to:
in an inference phase:
receive video data to be classified according to a selected classification category of the plurality of classification categories;
extract a plurality of frames and corresponding text features from the video data;
implement a classification model selection engine to select a multimodal classification model from a library of multimodal classification models based on the selected classification category;
query the selected multimodal classification model with a prompt to classify the plurality of frames and the corresponding text features according to the selected classification category;
and
receive a response from the selected multimodal classification model, the response including a prediction score indicating a likelihood that the plurality of frames and the corresponding text features includes content in the selected classification category.

5 more elements

A method for multimodal classification of video data, the method comprising:

in an inference phase:
receiving video data to be classified according to a selected classification category of a plurality of classification categories;
extracting a plurality of frames and corresponding text features from the video data;
implementing a classification model selection engine to select a multimodal classification model from a library of multimodal classification models based on the selected classification category;
querying the selected multimodal classification model with a prompt to classify the plurality of frames and the corresponding text features according to the selected classification category;
and
receiving a response from the selected multimodal classification model, the response including a prediction score indicating a likelihood that the plurality of frames and the corresponding text features includes content in the selected classification category.

4 more elements

Dependencies are inferred from the claim text.

Multimodal classification of video data

Life of this patent

Multimodal classification of video data

Life of this patent

Abstract

Claims20