# US 2025/0095328 A1 · Unsupervised video segment clustering and boundary positioning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095328 A1
- Document ID: US20250095328A1
- Status: Published application
- Kind: A1
- Application number: 18/469,688
- Filed: 2023-09-19
- Published: 2025-03-20
- Earliest claimed: 2023-09-19
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Li Bo ZHANG; Zhe YAN; Rong ZHAO; Li Li GUAN; Hao Xiang WU
- Web page: https://patentgrep.com/patent/US20250095328A1

## CPC

- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06F 40/30: Semantic analysis
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/762: using clustering, e.g. of similar faces in social networks
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 20/47: {Detecting features for summarising video content}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Method and apparatus for video processing. A plurality of videos are accessed. An index is generated based on one or more key frames and one or more video features extracted from the plurality of video. A user input is processed to identify user intentions. The index is filtered to identify a plurality of candidate key frames based on the user intentions. A recommended video is generated based on the plurality of candidate key frames, further comprising selecting one or more additional frames based on processing the plurality of candidate key frames using one or more clustering operations, and aggregating the plurality of candidate key frames and the one or more additional frames.

## Claims

### Claim 1

A method comprising:
accessing a plurality of videos;
generating an index for the plurality of videos based on one or more key frames and one or more video features extracted from each respective video of the plurality of videos;
processing a user input to identify user intentions;
filtering the index to identify a plurality of candidate key frames based on the user intentions; and
generating a recommended video based on the plurality of candidate key frames, comprising:
selecting one or more additional frames based on processing the plurality of candidate key frames using one or more clustering operations; and
aggregating the plurality of candidate key frames and the one or more additional frames.

### Claim 2 (depends on 1)

The method of claim 1, wherein processing a user input to identify user intentions comprises:
converting the user input into textual data; and
processing the textual data to predict the user intentions using natural language processing (NLP) techniques.

### Claim 3 (depends on 1)

The method of claim 1, wherein the user input comprises at least one of a textual query, an audio file, an image, or a video file, and wherein the user intentions are represented as a row vector.

### Claim 4 (depends on 1)

The method of claim 1, wherein selecting one or more additional frames based on processing the plurality of candidate key frames using one or more clustering operations comprises:
for each candidate key frame from a respective candidate video, identifying the one or more additional frames from the respective candidate video upon determining that a similarity between each of the one or more additional frames and the candidate key frame exceeds a defined threshold, and
clustering the candidate key frame with the one or more additional frames from the respective candidate video to generate a video clip.

### Claim 5 (depends on 1)

The method of claim 1, wherein the one or more key frames for each respective video of the plurality of videos are determined by:
dividing each respective video into a plurality of frames;
extracting the one or more video features from each respective frame of the plurality of frames;
clustering the plurality of frames based on similarities between different frames of the plurality of frames, wherein the similarities are determined based on the one or more video features extracted from each respective frame; and
identifying one or more cluster centers as the one or more key frames to present each respective video.

### Claim 6 (depends on 1)

The method of claim 1, wherein the one or more video features extracted from each respective video comprises at least one of a global feature, or a local feature.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
prior to generating the recommended video based on the plurality of candidate key frames, recommending one or more leading video features from the one or more video features extracted from each respective video based on at least in part on the user intentions.

### Claim 8 (depends on 7)

The method of claim 7, wherein the one or more additional frames are selected based on the one or more recommended leading video features.

### Claim 9 (depends on 7)

The method of claim 7, wherein the one or more recommended leading video features comprise at least one of (i) color, (ii) brightness, (iii) texture, (iv) gray scale, (v) object tracing, (vi) scale-invariant feature transform (SIFT), or (vii) image semantics data.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
prior to generating the recommended video based on the plurality of candidate key frames, modifying the plurality of candidate key frames, further comprising:
removing a candidate key frame of the plurality of candidate key frames upon determining that the candidate key frame is an outlier among the plurality of candidate key frames; and
adding a key frame of the one or more key frames within the index upon determining that the key frame is visually similar to one candidate key frame of the plurality of candidate key frames.

### Claim 11

A system comprising:
one or more computer processors; and
one or more memories collectively containing one or more programs which when executed by the one or more computer processors performs an operation, the operation comprising:
accessing a plurality of videos;
generating an index for the plurality of videos based on one or more key frames and one or more video features extracted from each respective video of the plurality of videos;
processing a user input to identify user intentions;
filtering the index to identify a plurality of candidate key frames based on the user intentions; and
generating a recommended video based on the plurality of candidate key frames, comprising:
selecting one or more additional frames based on processing the plurality of candidate key frames using one or more clustering operations; and
aggregating the plurality of candidate key frames and the one or more additional frames.

### Claim 12 (depends on 11)

The system of claim 11, wherein processing a user input to identify user intentions comprises:
converting the user input into textual data; and
processing the textual data to predict the user intentions using natural language processing (NLP) techniques.

### Claim 13 (depends on 11)

The system of claim 11, wherein the user input comprises at least one of a textual query, an audio file, an image, or a video file, and wherein the one or more video features extracted from each respective video comprises at least one of a global feature, or a local feature.

### Claim 14 (depends on 11)

The system of claim 11, wherein selecting one or more additional frames based on processing the plurality of candidate key frames using one or more clustering operations comprises:
for each candidate key frame from a respective candidate video, identifying one or more additional frames from the respective candidate video upon determining that a similarity between each of the one or more additional frames and the candidate key frame exceeds a defined threshold, and
clustering the candidate key frame with the one or more additional frames from the respective candidate video to generate a video clip.

### Claim 15 (depends on 11)

The system of claim 11, wherein the operation further comprises:
prior to generating the recommended video based on the plurality of candidate key frames, recommending one or more leading video features from the one or more video features extracted from each respective video based on at least in part on the user intentions.

### Claim 16 (depends on 15)

The system of claim 15, wherein the one or more additional frames are selected based on the one or more recommended leading video features.

### Claim 17 (depends on 15)

The system of claim 15, wherein the one or more recommended leading video features comprise at least one of (i) color, (ii) brightness, (iii) texture, (iv) gray scale, (v) object tracing, (vi) scale-invariant feature transform (SIFT), or (vii) image semantics data.

### Claim 18 (depends on 11)

The system of claim 11, wherein the operation further comprises:
prior to generating a recommended video based on the plurality of candidate key frames, modifying the plurality of candidate key frames, further comprising:
removing a candidate key frame of the plurality of candidate key frames upon determining that the candidate key frame is an outlier among the plurality of candidate key frames; and
adding a key frame of the one or more key frames within the index upon determining that the key frame is visually similar to one candidate key frame of the plurality of candidate key frames.

### Claim 19

A computer program product comprising one or more computer-readable storage media collectively containing computer-readable program code that, when executed by operation of one or more computer processors, performs an operation comprising:
accessing a plurality of videos;
generating an index for the plurality of videos based on one or more key frames and one or more video features extracted from each respective video of the plurality of videos;
processing a user input to identify user intentions;
filtering the index to identify a plurality of candidate key frames based on the user intentions; and
generating a recommended video based on the plurality of candidate key frames, comprising:
selecting one or more additional frames based on processing the plurality of candidate key frames using one or more clustering operations; and
aggregating the plurality of candidate key frames and the one or more additional frames.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein processing a user input to identify user intentions comprises:
converting the user input into textual data; and
processing the textual data to predict the user intentions using natural language processing (NLP) techniques.
