Video content indexing and searching

A system comprising:

• a processor;
• and
• memory storing instructions which, when executed by the processor, cause the system to perform operations comprising:
• dividing a decoded first video into a first plurality of frames;
• discarding at least one frame from the first plurality of frames, wherein a first color histogram corresponding to the at least one frame is different from a second color histogram corresponding to a reference frame of the first plurality of frames by at least a threshold value;
• for each remaining frame of the first plurality of frames in the first video, identifying:
• a first global feature, and
• a first plurality of local features;
• clustering the first plurality of local features around a first plurality of cluster centers;
• converting, based on the first plurality of cluster centers, the first plurality of local features and into a first plurality of binary signatures;
• generating an index that maps the first plurality of cluster centers and the first plurality of binary signatures to the first plurality of frames;
• receiving a search request associated with a second video;
• identifying, for a frame in the second video, identifying a second global feature and a second plurality of local features;
• identifying, for the frame in the second video and based on the second plurality of local features, a second cluster center;
• converting, for the frame in the second video and based on the second cluster center, the second plurality of local features into a second plurality of binary signatures;
• and
• selecting, from the index and based on the second cluster center and the second plurality of binary signatures, a candidate video.

A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to perform operations comprising:

• dividing a decoded first video into a first plurality of frames;
• discarding at least one frame from the first plurality of frames, wherein a first color histogram corresponding to the at least one frame is different from a second color histogram corresponding to a reference frame of the first plurality of frames by at least a threshold value;
• for each remaining frame of the first plurality of frames in the first video, identifying:
• a first global feature, and
• a first plurality of local features;
• clustering the first plurality of local features around a first plurality of cluster centers;
• converting, based on the first plurality of cluster centers, the first plurality of local features and into a first plurality of binary signatures;
• generating an index that maps the first plurality of cluster centers and the first plurality of binary signatures to the first plurality of frames;
• receiving a search request associated with a second video;
• identifying, for a frame in the second video, identifying a second global feature and a second plurality of local features;
• identifying, for the frame in the second video and based on the second plurality of local features, a second cluster center;
• converting, for the frame in the second video and based on the second cluster center, the second plurality of local features into a second plurality of binary signatures;
• and
• selecting, from the index and based on the second cluster center and the second plurality of binary signatures, a candidate video.

A method comprising:

• dividing a decoded first video into a first plurality of frames;
• applying a filter to the first plurality of frames to reject an edge area of a frame in the first plurality of frames, wherein the edge area has a brightness value below a threshold;
• for each frame of the first plurality of frames in the first video, identifying:
• a first global feature, and
• a first plurality of local features;
• clustering the first plurality of local features around a first plurality of cluster centers;
• converting, based on the first plurality of cluster centers, the first plurality of local features and into a first plurality of binary signatures;
• generating an index that maps the first plurality of cluster centers and the first plurality of binary signatures to the first plurality of frames;
• receiving a search request associated with a second video;
• identifying, for a frame in the second video, identifying a second global feature and a second plurality of local features;
• identifying, for the frame in the second video and based on the second plurality of local features, a second cluster center;
• converting, for the frame in the second video and based on the second cluster center, the second plurality of local features into a second plurality of binary signatures;
• and
• selecting, from the index and based on the second cluster center and the second plurality of binary signatures, a candidate video.

A method comprising:

• for each frame of a first plurality of frames in a first video, identifying:
• a first global feature, and
• a first plurality of local features;
• clustering the first plurality of local features around a first plurality of cluster centers, wherein the clustering comprises:
• identifying, among the first plurality of local features, a subset of local features having descriptors that are proximate to each other above a threshold, and
• identifying a cluster center that is a coordinate-wise mean of the descriptors of the subset of local features;
• converting, based on the first plurality of cluster centers, the first plurality of local features and into a first plurality of binary signatures;
• generating an index that maps the first plurality of cluster centers and the first plurality of binary signatures to the first plurality of frames;
• receiving a search request associated with a second video;
• identifying, for a frame in the second video, identifying a second global feature and a second plurality of local features;
• identifying, for the frame in the second video and based on the second plurality of local features, a second cluster center;
• converting, for the frame in the second video and based on the second cluster center, the second plurality of local features into a second plurality of binary signatures;
• and
• selecting, from the index and based on the second cluster center and the second plurality of binary signatures, a candidate video.

A method comprising:

• for each frame of a first plurality of frames in a first video, identifying:
• a first global feature, and
• a first plurality of local features;
• clustering the first plurality of local features around a first plurality of cluster centers;
• converting, based on the first plurality of cluster centers, the first plurality of local features and into a first plurality of binary signatures, wherein the converting comprises:
• projecting first descriptors of the first plurality of local features onto second descriptors by using a projection matrix, wherein each of the second descriptors has a smaller dimension than a corresponding descriptor of the first descriptors,
• selecting a median value of the second descriptors, and
• performing coordinate-wise binarization of a vector difference between the second descriptors and the median value;
• generating an index that maps the first plurality of cluster centers and the first plurality of binary signatures to the first plurality of frames;
• receiving a search request associated with a second video;
• identifying, for a frame in the second video, identifying a second global feature and a second plurality of local features;
• identifying, for the frame in the second video and based on the second plurality of local features, a second cluster center;
• converting, for the frame in the second video and based on the second cluster center, the second plurality of local features into a second plurality of binary signatures;
• and
• selecting, from the index and based on the second cluster center and the second plurality of binary signatures, a candidate video.

A method comprising:

• for each frame of a first plurality of frames in a first video, identifying:
• a first global feature, and
• a first plurality of local features;
• clustering the first plurality of local features around a first plurality of cluster centers;
• converting, based on the first plurality of cluster centers, the first plurality of local features and into a first plurality of binary signatures;
• generating an index that maps the first plurality of cluster centers and the first plurality of binary signatures to the first plurality of frames;
• receiving a search request associated with a second video;
• identifying, for a frame in the second video, identifying a second global feature and a second plurality of local features;
• identifying, for the frame in the second video and based on the second plurality of local features, a second cluster center;
• converting, for the frame in the second video and based on the second cluster center, the second plurality of local features into a second plurality of binary signatures;
• and
• selecting, from the index and based on the second cluster center and the second plurality of binary signatures, a candidate video, wherein the selecting comprises:
• identifying, from the index and based on the second cluster center and the second plurality of binary signatures, a plurality of candidate videos for the frame in the second video;
• ranking the plurality of candidate videos in relation to the frame in the second video;
• removing, from the plurality of candidate videos, at least one candidate video that does not satisfy a threshold condition;
• assigning votes to the plurality of candidate videos based on histogram cells associated with the plurality of candidate videos in relation to the frame of the second video;
• and
• selecting, among the plurality of candidate videos, the candidate video that has a highest vote.