# US 2022/0019805 A1 · Video watermark identification method and apparatus, device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0019805 A1
- Document ID: US20220019805A1
- Status: Published application
- Kind: A1
- Application number: 17/299,726
- Filed: 2019-12-03
- Published: 2022-01-20
- Earliest claimed: 2018-12-03
- Assignees: Bigo Technology Pte. Ltd.
- Applicants: Bigo Technology Pte. Ltd.
- Inventors: Yu Zou; Xuan Yang; Zhenqiang Liu; Yue Pan; Zhen Li
- Web page: https://patentgrep.com/patent/US20220019805A1

## CPC

- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/23: Clustering techniques
- G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes}
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/8358: involving watermark {(protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467)}
- G06V 10/762: using clustering, e.g. of similar faces in social networks
- G06V 2201/07: Target detection
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Disclosed are a video watermark identification method and apparatus, device, and a storage medium. The method includes: each video frame of multiple video frames of a video is divided into multiple image blocks to obtain an image sequence corresponding to the each video frame; multiple image sequences corresponding to the multiple video frames are input to a target detection model to obtain a classification result of each image block, and a feature vector of the video is obtained according to all classification results of the multiple image blocks; and the feature vector of the video is input to a watermark identification model to obtain a watermark identification probability output by the watermark identification model, and the video containing a watermark is determining in a case where the watermark identification probability is greater than or equal to a probability threshold.

## Claims

### Claim 1

A video watermark identification method, comprising:
dividing each video frame of a plurality of video frames of a video into a plurality of image blocks to obtain an image sequence corresponding to the each video frame; inputting a plurality of image sequences corresponding to the plurality of video frames to a target detection model to obtain a classification result of each image block of the plurality of image blocks, and obtaining a feature vector of the video according to all classification results of the plurality of image blocks; and
inputting the feature vector of the video to a watermark identification model to obtain a watermark identification probability output by the watermark identification model, and determining that the video contains a watermark in a case where the watermark identification probability is greater than or equal to a probability threshold.

### Claim 2 (depends on 1)

The method of claim 1, wherein the each image block has a plurality of classification results;
obtaining the feature vector of the video according to the all classification results of the plurality of image blocks comprises:
determining a watermark classification result of the each image block according to the plurality of classification results of the each image block of the each video frame;
obtaining a feature vector corresponding to the each video frame according to all watermark classification results of the plurality of image blocks of the each video frame; and
obtaining the feature vector of the video according to a plurality of feature vectors corresponding to the plurality of video frames.

### Claim 3 (depends on 2)

The method of claim 2, wherein the watermark classification result comprises a watermark containing probability;
obtaining the feature vector corresponding to the each video frame according to the all watermark classification results of the plurality of image blocks of the each video frame comprises:
sorting watermark classification results of the plurality of image blocks of the each video frame according to watermark containing probabilities of the plurality of image blocks of the each video frame; and
determining the feature vector of the each video frame from a sorting result corresponding to the each video frame.

### Claim 4 (depends on 2)

The method of claim 2, wherein the watermark classification result comprises a watermark containing probability;
obtaining the feature vector corresponding to the each video frame according to the all watermark classification results of the plurality of image blocks of the each video frame comprises:
in a case where the each image block of the each video frame has a plurality of water classification results, sorting the plurality of watermark classification results of the each image block according to a plurality of water containing probabilities of the each image block of the each video frame;
selecting part of the watermark classification results from watermark classification results of the plurality of image blocks of the each video frame according to a sorting result and sorting the part of the watermark classification results; and
determining the feature vector of the each video frame from the part of the watermark classification results sorted corresponding to the each video frame.

### Claim 5 (depends on 1)

The method of claim 1, wherein the target detection model is trained in following manners:
acquiring a first training sample, wherein the first training sample comprises a plurality of training images, classification categories of the plurality of training images, and pieces of position information of the plurality of training images;
dividing each training image of the plurality of training images into a plurality of first training image blocks, obtaining a classification category of each first training image block of the each training image according to a classification category of the each training image, and obtaining a piece of position information of the each first training image block of the each training image according to a piece of position information of the each training image; and
taking all the plurality of first training image blocks as an input variable, taking all classification categories of the plurality of first training image blocks and all pieces of position information of the plurality of first training image blocks as an output variable, and training a classifier model to obtain the target detection model.

### Claim 6 (depends on 5)

The method of claim 5, wherein taking the all the plurality of first training image blocks as the input variable, taking the all the classification categories of the plurality of first training image blocks and the all pieces of position information of the plurality of first training image blocks as the output variable, and training the classifier model to obtain the target detection model comprises:
acquiring a piece of size information of the each first training image block;
performing a clustering analysis on all pieces of size information of the plurality of first training image blocks, and determining a prior frame of the each first training image block; and
taking the all the plurality of first training image blocks and all prior frames of the plurality of first training image blocks as the input variable, taking the all the classification categories of the plurality of first training image blocks and the all pieces of position information of the plurality of first training image blocks as the output variable, and training the classifier model to obtain the target detection model.

### Claim 7 (depends on 1)

The method of claim 1, wherein the watermark identification model is trained in following manners:
acquiring a second training sample, wherein the second training sample comprises a training video and a classification category of the training video;
dividing each training video frame of a plurality of training video frames of the training video into a plurality of second training image blocks to obtain an image sequence corresponding to the each training video frame;
inputting a plurality of image sequences corresponding to the plurality of training video frames into the target detection model to obtain a classification result of each second training image block of the plurality of second training image blocks, and obtaining a feature vector of the training video according to all classification results of the plurality of second training image blocks; and
taking the feature vector of the training video as an input variable, taking the classification category of the training video as an output variable, and training an extreme gradient boosting (Xgboost) model to obtain the watermark identification model.

### Claim 8 (depends on 7)

The method of claim 7, wherein the each second training image block has a plurality of classification results;
obtaining the feature vector of the training video according to the all classification results of the plurality of second training image blocks comprises:
determining a watermark classification result of the each second training image block according to the plurality of classification results of the each second training image block of the each training video frame;
obtaining a feature vector corresponding to the each training video frame according to all watermark classification results of the plurality of second training image blocks of the each training video frame; and
obtaining a feature vector of the training video according to a plurality of feature vectors corresponding to the plurality of training video frames.

### Claim 9 (depends on 8)

The method of claim 8, wherein the watermark classification result comprises a watermark containing probability;
obtaining the feature vector corresponding to the each training video frame according to the all watermark classification results of the plurality of second training image blocks of the each training video frame comprises:
sorting watermark classification results of the plurality of second training image blocks of the each training video frame according to watermark containing probabilities of the plurality of second training image blocks of the each training video frame; and
determining the feature vector of the each training video frame from a sorting result corresponding to the each training video frame.

### Claim 10 (depends on 8)

The method of claim 8, wherein the watermark classification result comprises a watermark containing probability;
obtaining the feature vector corresponding to the each training video frame according to the all watermark classification results of the plurality of second training image blocks of the each training video frame comprises:
in a case where the each second training image block of the each training video frame has a plurality of watermark classification results, sorting the plurality of watermark classification results of the each second training image block according to a plurality of watermark containing probabilities of the each second training image block of the each training video frame;
selecting part of the watermark classification results from watermark classification results of the plurality of second training image blocks of the each training video frame according to a sorting result and sorting the part of the watermark classification results; and
determining the feature vector of the each training video frame from the part of the watermark classification results sorted corresponding to the each training video frame.

### Claim 11

A video watermark identification apparatus, comprising:
an image sequence acquisition module, which is configured to divide each video frame of a plurality of video frames of a video into a plurality of image blocks to obtain an image sequence corresponding to the each video frame;
a video feature vector acquisition module, which is configured to input a plurality of image sequences corresponding to the plurality of video frames to a target detection model to obtain a classification result of each image block of the plurality of image blocks, and obtain a feature vector of the video according to all classification results of the plurality of image blocks; and
a watermark identification result determination module, which is configured to input the feature vector of the video to a watermark identification model to obtain a watermark identification probability output by the watermark identification model, and determine that the video contains a watermark in a case where the watermark identification probability is greater than or equal to a probability threshold.

### Claim 12 (depends on 1)

A device, comprising:
at least one processor; and
a memory configured to store at least one program;
wherein the at least one program, when executed by the at least one processor, causes the at least one processor to implement the method of claim 1.

### Claim 13 (depends on 1)

A computer-readable storage medium, stored therein computer programs, wherein the computer programs, when executed by a processor, implement the method of claim 1.

### Claim 14 (depends on 2)

The method of claim 2, wherein the target detection model is trained in following manners:
acquiring a first training sample, wherein the first training sample comprises a plurality of training images, classification categories of the plurality of training images, and pieces of position information of the plurality of training images;
dividing each training image of the plurality of training images into a plurality of first training image blocks, obtaining a classification category of each first training image block of the each training image according to a classification category of the each training image, and obtaining a piece of position information of the each first training image block of the each training image according to a piece of position information of the each training image; and
taking all the plurality of first training image blocks as an input variable, taking all classification categories of the plurality of first training image blocks and all pieces of position information of the plurality of first training image blocks as an output variable, and training a classifier model to obtain the target detection model.

### Claim 15 (depends on 3)

The method of claim 3, wherein the target detection model is trained in following manners:
acquiring a first training sample, wherein the first training sample comprises a plurality of training images, classification categories of the plurality of training images, and pieces of position information of the plurality of training images;
dividing each training image of the plurality of training images into a plurality of first training image blocks, obtaining a classification category of each first training image block of the each training image according to a classification category of the each training image, and obtaining a piece of position information of the each first training image block of the each training image according to a piece of position information of the each training image; and
taking all the plurality of first training image blocks as an input variable, taking all classification categories of the plurality of first training image blocks and all pieces of position information of the plurality of first training image blocks as an output variable, and training a classifier model to obtain the target detection model.

### Claim 16 (depends on 4)

The method of claim 4, wherein the target detection model is trained in following manners:
acquiring a first training sample, wherein the first training sample comprises a plurality of training images, classification categories of the plurality of training images, and pieces of position information of the plurality of training images;
dividing each training image of the plurality of training images into a plurality of first training image blocks, obtaining a classification category of each first training image block of the each training image according to a classification category of the each training image, and obtaining a piece of position information of the each first training image block of the each training image according to a piece of position information of the each training image; and
taking all the plurality of first training image blocks as an input variable, taking all classification categories of the plurality of first training image blocks and all pieces of position information of the plurality of first training image blocks as an output variable, and training a classifier model to obtain the target detection model.

### Claim 17 (depends on 2)

The method of claim 2, wherein the watermark identification model is trained in following manners:
acquiring a second training sample, wherein the second training sample comprises a training video and a classification category of the training video;
dividing each training video frame of a plurality of training video frames of the training video into a plurality of second training image blocks to obtain an image sequence corresponding to the each training video frame;
inputting a plurality of image sequences corresponding to the plurality of training video frames into the target detection model to obtain a classification result of each second training image block of the plurality of second training image blocks, and obtaining a feature vector of the training video according to all classification results of the plurality of second training image blocks; and
taking the feature vector of the training video as an input variable, taking the classification category of the training video as an output variable, and training an extreme gradient boosting (Xgboost) model to obtain the watermark identification model.

### Claim 18 (depends on 3)

The method of claim 3, wherein the watermark identification model is trained in following manners:
acquiring a second training sample, wherein the second training sample comprises a training video and a classification category of the training video;
dividing each training video frame of a plurality of training video frames of the training video into a plurality of second training image blocks to obtain an image sequence corresponding to the each training video frame;
inputting a plurality of image sequences corresponding to the plurality of training video frames into the target detection model to obtain a classification result of each second training image block of the plurality of second training image blocks, and obtaining a feature vector of the training video according to all classification results of the plurality of second training image blocks; and
taking the feature vector of the training video as an input variable, taking the classification category of the training video as an output variable, and training an extreme gradient boosting (Xgboost) model to obtain the watermark identification model.

### Claim 19 (depends on 4)

The method of claim 4, wherein the watermark identification model is trained in following manners:
acquiring a second training sample, wherein the second training sample comprises a training video and a classification category of the training video;
dividing each training video frame of a plurality of training video frames of the training video into a plurality of second training image blocks to obtain an image sequence corresponding to the each training video frame;
inputting a plurality of image sequences corresponding to the plurality of training video frames into the target detection model to obtain a classification result of each second training image block of the plurality of second training image blocks, and obtaining a feature vector of the training video according to all classification results of the plurality of second training image blocks; and
taking the feature vector of the training video as an input variable, taking the classification category of the training video as an output variable, and training an extreme gradient boosting (Xgboost) model to obtain the watermark identification model.

### Claim 20 (depends on 5)

The method of claim 5, wherein the watermark identification model is trained in following manners:
acquiring a second training sample, wherein the second training sample comprises a training video and a classification category of the training video;
dividing each training video frame of a plurality of training video frames of the training video into a plurality of second training image blocks to obtain an image sequence corresponding to the each training video frame;
inputting a plurality of image sequences corresponding to the plurality of training video frames into the target detection model to obtain a classification result of each second training image block of the plurality of second training image blocks, and obtaining a feature vector of the training video according to all classification results of the plurality of second training image blocks; and
taking the feature vector of the training video as an input variable, taking the classification category of the training video as an output variable, and training an extreme gradient boosting (Xgboost) model to obtain the watermark identification model.
