# US 2025/0095362 A1 · Machine learning to detect fake videos

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095362 A1
- Document ID: US20250095362A1
- Status: Published application
- Kind: A1
- Application number: 18/467,160
- Filed: 2023-09-14
- Published: 2025-03-20
- Earliest claimed: 2023-09-14
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Mikael HILLBORG; Shaheer Jamal SYED
- Web page: https://patentgrep.com/patent/US20250095362A1

## CPC

- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06F 40/279: Recognition of textual entities
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 20/48: {Matching video sequences}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Method and apparatus for detection of fake videos. A statement asserting that a video is fake is accessed. One or more characteristics of the statement is identified, where the one or more characteristics comprises at least one of source information for the video, source information for an reference video, or one or more timestamps in the video. Consistency checks between audio and video of the video is performed. Video features of the video is examined to detect modifications. Overlapping clips between the video and the reference video is identified. The video is determined to be fake based at least in part on the one or more characteristics of the statement, the consistency checks, the video features, and the identified overlapping clips.

## Claims

### Claim 1

A method comprising:
accessing a statement asserting that a video is fake;
identifying one or more characteristics of the statement, wherein the one or more characteristics comprise at least one of source information for the video, source information for a reference video, or one or more timestamps in the video;
performing consistency checks between audio and video of the video;
examining one or more video features of the video to detect modifications;
identifying overlapping clips between the video and the reference video; and
determining that the video is fake based at least in part on the one or more characteristics of the statement, the consistency checks, the video features, and the identified overlapping clips.

### Claim 2 (depends on 1)

The method of claim 1, further comprising synchronizing the consistency checks, the video features, and the identified overlapping clips with the one or more timestamps within the video identified by the statement.

### Claim 3 (depends on 1)

The method of claim 1, wherein the statement further comprises one or more tags describing contents of the video.

### Claim 4 (depends on 1)

The method of claim 1, wherein accessing the statement alleging that the video is fake comprises analyzing texts within the statement using a binary classifier.

### Claim 5 (depends on 1)

The method of claim 1, wherein performing the consistency checks between audio and video of the video comprises:
extracting audio track of the video;
converting the audio track into a sequence of syllables, wherein the sequence of syllables comprises a plurality of syllables that are spoken at different time points within the video;
processing the video to create a plurality of sampling frames, wherein each respective sampling frame, of the plurality of sampling frame, is captured at a time point that a respective syllable is spoken; and
comparing each respective syllable, of the plurality of syllables, with each respective sampling frame, of the plurality of sampling frames, to generate a plurality of consistency values.

### Claim 6 (depends on 5)

The method of claim 5, wherein the plurality of consistency values is mapped onto a timeline, with each respective value represents a degree of consistency between the audio and video of the video at a time point that a respective syllable is spoken.

### Claim 7 (depends on 1)

The method of claim 1, wherein examining the one or more video features of the video to detect modifications comprises:
processing the video to create sampling frames at a defined interval;
analyzing each of the sampling frames to extract the one or more video features using one or more image processing algorithms, wherein the one or more vide features comprise at least one of (i) pixilation, (ii) shadows, (iii) colors, (iv) edges, or (iv) textures; and
determining the video has been manipulated based on processing the one or more video features using a machine learning model.

### Claim 8 (depends on 7)

The method of claim 7, wherein analyzing each of the sampling frames to extract the one or more video features further comprises passing outputs of the one or more image processing algorithms into a lowpass filter to remove high frequency noise in the extracted one or more video features.

### Claim 9 (depends on 1)

The method of claim 1, wherein identifying the overlapping clips between the video and the reference video comprises:
processing the video to create sampling frames at a defined interval;
hashing the sampling frames to generate a signature for the video;
retrieving a signature for the reference video from a database; and
comparing the signature for the video with the signature for the reference video to identify the overlapping clips between the video and the reference video.

### Claim 10 (depends on 9)

The method of claim 9, wherein retrieving the signature for the reference video from the database further comprises:
extracting one or more tags from the statements;
converting audio track of the video into written texts using one or more speech-to-text algorithms;
identifying one or more key entities from the written texts representing topics of the video; and
retrieving the signature for the reference video from the database based on the extracted tags and the identified key entities.

### Claim 11

A system, comprising:
one or more computer processors; and
one or more memories collectively containing one or more programs which when executed by the one or more computer processors performs an operation, the operation comprising:
accessing a statement asserting that a video is fake;
identifying one or more characteristics of the statement, wherein the one or more characteristics comprise source information for the video, source information for an reference video, and one or more timestamps in the video;
performing consistency checks between audio and video of the video;
examining one or more video features of the video to detect modifications;
identifying overlapping clips between the video and the reference video; and
determining that the video is fake based at least in part on the one or more characteristics of the statement, the consistency checks, the video features, and the identified overlapping clips.

### Claim 12 (depends on 11)

The system of claim 11, wherein the operation further comprises synchronizing the consistency checks, the video features, and the identified overlapping clips with the one or more timestamps within the video identified by the statement.

### Claim 13 (depends on 11)

The system of claim 11, wherein performing the consistency checks between audio and video of the video comprises:
extracting audio track of the video;
converting the audio track into a sequence of syllables, wherein the sequence of syllables comprises a plurality of syllables that are spoken at different time points within the video;
processing the video to create a plurality of sampling frames, wherein each respective sampling frame, of the plurality of sampling frame, is captured at a time point that a respective syllable is spoken; and
comparing each respective syllable, of the plurality of syllables, with each respective sampling frame, of the plurality of sampling frames, to generate a plurality consistency values.

### Claim 14 (depends on 11)

The system of claim 11, wherein examining the one or more video features of the video to detect modifications comprises:
processing the video to create sampling frames at a defined interval;
analyzing each of the sampling frames to extract the one or more video features using one or more image processing algorithms, wherein the one or more vide features comprise at least one of (i) pixilation, (ii) shadows, (iii) colors, (iv) edges, (iv) textures; and
determining the video has been manipulated based on processing the one or more video features using a machine learning model.

### Claim 15 (depends on 11)

The system of claim 11, wherein identifying overlapping clips between the video and the reference video comprises:
processing the video to create sampling frames at a defined interval;
hashing the sampling frames to generate a signature for the video;
retrieving a signature for the reference video from a database; and
comparing the signature for the video with the signature for the reference video to identify the overlapping clips between the video and the reference video.

### Claim 16

A computer program product comprising one or more computer-readable storage media collectively containing computer-readable program code that, when executed by operation of one or more computer processors, performs an operation comprising:
accessing a statement alleging that a video is fake;
identifying one or more characteristics of the statement, wherein the one or more characteristics comprise source information for the video, source information for an reference video, and one or more timestamps in the video;
performing consistency checks between audio and video of the video;
examining one or more video features of the video to detect modifications;
identifying overlapping clips between the video and the reference video; and
determining that the video is fake based at least in part on the one or more characteristics of the statement, the consistency checks, the video features, and the identified overlapping clips.

### Claim 17 (depends on 16)

The computer program product of claim 16, wherein the operation further comprises synchronizing the consistency checks, the video features, and the identified overlapping clips with the one or more timestamps within the video identified by the statement.

### Claim 18 (depends on 16)

The computer program product of claim 16, wherein performing the consistency checks between audio and video of the video comprises:
extracting audio track of the video;
converting the audio track into a sequence of syllables, wherein the sequence of syllables comprises a plurality of syllables that are spoken at different time points within the video;
processing the video to create a plurality of sampling frames, wherein each respective sampling frame, of the plurality of sampling frame, is captured at a time point that a respective syllable is spoken; and
comparing each respective syllable, of the plurality of syllables, with each respective sampling frame, of the plurality of sampling frames, to generate a plurality of consistency values.

### Claim 19 (depends on 16)

The computer program product of claim 16, wherein examining the one or more video features of the video to detect modifications comprises:
processing the video to create sampling frames at a defined interval;
analyzing each of the sampling frames to extract the one or more video features using one or more image processing algorithms, wherein the one or more vide features comprise at least one of (i) pixilation, (ii) shadows, (iii) colors, (iv) edges, (iv) textures; and
determining the video has been manipulated based on processing the one or more video features using a machine learning model.

### Claim 20 (depends on 16)

The computer program product of claim 16, wherein identifying overlapping clips between the video and reference video comprises:
processing the video to create sampling frames at a defined interval;
hashing the sampling frames to generate a signature for the video;
retrieving a signature for the reference video from a database; and
comparing the signature for the video with the signature for the reference video to identify the overlapping clips between the video and the reference video.
