# US 2017/0286775 A1 · Method and device for detecting violent contents in a video , and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0286775 A1
- Document ID: US20170286775A1
- Status: Published application
- Kind: A1
- Application number: 15/247,765
- Filed: 2016-08-25
- Published: 2017-10-05
- Earliest claimed: 2016-03-29
- Assignees: Le Holdings (Beijing) Co., Ltd.; Le Shi Zhi Xin Electronic Technology (Tian Jin) Limited
- Applicants: Le Holdings(Beijing)Co., Ltd.; Le Shi Zhi Xin Electronic Technology (Tian Jin) Limited
- Inventors: Wei Cai
- Web page: https://patentgrep.com/patent/US20170286775A1

## CPC

- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06V 10/56: relating to colour
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- H04N 21/234: Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Embodiments of the disclosure provide a method and device for detecting violent contents in a video, and a non-transitory computer-readable storage medium. The method for detecting violent contents in a video includes: determining an average shot length of any scene in the video to be detected, and an average motion intensity of the shot in the scene; and extracting feature data of a number of elements in the scene upon determining that the average shot length is below a first preset threshold, and/or the average motion intensity of the shot is above a second preset threshold, and determining that there are violent contents in the video to be detected upon determining that the feature data of at least one element among the extracted feature data of the elements lie in a range of feature data of the element extracted in advance from a specific scene.

## Claims

### Claim 1

A method for detecting violent contents in a video, the method comprising:
at an electronic device:
determining an average shot length of any scene in the video to be detected, and an average motion intensity of a shot in the scene; and
extracting feature data of a number of elements in the scene upon determining that the average shot length is below a first preset threshold, and/or the average motion intensity of the shot is above a second preset threshold, and determining that there are violent contents in the video to be detected upon determining that the feature data of at least one element among the extracted feature data of the elements lie in a range of feature data of the element extracted in advance from a specific scene.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the feature data of the elements comprise image feature data of each frame of picture in the scene, and audio feature data in the scene.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the image feature data of each frame of picture comprise a color histogram of each frame of picture; and
when the feature data of the elements comprise the image feature data of each frame of picture in the scene, then determining whether the image feature data of each frame of picture lie in a range of image feature data of the picture extracted in advance from the specific scene comprises:
for each frame of picture in the scene, extracting the color histogram of the frame of picture, and determining that the image feature data of the frame of picture lie in the range of image feature data of the picture extracted in advance from the specific scene upon determining that counted amounts of a preset number of colors in the color histogram of the frame of picture lie in ranges of counted amounts of the corresponding colors in a color histogram of the picture extracted from the specific scene.

### Claim 4 (depends on 3)

The method according to claim 3, wherein after it is determined that the counted amounts of the preset number of colors in the color histogram of the frame of picture lie in the ranges of counted amounts of the corresponding colors in the color histogram of the picture extracted from the specific scene, the method further comprises:
determining the counted amounts of the preset number of colors in a number of frames of pictures adjacent to the frame of picture; and
determining that the image feature data of the frame of picture lie in the range of image feature data of the picture extracted in advance from the specific scene comprises:
determining that the image feature data of the frame of picture lie in the range of image feature data of the picture extracted in advance from the specific scene upon determining that the counted number of each one of the preset number of colors in the frame of picture and the adjacent frames of pictures is increasing gradually along a time order of the frames of pictures.

### Claim 5 (depends on 2)

The method according to claim 2, wherein the audio feature data comprise a sample vector of the audio data and a covariance matrix of the audio data; and
when the feature data of the elements comprise the audio feature data in the scene, then determining whether the audio feature data in the scene lie in a range of audio feature data extracted in advance from the specific scene comprises:
calculating the sample vector and the covariance matrix of the audio data in the scene, and determining that the audio feature data in the scene lie in the range of audio feature data extracted in advance from the specific scene upon determining that the similarity between the sample vector and the covariance matrix of the audio data in the scene, and a sample vector and a covariance matrix of the audio data extracted in advance from the specific scene is above a third preset threshold.

### Claim 6 (depends on 2)

The method according to claim 2, wherein the audio feature data comprise an energy entropy of the audio data; and
when the feature data of the elements comprise the audio feature data in the scene, then determining whether the audio feature data in the scene lie in a range of audio feature data extracted in advance from the specific scene comprises:
segmenting the audio data in the scene into a number of segments, calculating an energy entropy of each segment of audio data, and when the energy entropy of at least one segment of audio data among the energy entropies of the segments of audio data is below a fourth preset threshold, then determining that the audio feature data in the scene lie in the range of audio feature data extracted in advance from the specific scene.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the energy entropy of each segment of audio data is calculated in the equation of:
I
=
-
∑
i
=
1
J

σ
i
2

log
2

σ
i
2
,
wherein I represents the energy entropy of each segment of audio data, J represents a total number of segments into which the audio data in the scene are segmented, and σ2 represents a normalized energy value of the i-th segment of audio data.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the average motion intensity of the shot is equal to a ratio of a sum of motion intensities of all the shots in the scene to a total number of shots in the scene, wherein the motion intensity of each shot in the scene is calculated in the equation of:
SS
=
1
T

∑
i
=
b
+
1
e

{
∑
m
,
n


m
l
k

(
m
,
n
)

}
,
wherein SS represents the motion intensity of each shot, mlk(m,n) represents the i-th frame in the k-th shot of motion sequence images of the current scene, wherein m and n represent horizontal and vertical resolutions of the motion sequence images, b and e represent start frame number and end frame number of the k-th shot, and T represents a length T=e−b of the k-th shot.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the average shot length is equal to a ratio of a total length of time of the scene to a number of shots in the scene.

### Claim 10

An electronic device, comprising:
at least one processor; and
a memory communicably connected with the at least one processor for storing instructions executable by the at least one processor, wherein execution of the instructions by the at least one processor causes the at least one processor to:
determine an average shot length of any scene in the video to be detected, and an average motion intensity of a shot in the scene; and
extract feature data of a number of elements in the scene upon determining that the average shot length is below a first preset threshold, and/or the average motion intensity of the shot is above a second preset threshold, and determine that there are violent contents in the video to be detected upon determining that the feature data of at least one element among the extracted feature data of the elements lie in a range of feature data of the element extracted in advance from a specific scene.

### Claim 11 (depends on 10)

The electronic device according to claim 10, wherein the feature data of the elements comprise image feature data of each frame of picture in the scene, and audio feature data in the scene.

### Claim 12 (depends on 11)

The electronic device according to claim 11, wherein the image feature data of each frame of picture comprise a color histogram of each frame of picture; and
when the feature data of the elements comprise the image feature data of each frame of picture in the scene, then determine whether the image feature data of each frame of picture lie in a range of image feature data of the picture extracted in advance from the specific scene comprises:
for each frame of picture in the scene, extract the color histogram of the frame of picture, and determine that the image feature data of the frame of picture lie in the range of image feature data of the picture extracted in advance from the specific scene upon determining that counted amounts of a preset number of colors in the color histogram of the frame of picture lie in ranges of counted amounts of the corresponding colors in a color histogram of the picture extracted from the specific scene.

### Claim 13 (depends on 12)

The electronic device according to claim 12, wherein after it is determined that the counted amounts of the preset number of colors in the color histogram of the frame of picture lie in the ranges of counted amounts of the corresponding colors in the color histogram of the picture extracted from the specific scene, the at least one processor is further caused to:
determine the counted amounts of the preset number of colors in a number of frames of pictures adjacent to the frame of picture; and
determine that the image feature data of the frame of picture lie in the range of image feature data of the picture extracted in advance from the specific scene comprises:
determine that the image feature data of the frame of picture lie in the range of image feature data of the picture extracted in advance from the specific scene upon determining that the counted number of each one of the preset number of colors in the frame of picture and the adjacent frames of pictures is increasing gradually along a time order of the frames of pictures.

### Claim 14 (depends on 11)

The electronic device according to claim 11, wherein the audio feature data comprise a sample vector of the audio data and a covariance matrix of the audio data; and
when the feature data of the elements comprise the audio feature data in the scene, then determine whether the audio feature data in the scene lie in a range of audio feature data extracted in advance from the specific scene comprises:
calculate the sample vector and the covariance matrix of the audio data in the scene, and determine that the audio feature data in the scene lie in the range of audio feature data extracted in advance from the specific scene upon determining that the similarity between the sample vector and the covariance matrix of the audio data in the scene, and a sample vector and a covariance matrix of the audio data extracted in advance from the specific scene is above a third preset threshold.

### Claim 15 (depends on 11)

The electronic device according to claim 11, wherein the audio feature data comprise an energy entropy of the audio data; and
when the feature data of the elements comprise the audio feature data in the scene, then determine whether the audio feature data in the scene lie in a range of audio feature data extracted in advance from the specific scene comprises:
segment the audio data in the scene into a number of segments, calculating an energy entropy of each segment of audio data, and when the energy entropy of at least one segment of audio data among the energy entropies of the segments of audio data is below a fourth preset threshold, then determine that the audio feature data in the scene lie in the range of audio feature data extracted in advance from the specific scene.

### Claim 16 (depends on 15)

The electronic device according to claim 15, wherein the energy entropy of each segment of audio data is calculated in the equation of:
I
=
-
∑
i
=
1
J

σ
i
2

log
2

σ
i
2
,
wherein I represents the energy entropy of each segment of audio data, J represents a total number of segments into which the audio data in the scene are segmented, and σ2 represents a normalized energy value of the i-th segment of audio data.

### Claim 17 (depends on 10)

The electronic device according to claim 10, wherein the average motion intensity of the shot is equal to a ratio of a sum of motion intensities of all the shots in the scene to a total number of shots in the scene, wherein the motion intensity of each shot in the scene is calculated in the equation of:
SS
=
1
T

∑
i
=
b
+
1
e

{
∑
m
,
n


m
l
k

(
m
,
n
)

}
,
wherein SS represents the motion intensity of each shot, mlk(m,n) represents the i-th frame in the k-th shot of motion sequence images of the current scene, wherein m and n represent horizontal and vertical resolutions of the motion sequence images, b and e represent start frame number and end frame number of the k-th shot, and T represents a length T=e−b of the k-th shot.

### Claim 18 (depends on 10)

The electronic device according to claim 10, wherein the average shot length is equal to a ratio of a total length of time of the scene to a number of shots in the scene.

### Claim 19

A non-transitory computer-readable storage medium storing executable instructions that, when executed by an electronic device with a touch-sensitive display, cause the electronic device to:
determine an average shot length of any scene in the video to be detected, and an average motion intensity of a shot in the scene; and
extract feature data of a number of elements in the scene upon determining that the average shot length is below a first preset threshold, and/or the average motion intensity of the shot is above a second preset threshold, and determine that there are violent contents in the video to be detected upon determining that the feature data of at least one element among the extracted feature data of the elements lie in a range of feature data of the element extracted in advance from a specific scene.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium according to claim 19, wherein the feature data of the elements comprise image feature data of each frame of picture in the scene, and audio feature data in the scene.