# US 2024/0005757 A1 · Method and device for smoke or fire recognition, computer apparatus, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005757 A1
- Document ID: US20240005757A1
- Status: Published application
- Kind: A1
- Application number: 17/919,511
- Filed: 2021-03-05
- Published: 2024-01-04
- Earliest claimed: 2020-12-12
- Assignees: Csg Power Generation Co., Ltd.
- Applicants: Csg Power Generation Co., Ltd.
- Inventors: Liqun SUN; Man CHEN; Hao ZHANG; Yong LU; Tao LIU; Ming XU; Jianhui LI; Miaogeng WANG; Zhipeng LV; Kai LIN; Yulin HAN; Yu GONG; Haifeng GUO; Xiaoyi WANG; Hanlong WANG; Rufei HE
- Web page: https://patentgrep.com/patent/US20240005757A1

## CPC

- G08B 17/125: {by using a video camera to detect fire or smoke}
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/56: relating to colour
- G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method and device for smoke or fire recognition, a computer device and a storage medium are disclosed. The method includes: acquiring a to-be-recognized image in a smoke or fire monitoring region; recognizing a smoke or fire suspected region in the to-be-recognized image according to the to-be-recognized image, including recognizing a smoke or fire suspected region in a visible light image on the basis of colors, and recognizing a smoke or fire suspected region in an infrared image on the basis of brightness; and inputting the to-be-recognized image including the smoke or fire suspected region into a preset model, and recognizing a smoke or fire state in the to-be-recognized image according to an output result of the preset model, the preset model being obtained by training based on the visible light image pre-marked with a smoke or fire state or the infrared image pre-marked with a smoke or fire state.

## Claims

### Claim 1

A method for smoke or fire recognition, the method comprising:
acquiring a to-be-recognized image in a smoke or fire monitoring region, the to-be-recognized image comprising a visible light image and/or an infrared image;
recognizing a smoke or fire suspected region in the to-be-recognized image according to the to-be-recognized image, comprising: recognizing a smoke or fire suspected region in the visible light image based on colors, and recognizing a smoke or fire suspected region in the infrared image based on brightness; and
inputting the to-be-recognized image comprising the smoke or fire suspected region into a preset model, and recognizing a smoke or fire state in the to-be-recognized image according to an output result of the preset model, the preset model being obtained by training based on the visible light image pre-marked with a smoke or fire state or the infrared image pre-marked with a smoke or fire state.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the acquiring the to-be-recognized image in the smoke or fire monitoring region comprises:
acquiring a monitoring image in the smoke or fire monitoring region;
filtering the monitoring image correspondingly according to an image type of the monitoring image to obtain a filtered monitoring image; and
eliminating a device motion optical flow in the filtered monitoring image according to multiple frames of the filtered monitoring image to obtain the to-be-recognized image.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the filtering the monitoring image correspondingly according to the image type of the monitoring image to obtain the filtered monitoring image comprises:
acquiring, if the monitoring image is the visible light image, color values of pixels in the visible light image in color channels of a trichromatic color space;
acquiring a ratio of each color value to a sum of all the color values;
acquiring products of the color value having a maximum ratio and the ratios as normalized color values corresponding to the color values;
determining hue values, saturation values, and tone values corresponding to the pixels according to a maximum value and a minimum value of the normalized color values and the color values corresponding to the pixels;
obtaining luminance values, chrominance values, and concentration values corresponding to the pixels according to the normalized color values and a preset brightness-chrominance transformation matrix; and
obtaining the filtered monitoring image according to the color values, the hue values, the saturation values, the tone values, the luminance values, the chrominance values, and the concentration values corresponding to the pixels in the monitoring image.

### Claim 4 (depends on 2)

The method according to claim 2, wherein the eliminating the device motion optical flow in the filtered monitoring image according to the multiple frames of the filtered monitoring image to obtain the to-be-recognized image comprises:
obtaining a projection transformation matrix corresponding to the filtered monitoring image according to a filtered monitoring image at a current moment and a filtered monitoring image at a next moment corresponding to the current moment; and
obtaining, according to an inverse of the projection transformation matrix and the filtered monitoring image corresponding to the current moment, a filtered monitoring image after elimination of the device motion optical flow as the to-be-recognized image.

### Claim 5 (depends on 3)

The method according to claim 3, wherein the recognizing a smoke or fire suspected region in the to-be-recognized image according to the to-be-recognized image comprises:
obtaining, if the to-be-recognized image is the visible light image, a minimum smoke region in the to-be-recognized image according to the brightness values, the chrominance values, the concentration values, the color values, a first maximum threshold, and a first minimum threshold corresponding to pixels in the to-be-recognized image;
obtaining a first minimum flame region in the to-be-recognized image according to a first color threshold, a second color threshold, and the color values corresponding to the pixels in the to-be-recognized image;
obtaining a second minimum flame region in the to-be-recognized image according to the hue values, the saturation values, and the tone values corresponding to the pixels in the to-be-recognized image;
obtaining a third minimum flame region in the to-be-recognized image according to the luminance values, the chrominance values, the concentration values, a luminance threshold, a brightness threshold, and a chrominance threshold corresponding to the pixels in the to-be-recognized image;
extracting an optical flow from an image sequence of a plurality of to-be-recognized images to obtain a motion region in the to-be-recognized image; and
obtaining the smoke or fire suspected region in the to-be-recognized image according to the minimum smoke region, the first minimum flame region, the second minimum flame region, the third minimum flame region, and the motion region.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the preset model comprises: a first preset model and a second preset model; and
wherein the inputting the to-be-recognized image comprising the smoke or fire suspected region into the preset model, and recognizing the smoke or fire state in the to-be-recognized image according to the output result of the preset model comprises:
inputting the to-be-recognized image into the first preset model if the to-be-recognized image is the visible light image;
inputting the to-be-recognized image into the second preset model if the to-be-recognized image is the infrared image;
acquiring a judgment vector obtained after feature extraction on the smoke or fire suspected region through the first preset model and/or the second preset model; and
determining the smoke or fire state in the to-be-recognized image according to the judgment vector.

### Claim 7 (depends on 1)

The method according to claim 1, subsequent to the recognizing the smoke or fire state in the to-be-recognized image according to the output result of the preset model, further comprising:
triggering, if the smoke or fire state is smoke or fire, alarm information, and inputting the to-be-recognized image into a smoke or fire detection model to acquire a smoke or fire region prediction box outputted by the smoke or fire detection model, the smoke or fire detection model being obtained by training based on a plurality of images with smoke or fire; and
continuously recognizing the smoke or fire region prediction box according to a preset discriminant tracking algorithm, in order to track the smoke or fire state in the to-be-recognized image, a smoke or fire region prediction box in the to-be-recognized image at a current moment being obtained through the preset discriminant tracking algorithm based on a to-be-recognized image comprising a smoke or fire region prediction box at a previous moment corresponding to the current moment.

### Claim 8

(canceled)

### Claim 9

A computer apparatus, comprising a memory and a processor, the memory storing a computer program, wherein the computer program, when executed by the processor, causes the processor to:
acquire a to-be-recognized image in a smoke or fire monitoring region, the to-be-recognized image comprising a visible light image and/or an infrared image;
recognize a smoke or fire suspected region in the to-be-recognized image according to the to-be-recognized image, comprising: recognizing a smoke or fire suspected region in the visible light image based on colors, and recognizing a smoke or fire suspected region in the infrared image based on brightness; and
input the to-be-recognized image comprising the smoke or fire suspected region into a preset model, and recognizing a smoke or fire state in the to-be-recognized image according to an output result of the preset model, the preset model being obtained by training based on the visible light image pre-marked with a smoke or fire state or the infrared image pre-marked with a smoke or fire state.

### Claim 10

A computer-readable non-volatile storage medium, storing a computer program, wherein the computer program, when executed by a processor, causes the processor to:
acquire a to-be-recognized image in a smoke or fire monitoring region, the to-be-recognized image comprising a visible light image and/or an infrared image;
recognize a smoke or fire suspected region in the to-be-recognized image according to the to-be-recognized image, comprising: recognizing a smoke or fire suspected region in the visible light image based on colors, and recognizing a smoke or fire suspected region in the infrared image based on brightness; and
input the to-be-recognized image comprising the smoke or fire suspected region into a preset model, and recognizing a smoke or fire state in the to-be-recognized image according to an output result of the preset model, the preset model being obtained by training based on the visible light image pre-marked with a smoke or fire state or the infrared image pre-marked with a smoke or fire state.

### Claim 11 (depends on 2)

The method according to claim 2, wherein the filtering the monitoring image correspondingly according to the image type of the monitoring image to obtain the filtered monitoring image comprises:
sorting, if the monitoring image is an infrared image, all pixels in a neighborhood of each pixel in the infrared image, acquiring an intermediate grayscale value in the sorted pixels, and assigning the intermediate grayscale value to the corresponding pixel to obtain the filtered monitoring image.

### Claim 12 (depends on 3)

The method according to claim 3, wherein the recognizing a smoke or fire suspected region in the to-be-recognized image according to the to-be-recognized image comprises:
obtaining, if the to-be-recognized image is the infrared image, a flame region in the to-be-recognized image according to a preset infrared brightness threshold;
obtaining a pixel difference corresponding to the to-be-recognized image according to a to-be-recognized image at a current moment and a to-be-recognized image at a next moment corresponding to the current moment;
determining a motion region in the to-be-recognized image according to the pixel difference and a preset motion image threshold; and
obtaining the smoke or fire suspected region in the to-be-recognized image according to the flame region and the motion region.
