# US 7,155,029 B2 · Method and apparatus of detecting fire by flame imaging > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,155,029 B2 - Document ID: US7155029B2 - Status: Granted - Kind: B2 - Application number: 10/143,386 - Filed: 2002-05-10 - Published: 2003-03-06 - Granted: 2006-12-26 - Earliest claimed: 2001-05-11 - Assignees: Detector Electronics Corporation - Inventors: John D. King; Paul M. Junck - Web page: https://patentgrep.com/patent/US7155029B2 ## CPC - G08B 17/125: {by using a video camera to detect fire or smoke} - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS ## Abstract An apparatus and method for performing first and second imaging processes contemporaneously, in particular where one of the processes is detecting fires based on images of the flames. The apparatus includes an image sensor for producing a video image, a frame grabber for capturing first frames and second frames, a processor for processing the data within the frames, and an output device. The apparatus may also include an adjustment mechanism for adjusting the image settings of the image sensor between settings suitable for flame imaging and non-flame imaging, and a control mechanism for controlling the image settings of the image sensor. In the method at least two first frames are obtained, and a plurality of second frames are obtained. The first and second frames are used for first and second processes. The first and second processes are contemporaneous, so that they are carried out within the same time period without interfering with one another. When the first process is flame detection, individual pairs of pixels having a property such as intensity that meets a first threshold are identified within the first frames, and are assembled into blobs. Additional properties of the pixel pairs and the blobs overall are evaluated in relation with additional thresholds. Blobs or pixels that do not meet the thresholds are excluded. Any blobs remaining after all evaluations are considered fires. ## Claims ### Claim 1 Method of detecting a flame within a field of view of a video sensor, comprising the steps of: generating a video image with the video sensor, the video image including a sequence of video frames created within a time unit; the video sensor having adjustable image settings to create frames within the sequence at different image settings; generating a first set of frames within the sequence at a first image setting selected for imaging a flame, the first set including at least a base frame and a comparison frame; and generating a remainder of the frames within the sequence at a second image setting selected for imaging non-flame objects; performing a flame detection process using the base and comparison frames, the flame detection process including comparing a location in the base frame with a spatially corresponding location in the comparison frame and identifying differences in the locations as indications of a flame. ### Claim 2 (depends on 1) Method according to claim 1, wherein: said base and comparison frames comprise consecutive image frames. ### Claim 3 (depends on 1) Method according to claim 1, wherein: said video image is a color image. ### Claim 4 (depends on 1) Method according to claim 1, wherein: said remainder of the frames are generated for display as a human-viewable output. ### Claim 5 (depends on 1) Method according to claim 1, wherein said comparing includes: identifying a plurality of base pixels in said base frame, and a plurality of comparison pixels in said comparison frame, wherein for each base frame pixel there is a spatially corresponding comparison frame pixel, each said base frame pixel and said corresponding comparison frame pixel forming a pair, such that said pluralities of base and comparison image pixels comprise a plurality of pairs. ### Claim 6 Method of using a video sensor, comprising the steps of: generating a video image with said video sensor; obtaining at least two first frames from said video image; obtaining a plurality of second frames from said video image; and contemporaneously performing a first process using said first frames and a second process using said second frames; wherein said first process comprises flame detection; said at least two first frames comprise a base frame and a comparison frame; and said first process comprises the steps of: identifying a plurality of base pixels in said base frame, and a plurality of comparison pixels in said comparison frame, wherein for each base frame pixel there is a spatially corresponding comparison frame pixel, each said base frame pixel and said corresponding comparison frame pixel forming a pair, such that said pluralities of base and comparison image pixels comprise a plurality of pairs; determining a first property of at least some of said pairs; categorizing said pairs as blob pairs if said first property meets a first threshold; and assembling said blob pairs into at least one blob. ### Claim 7 Method of using a video sensor, comprising the steps of: generating a video image with said video sensor; obtaining at least two first frames from said video image; obtaining a plurality of second frames from said video image; and contemporaneously performing a first process using said first frames and a second process using said second frames; wherein said first process comprises flame detection; said at least two first frames comprise a base frame and a comparison frame; and said first process comprises the steps of: identifying a plurality of base pixels in said base frame, and a plurality of comparison pixels in said comparison frame, wherein for each base frame pixel there is a spatially corresponding comparison frame pixel, each said base frame pixel and said corresponding comparison frame pixel forming a pair, such that said pluralities of base and comparison image pixels comprise a plurality of pairs; determining a first property of at least some of said pairs; categorizing said pairs as blob pairs if said first property meets a first threshold; assembling said blob pairs into at least one blob; and indicating said at least one blob as a fire. ### Claim 8 (depends on 7) Method according to claim 7, wherein: said first property is intensity, and said first threshold is a minimum intensity threshold. ### Claim 9 (depends on 7) Method according to claim 7, wherein said first process further comprises the steps of: determining a second property of said blob pairs; and excluding said blob pairs from said blob if said second property of said blob pairs does not meet a second threshold. ### Claim 10 (depends on 9) Method according to claim 9, wherein: said video image is a color image; said first and second frames are color frames; and said second property is color, and said second threshold is a color range. ### Claim 11 (depends on 10) Method according to claim 10, wherein: said color is measured in YCRCB coordinates, and said color range is defined in YCRCB coordinates. ### Claim 12 (depends on 9) Method according to claim 9, wherein said first process further comprises the steps of: determining a third property of said blob pairs; excluding said blob as a non-fire if said third property of said blob pairs does not meet a third threshold. ### Claim 13 (depends on 12) Method according to claim 12, wherein: determining said third property comprises determining derivatives of differences in intensity and color between said base pixels and said comparison pixels in said blob pairs. ### Claim 14 (depends on 13) Method according to claim 13, wherein: determining said third property further comprises: plotting said derivatives as at least one histogram and an incidence in at least two bands in said at least one histogram. ### Claim 15 (depends on 14) Method according to claim 14, wherein: determining said third property further comprises: plotting an incidence from at least one of said at least two bands against an incidence of at least another of said at least two bands as at least one x-y plot. ### Claim 16 (depends on 15) Method according to claim 15, wherein: determining said third property further comprises: determining a ratio of a number of points on a first side of a cut-off line on said at least one x-y plot to a number of points not on said first side of said cut-off line; and said third property comprises said ratio from said at least one x-y plot. ### Claim 17 (depends on 12) Method according to claim 12, wherein: said second property is color; said second threshold is a color range; said color is measured in YCRCB coordinates; said color range is defined in YCRCB coordinates; and said derivatives comprise dY/dt, dY/dx, dCR/dt, dCR/dx, dCB/dt, and dCB/dx, wherein dY/dt is a derivative of intensity with respect to time; dY/dx is a derivative of intensity with respect to position; dCR/dt is a derivative of red chrominance with respect to time; dCR/dx is a derivative of red chrominance with respect to position; dCB/dt is a derivative of blue chrominance with respect to time; dCB/dx is a derivative of blue chrominance with respect to position. ### Claim 18 (depends on 12) Method according to claim 12, wherein said first process further comprises the steps of: determining a fourth property of said blob pairs; and excluding said blob as a non-fire if said fourth property of said blob pairs does not meet a fourth threshold. ### Claim 19 (depends on 18) Method according to claim 18, wherein: said fourth property comprises a count of a number of instances of meeting said fourth threshold, and said fourth threshold is a minimum count value. ### Claim 20 (depends on 7) Method according to claim 7, wherein said first process further comprises the steps of: determining a second property of said blob pairs; and excluding said blob as a non-fire if said second property of said blob pairs does not meet a second threshold. ### Claim 21 Apparatus for performing multiple contemporaneous image processes, comprising: a video sensor adapted to generate a video image; a frame grabber in communication with said video sensor, adapted to obtain at least two first frames and a plurality of second frames from said video sensor; a processor in communication with said frame grabber, adapted to contemporaneously perform a first process using said first frames and a second process using said second frames; and at least one output mechanism in communication with said processor, adapted to generate a first output from said first process, and a second output from said second process; wherein said processor is adapted to perform flame detection as said first process; wherein said processor is adapted to perform the following as part of said first process: determining a first property of at least some of said pairs; categorizing said pairs as blob pairs if said first property meets a first threshold; and assembling said blob pairs into at least one blob. ### Claim 22 Apparatus for performing multiple contemporaneous image processes, comprising: a video sensor adapted to generate a video image; a frame grabber in communication with said video sensor, adapted to obtain at least two first frames and a plurality of second frames from said video sensor; a processor in communication with said frame grabber, adapted to contemporaneously perform a first process using said first frames and a second process using said second frames; and at least one output mechanism in communication with said processor, adapted to generate a first output from said first process, and a second output from said second process; wherein said processor is adapted to perform flame detection as said first process; said at least two first frames comprise a base frame and a comparison frame; and at least one of said video sensor, said frame grabber, and said processor is adapted to identify a plurality of base pixels in said base frame and a plurality of comparison pixels in said comparison frame as part of said first process, wherein for each base frame pixel there is a spatially corresponding comparison frame pixel, each said base frame pixel and said corresponding comparison frame pixel forming a pair, such that said pluralities of base and comparison image pixels comprise a plurality of pairs; wherein said processor is adapted to perform the following as part of said first process: determining a first property of at least some of said pairs; categorizing said pairs as blob pairs if said first property meets a first threshold; assembling said blob pairs into at least one blob; and indicating said at least one blob as a fire. ### Claim 23 (depends on 22) Apparatus according to claim 22, wherein said processor is adapted to perform the following as part of said first process: determining a second property of said blob pairs; and excluding said blob pairs if said second property of said blob pairs does not meet a second threshold. ### Claim 24 (depends on 23) Apparatus according to claim 23, wherein said processor is adapted to perform the following as part of said first process: determining a third property of said blob pairs; excluding said blob as a non-fire if said third property of said blob pairs does not meet a third threshold. ### Claim 25 (depends on 24) Apparatus according to claim 24, wherein: determining said third property comprises calculating derivatives, and said processor is adapted to calculate said derivatives. ### Claim 26 (depends on 25) Apparatus according to claim 25, wherein: determining said third property further comprises: plotting said third property as at least one histogram and determining a number of qualified points thereof for at least two bands in said at least one histogram; and plotting an incidence of at least one of said at least two bands against an incidence of at least another of said at least two bands as at least one x-y plot. ### Claim 27 (depends on 24) Apparatus according to claim 24, wherein said processor is adapted to perform the following as part of the first process: determining a fourth property of said blob pairs; and excluding said blob as a non-fire if said fourth property of said blob pairs does not meet a fourth threshold. ### Claim 28 (depends on 22) Apparatus according to claim 22, wherein said processor is adapted to perform the following as part of said first process: determining a second property of said blob pairs; and excluding said blob as a non-fire if said second property of said blob pairs does not meet a second threshold.