# US 2003/0228058 A1 · Object detection for sudden illumination changes using order consistency

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2003/0228058 A1
- Document ID: US20030228058A1
- Status: Published application
- Kind: A1
- Application number: 10/158,632
- Filed: 2002-05-30
- Published: 2003-12-11
- Earliest claimed: 2002-05-30
- Assignees: Siemens Corporate Research, Inc.
- Inventors: Binglong Xie; Visvanathan Ramesh; Terrance Boult
- Web page: https://patentgrep.com/patent/US20030228058A1

## CPC

- G08B 13/19604: {involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change}
- G06T 7/254: involving subtraction of images
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system and method for object detection with sudden illumination changes provides a system including a camera model unit for computing a difference map between a first input image and a second input image, an illumination model unit in signal communication with the camera model unit for comparing the sign of the difference between two pixels in a first input image against the sign of the difference between two pixels in a second input image, and a CPU in signal communication with the camera model unit and the illumination model unit for detecting a changed object in accordance with said comparison when said sign of the difference is changed; and a corresponding method including providing a first image having a number of pixels, receiving a second image subsequent to the first image, calculating a difference map between the first and second images, selecting a pair of pixel locations from the first or second images in accordance with the difference map, statistically modeling a camera noise level for the first and second images, computing the likelihood of order inconsistency between a pair of pixel values at the selected locations in the first image and a pair of pixel values at the selected locations in the second image in accordance with the camera noise level; and detecting a changed object in accordance with the computed likelihood.

## Claims

### Claim 1

A method for object detection with sudden illumination changes, the method comprising:
providing a first image having a plurality of pixels;
receiving a second image corresponding to a time other than that of said first image;
calculating a difference map between said first and second images;
selecting a pair of pixel locations from at least one of said first and second images in accordance with said difference map;
statistically modeling a camera noise level for at least one of said first and second images;
computing the likelihood of order inconsistency between a pair of pixel values at said selected locations in said first image and a pair of pixel values at said selected locations in said second image in accordance with said camera noise level; and
detecting a changed object in accordance with said computed likelihood.

### Claim 2 (depends on 1)

A method as defined in claim 1 wherein said second image is temporally consecutive with said first image.

### Claim 3 (depends on 1)

A method as defined in claim 1, further comprising maintaining said first image as a reference for calculating a plurality of difference maps corresponding to a plurality of second images.

### Claim 4 (depends on 1)

A method as defined in claim 1, further comprising mixing said first image with at least one past image.

### Claim 5 (depends on 4)

A method as defined in claim 4 wherein mixing comprises pixel averaging.

### Claim 6 (depends on 1)

A method as defined in claim 1 wherein said first image is a separately maintained background image.

### Claim 7 (depends on 1)

A method as defined in claim 1, further comprising photometrically calibrating said statistical model for camera noise.

### Claim 8 (depends on 1)

A method as defined in claim 1, further comprising applying a monotone but non-linear camera response function to at least one of said first and second images.

### Claim 9 (depends on 8)

A method as defined in claim 8, further comprising photometrically calibrating said camera response function.

### Claim 10 (depends on 1)

A method as defined in claim 1 wherein the sign of the difference between the incident radiances of a pair of pixels in said first image is the same as the sign of the difference between the sensed pixel values.

### Claim 11 (depends on 10)

A method as defined in claim 10 wherein a change in at least one of camera gain and illumination level does not alter the difference between the incident radiances of said pair of pixels for an unchanged object, which remains the same as the sign of the difference between the sensed pixel values.

### Claim 12 (depends on 10)

A method as defined in claim 10 wherein said comparing the sign comprises applying a Phong shading model for the surface material.

### Claim 13 (depends on 1)

A method as defined in claim 1 wherein said detecting a changed object is responsive to local pixel information.

### Claim 14 (depends on 1)

A method as defined in claim 1 wherein said detecting a changed object is not compromised by a sudden illumination change.

### Claim 15

A system (100) for object detection with sudden illumination changes, the system comprising:
a camera model unit (180) for computing a difference map between a first input image and a second input image;
an illumination model unit (170) in signal communication with the camera model unit for comparing the sign of the difference between two pixels in a first input image against the sign of the difference between two pixels in a second input image; and
a CPU (102) in signal communication with said camera model unit and said illumination model unit for detecting a changed object in accordance with said comparison when said sign of the difference is changed.

### Claim 16 (depends on 15)

A system (100) as defined in claim 15 wherein at least one of said first and second input images comprises a surveillance image.

### Claim 17 (depends on 15)

A system (100) as defined in claim 15, further comprising:
a display adapter (110) in signal communication with the CPU (102) for displaying at least one of said map and said first and second input images; and
an I/O adapter (112) in signal communication with the CPU (102) for recalling the locations of the pixels from the displayed image to provide an indication of the location of a changed object within an input image.

### Claim 18 (depends on 15)

A system (100) as defined in claim 15, further comprising:
a user interface adapter (114) in signal communication with the CPU (102) for at least receiving a selection decision for at least one image from a user.

### Claim 19

A system for object detection with sudden illumination changes, the system comprising:
providing means for providing a first image having a plurality of pixels;
receiving means for receiving a second image corresponding to a time other than that of said first image;
calculating means for calculating a difference map between said first and second images;
selecting means for selecting a pair of pixel locations from at least one of said first and second images in accordance with said difference map;
modeling means for statistically modeling a camera noise level for at least one of said first and second images;
computing means for computing the likelihood of order inconsistency between a pair of pixel values at said selected locations in said first image and a pair of pixel values at said selected locations in said second image in accordance with said camera noise level; and
detecting means for detecting a changed object in accordance with said computed likelihood.

### Claim 20 (depends on 19)

A system as defined in claim 19 wherein said second image is temporally consecutive with said first image.

### Claim 21 (depends on 19)

A system as defined in claim 19, further comprising maintaining means for maintaining said first image as a reference for calculating a plurality of difference maps corresponding to a plurality of second images.

### Claim 22 (depends on 19)

A system as defined in claim 19, further comprising mixing means for mixing said first image with at least one past image.

### Claim 23 (depends on 22)

A system as defined in claim 22 wherein said mixing means comprises pixel averaging means.

### Claim 24 (depends on 19)

A system as defined in claim 19 wherein said first image is a separately maintained background image.

### Claim 25 (depends on 19)

A system as defined in claim 19, further comprising first calibrating means for photometrically calibrating said statistical model for camera noise.

### Claim 26 (depends on 19)

A system as defined in claim 19, further comprising second modeling means for applying a monotone but non-linear camera response function to at least one of said first and second images.

### Claim 27 (depends on 26)

A system as defined in claim 26, further comprising second calibration means for photometrically calibrating said camera response function.

### Claim 28 (depends on 19)

A system as defined in claim 19 wherein the sign of the difference between the incident radiances of a pair of pixels in said first image is the same as the sign of the difference between the sensed pixel values.

### Claim 29 (depends on 28)

A system as defined in claim 28 wherein a change in at least one of camera gain and illumination level does not alter the difference between the incident radiances of said pair of pixels for an unchanged object, which remains the same as the sign of the difference between the sensed pixel values.

### Claim 30 (depends on 28)

A system as defined in claim 28 wherein said comparing the sign comprises applying a Phong shading model for the surface material.

### Claim 31 (depends on 19)

A system as defined in claim 19 wherein said detecting a changed object is responsive to local pixel information.

### Claim 32 (depends on 19)

A system as defined in claim 19 wherein said detecting a changed object is not compromised by a sudden illumination change.

### Claim 33

A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for object detection with sudden illumination changes, the method steps comprising:
providing a first image having a plurality of pixels;
receiving a second image corresponding to a time other than that of said first image;
calculating a difference map between said first and second images;
selecting a pair of pixel locations from at least one of said first and second images in accordance with said difference map;
statistically modeling a camera noise level for at least one of said first and second images;
computing the likelihood of order inconsistency between a pair of pixel values at said selected locations in said first image and a pair of pixel values at said selected locations in said second image in accordance with said camera noise level; and
detecting a changed object in accordance with said computed likelihood.

### Claim 34 (depends on 33)

A program storage device as defined in claim 33 wherein said second image is temporally consecutive with said first image.

### Claim 35 (depends on 33)

A program storage device as defined in claim 33, the method steps further comprising maintaining said first image as a reference for calculating a plurality of difference maps corresponding to a plurality of second images.

### Claim 36 (depends on 33)

A program storage device as defined in claim 33, the method steps further comprising mixing said first image with at least one past image.

### Claim 37 (depends on 36)

A program storage device as defined in claim 36 wherein the method step of mixing comprises pixel averaging.

### Claim 38 (depends on 33)

A program storage device as defined in claim 33 wherein said first image is a separately maintained background image.

### Claim 39 (depends on 33)

A program storage device as defined in claim 33, the method steps further comprising photometrically calibrating said statistical model for camera noise.

### Claim 40 (depends on 33)

A program storage device as defined in claim 33, the method steps further comprising applying a monotone but non-linear camera response function to at least one of said first and second images.

### Claim 41 (depends on 40)

A program storage device as defined in claim 40, the method steps further comprising photometrically calibrating said camera response function.

### Claim 42 (depends on 33)

A program storage device as defined in claim 33 wherein the sign of the difference between the incident radiances of a pair of pixels in said first image is the same as the sign of the difference between the sensed pixel values.

### Claim 43 (depends on 42)

A program storage device as defined in claim 42 wherein a change in at least one of camera gain and illumination level does not alter the difference between the incident radiances of said pair of pixels for an unchanged object, which remains the same as the sign of the difference between the sensed pixel values.

### Claim 44 (depends on 42)

A program storage device as defined in claim 42 wherein said comparing the sign comprises applying a Phong shading model for the surface material.

### Claim 45 (depends on 33)

A program storage device as defined in claim 33 wherein said detecting a changed object is responsive to local pixel information.

### Claim 46 (depends on 33)

A program storage device as defined in claim 33 wherein said detecting a changed object is not compromised by a sudden illumination change.
