# US 12,602,746 B2 · System and method for background modelling for a video stream > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,746 B2 - Document ID: US12602746B2 - Status: Granted - Kind: B2 - Application number: 18/678,551 - Filed: 2024-05-30 - Published: 2024-12-05 - Granted: 2026-04-14 - Earliest claimed: 2023-06-05 - Assignees: Axis AB - Applicants: Axis AB - Inventors: Ludvig Hassbring; Song Yuan - Web page: https://patentgrep.com/patent/US12602746B2 ## CPC - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 7/254: involving subtraction of images - G06T 7/194: involving foreground-background segmentation - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06V 10/762: using clustering, e.g. of similar faces in social networks - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - G06T 2207/10016: Video; Image sequence - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/20221: Image fusion; Image merging - G06T 2207/30232: Surveillance - G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes} - G06V 2201/07: Target detection - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The present disclosure relates to a method of background modelling for a video stream acquired by a camera having movement capabilities. The method comprises: acquiring a video stream; repeatedly updating a background model for the video stream by analyzing changes in the sequence of image frames and categorizing image areas in the image frames which do not change over time as background; detecting camera movement; performing a reset of the background model by: applying an image segmentation and/or object detection algorithm to identify at least foreground objects; after having performed the reset of the background model, returning to repeatedly updating the background model for the video stream by analyzing changes in the sequence of image frames and categorizing image areas in the image frames which do not change over time as background. The disclosure further relates to an image processing system. ## Claims ### Claim 1 A computer-implemented method of background modelling for a video stream acquired by a camera having movement capabilities, the method comprising: acquiring a video stream comprising a sequence of image frames; repeatedly updating a background model for the video stream by analyzing changes in the sequence of image frames and categorizing image areas in the image frames which do not change over time as background; detecting camera movement, and stopping repeatedly updating the background model; upon detecting a camera movement stop or the camera movement going below a movement threshold, performing a reset of the background model by: acquiring an image frame; applying an image segmentation or object detection algorithm on the image frame to identify at least foreground objects, wherein the image segmentation or object detection algorithm is trained to recognize objects that are known to be capable of moving in a video stream without analyzing changes between image frames over time; and categorizing areas outside the identified foreground objects as background; and after having performed the reset of the background model, returning to repeatedly updating the background model for the video stream by analyzing changes in the sequence of image frames and categorizing image areas in the image frames which do not change over time as background. ### Claim 2 (depends on 1) The method of background modelling for a video stream according to claim 1, wherein the step of performing the reset of the background model is performed substantially immediately after the camera movement stop or the camera movement going below a movement threshold has been detected. ### Claim 3 (depends on 1) The method of background modelling for a video stream according to claim 1, wherein the camera movement comprises panning or tilting or zooming. ### Claim 4 (depends on 1) The method of background modelling for a video stream according to claim 1, further comprising the step of performing background overlaying or masking based on the background model. ### Claim 5 (depends on 1) The method of background modelling for a video stream according to claim 1, further comprising the step of, upon having detected camera movement, but before having detected a camera movement stop or the camera movement going below a movement threshold, making a background overlay visible for an image area covering the entire image frame or applying full masking or pixelation over an image area covering the entire image frame. ### Claim 6 (depends on 1) The method of background modelling for a video stream according to claim 1, wherein the step of repeatedly updating a background model is a continuous process, and wherein the reset of the background model is a single event. ### Claim 7 (depends on 1) The method of background modelling for a video stream according to claim 1, wherein the step of repeatedly updating a background model comprises analyzing sub-areas of the sequence of image frames, wherein, if no significant change occurs between the image frames in a sub-area during a predetermined period of time, the sub-area is categorized as a background sub-area. ### Claim 8 (depends on 1) The method of background modelling for a video stream according to claim 1, wherein the step of applying an image segmentation or object detection algorithm on the one or more image frames to identify foreground objects comprises thresholding or clustering or histogram-based segmentation or edge detection. ### Claim 9 (depends on 1) The method of background modelling for a video stream according to claim 1, wherein the steps of repeatedly updating a background model; detecting camera movement; detecting camera movement stop or the camera movement going below a movement threshold; and performing a reset of the background model are sequential. ### Claim 10 (depends on 1) The method of background modelling for a video stream according to claim 1, further comprising displaying the sequence of image frames on a display. ### Claim 11 (depends on 1) A non-transitory storage media comprising a computer program having instructions which, when executed by a computing device or computing system, cause the computing device or computing system to carry out the method of background modelling for a video stream according to claim 1. ### Claim 12 An image processing system comprising: at least one camera, such as a pan-tilt-zoom camera; and a processing unit configured to: acquire a video stream comprising a sequence of image frames; repeatedly update a background model for the video stream by analyzing changes in the sequence of image frames and categorizing image areas in the image frames which do not change over time as background; detect camera movement, and stop repeatedly updating the background model; upon detecting a camera movement stop or the camera movement going below a movement threshold, perform a reset of the background model by: acquiring an image frame; applying an image segmentation or object detection algorithm on the image frame to identify foreground objects, wherein the image segmentation or object detection algorithm is trained to recognize objects that are known to be capable of moving in a video stream without analyzing changes between image frames over time; and categorizing areas outside the identified foreground objects as background; and after having performed the reset of the background model, return to repeatedly updating the background model for the video stream by analyzing changes in the sequence of image frames and categorizing image areas in the image frames which do not change over time as background.