# US 2022/0012857 A1 · Method, computer program and system for detecting changes and moving objects in a video view

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0012857 A1
- Document ID: US20220012857A1
- Status: Published application
- Kind: A1
- Application number: 17/349,252
- Filed: 2021-06-16
- Published: 2022-01-13
- Earliest claimed: 2020-06-17
- Assignees: Pexip AS
- Applicants: Pexip AS
- Inventors: Håvard Graff
- Web page: https://patentgrep.com/patent/US20220012857A1

## CPC

- H04N 19/85: using pre-processing or post-processing specially adapted for video compression
- G06T 5/30: Erosion or dilatation, e.g. thinning
- G06T 7/11: Region-based segmentation
- G06T 7/155: involving morphological operators
- G06T 7/194: involving foreground-background segmentation
- G06T 7/248: {involving reference images or patches}
- G06T 7/254: involving subtraction of images
- G06V 10/34: Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V 40/167: {using comparisons between temporally consecutive images}
- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/20052: Discrete cosine transform [DCT]
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present invention relates to an image processing device and a method of framing changes and movements in a video image divided into N×N blocks of pixel positions. The method comprises calculating a first bitmap of the video image by a DCT transform on each of the N×N blocks of pixel positions, assigning a first binary value to the pixel positions of the N×N blocks when more than an amount of change, and a second binary value to the pixel positions of the N×N blocks when less than an amount of change. Calculating a third bitmap by an OR operation between a number of bitmaps representing past time frames of the video image, calculating a fourth bitmap by performing a dilation process of the third bitmap representing the current time frame of the video image, and creating one or more frames identifying area of changes and movements in the video image based on detecting BLOBs (Binary Large Objects) in the fourth bitmaps.

## Claims

### Claim 1

A method of framing changes and movements in a video image divided into N×N blocks of pixel positions, comprising:
calculating a first bitmap representing a current time frame of the video image by a DCT (Discrete Cosine Transform) transform on each of the N×N blocks of pixel positions determining the extent of change in the respective N×N block by assigning a first one of a binary value to the pixel positions of the N×N blocks considered to have more than a predefined amount of change, and a second one of the binary value to the pixel positions of the N×N blocks considered to have less than a predefined amount of change;
calculating a third bitmap representing the current time frame of the video image by performing an OR operation on a number of first bitmaps representing past time frames of the video image;
calculating a fourth bitmap representing the current time frame of the video image by performing a dilation process of the third bitmap representing the current time frame of the video image; and
creating one or more frames identifying area of changes and movements in the video image based on detecting BLOBs (Binary Large Objects) in the fourth bitmaps.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
calculating a second bitmap representing the current time frame of the video image by performing an erosion process of the first bitmap representing the current time frame of the video image, and the step of calculating the third bitmap by performing the OR operation on the number of first bitmaps representing past time frames of the video image is performed by an OR operation on a number of second bitmaps representing past time frames of the video image.

### Claim 3 (depends on 2)

The method of claim 2, wherein the erosion process includes performing a convolution between the first bitmap and the following first kernel:
0, 1, 0,
1, 1, 1,
0, 1, 0,

### Claim 4 (depends on 1)

The method of claim 1, wherein the step of calculating the first bitmap further includes:
calculating DCT coefficients of each of the N×N dimensioned block in the video image,
comparing each of the DCT coefficients of the N×N blocks with a predefined first threshold,
if all or substantially all of the DCT coefficients of the N×N blocks are below a predefined first threshold, then the pixel positions of the respective N×N block is assigned to the second one of the binary value “0”,
if not, the pixel positions of the respective N×N block is assigned to the first one of the binary value “1”.

### Claim 5 (depends on 1)

The method of claim 1, wherein the dilution process includes performing a convolution between the third bitmap and the following second kernel:
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,

### Claim 6 (depends on 1)

The method of claim 1, wherein N=8.

### Claim 7 (depends on 1)

The method of claim 1, wherein the number of first bitmaps representing past time frames is 30.

### Claim 8 (depends on 1)

The method of claim 1, wherein the step of creating one or more frames identifying area of changes and movements includes extracting the respective left bottom (LB) and right top (RT) pixel positions of each of the detected BLOBs.

### Claim 9

An image processing device for framing changes and movements in a video image divided into N×N blocks of pixel positions, comprising
at least one processor,
an input/output circuitry adapted to receive the video image,
at least one memory,
the memory comprising instructions that, when executed by the at least one processor, cause the image processing device to:
calculating a first bitmap representing a current time frame of the video image by a DCT transform on each of the N×N blocks of pixel positions determining the extent of change in the respective N×N block by assigning a first one of a binary value to the pixel positions of the N×N blocks considered to have more than a predefined amount of change, and a second one of the binary value to the pixel positions of the N×N blocks considered to have less than a predefined amount of change,
calculating a third bitmap representing the current time frame of the video image by performing an OR operation one a number of first bitmaps representing past time frames of the video image,
calculating a fourth bitmap representing the current time frame of the video image by performing a dilation process of the third bitmap representing the current time frame of the video image,
creating one or more frames identifying area of changes and movements in the video image based on detecting BLOBs (Binary Large Objects) in the fourth bitmaps.

### Claim 10 (depends on 9)

The image processing device of claim 9, wherein the instructions when executed by at least one processor, further causes the image processing device to perform the following additional step:
calculating a second bitmap representing the current time frame of the video image by performing an erosion process of the first bitmap representing the current time frame of the video image, and the step of calculating the third bitmap by performing the OR operation on the number of first bitmaps representing past time frames of the video image is performed by an OR operation on a number of second bitmaps representing past time frames of the video image.

### Claim 11 (depends on 10)

The image processing device of claim 10, wherein the erosion process includes performing a convolution between the first bitmap and the following first kernel:
0, 1, 0,
1, 1, 1,
0, 1, 0,

### Claim 12 (depends on 9)

The image processing device of claim 9, wherein the step of calculating the first bitmap further includes:
calculating DCT coefficients of each of the N×N dimensioned block in the video image,
comparing each of the DCT coefficients of the N×N blocks with a predefined first threshold,
if all or substantially all of the DCT coefficients of the N×N blocks are below a predefined first threshold, then the pixel positions of the respective N×N block is assigned to the second one of the binary value “0”,
if not, the pixel positions of the respective N×N block is assigned to the first one of the binary value “1”.

### Claim 13 (depends on 9)

The image processing device of claim 9, wherein the dilution process includes performing a convolution between the third bitmap and the following second kernel:
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1,

### Claim 14 (depends on 9)

The image processing device of claim 9, wherein N=8.

### Claim 15 (depends on 9)

The image processing device of claim 9, wherein the number of first bitmaps representing past time frames is 30.

### Claim 16 (depends on 9)

The image processing device of claim 9, wherein the step of creating one or more frames identifying area of changes and movements includes extracting the respective left bottom (LB) and right top (RT) pixel positions of each of the detected BLOBs.

### Claim 17 (depends on 1)

A computer program product comprising a non-transitory computer readable storage medium comprising instructions that, when executed on a processor in an image processing device, enables the image processing device to perform the steps of the method according to claim 1.
