# US 2023/0055581 A1 · Privacy preserving anomaly detection using semantic segmentation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0055581 A1
- Document ID: US20230055581A1
- Status: Published application
- Kind: A1
- Application number: 17/498,537
- Filed: 2021-10-11
- Published: 2023-02-23
- Earliest claimed: 2021-08-12
- Applicants: Milestone Systems A/S
- Inventors: Michael BIDSTRUP; Jacob Velling DUEHOLM; Kamal NASROLLAHI
- Web page: https://patentgrep.com/patent/US20230055581A1

## CPC

- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- G06F 21/6254: {by anonymising data, e.g. decorrelating personal data from the owner's identification}
- G06N 3/045: Combinations of networks
- G06T 7/11: Region-based segmentation
- G06T 7/12: Edge-based segmentation
- G06T 11/00: Two-dimensional [2D] image generation
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V 10/778: Active pattern-learning, e.g. online learning of image or video features
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay
- H04N 7/181: {for receiving images from a plurality of remote sources}
- H04N 7/183: {for receiving images from a single remote source}
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30232: Surveillance
- G06V 20/44: {Event detection}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A computer implemented method of anonymising video surveillance data of a scene and detecting an object or event of interest in such anonymised video surveillance data, the method comprising segmenting frames of video surveillance data of at least one scene into corresponding frames of segmented data using image segmentation, wherein a mask label is assigned to every pixel of each frame of the segmented data based either on a class of objects or of surfaces or on an instance of such a class that pixel belongs to, and detecting at least one object and/or event of interest based on at least one shape and/or motion in at least one frame of the segmented data.

## Claims

### Claim 1

A computer implemented method of anonymising video surveillance data of a scene and detecting an object or event of interest in such anonymised video surveillance data, the method comprising:
segmenting frames of video surveillance data of at least one scene into corresponding frames of segmented data using image segmentation, wherein a mask label is assigned to every pixel of each frame of the segmented data based either on a class of objects or of surfaces or on an instance of such a class that pixel belongs to; and
detecting at least one object and/or event of interest based on at least one shape and/or motion in at least one frame of the segmented data.

### Claim 2 (depends on 1)

A computer implemented method according to claim 1, wherein segmenting frames of video surveillance data comprises carrying out semantic segmentation of the said video surveillance data.

### Claim 3 (depends on 2)

A computer implemented method according to claim 2, wherein segmenting frames of video surveillance data comprises carrying out image segmentation with a first artificial neural network.

### Claim 4 (depends on 1)

A computer implemented method according to claim 1, further comprising determining a user's right to view the video surveillance data, the segmented data and/or at least a part thereof, and displaying to the user the video surveillance data, the segmented data and/or at least a part thereof, based on that determination.

### Claim 5 (depends on 1)

A computer implemented method according to claim 1, wherein each segmented frame comprises all segments obtained from a corresponding frame of the video surveillance data.

### Claim 6 (depends on 1)

A computer implemented method according to claim 1, further comprising acquiring the video surveillance data from at least one physical video camera, and wherein segmenting the video surveillance data comprises segmenting the video surveillance data within the physical video camera.

### Claim 7 (depends on 1)

A computer implemented method according to claim 1, wherein the video surveillance data comprises video surveillance data of different scenes from a plurality of physical video cameras.

### Claim 8 (depends on 1)

A computer implemented method according to claim 1, further comprising storing in a recording server the said at least one frame of segmented data based on which the said object or event of interest has been detected.

### Claim 9 (depends on 1)

A computer implemented method according to claim 1, wherein each segment substantially traces the contour of one or more objects or surfaces represented by that segment.

### Claim 10 (depends on 1)

A computer implemented method according to claim 1, wherein each segment is represented as a colour.

### Claim 11 (depends on 1)

A computer implemented method according to claim 1, further comprising generating a composite video and/or image of the video surveillance data on which at least one segment is represented, and providing anonymity to an object or surface in the video surveillance data by masking that object or surface with that segment.

### Claim 12 (depends on 1)

A computer implemented method according to claim 1, further comprising enhancing at least part of at least one segment based on a predetermined change between two or more frames in the video surveillance data, such that detecting the said at least one object or event of interest is facilitated.

### Claim 13 (depends on 12)

A computer implemented method according to claim 12, wherein the said predetermined change comprises a change in an appearance or motion of the said at least one object between the said two or more frames.

### Claim 14 (depends on 1)

A computer implemented method according to claim 1, wherein detecting at least one object or event of interest comprises carrying out anomaly detection.

### Claim 15 (depends on 1)

A computer implemented method according to claim 1, wherein detecting at least one object or event of interest comprises carrying out detection with a second artificial neural network.

### Claim 16 (depends on 1)

A computer implemented method according to claim 1, wherein the objects in the said class of objects are chosen from a group consisting of people and vehicles.

### Claim 17

A video processing apparatus, comprising at least one processor configured to:
segment frames of video surveillance data of at least one scene into corresponding frames of segmented data using image segmentation, wherein a mask label is assigned to every pixel of each frame of the segmented data based either on a class of objects or of surfaces or on an instance of such a class that pixel belongs to;
and configured to detect at least one object and/or event of interest based on at least one shape and/or motion in at least one frame of the segmented data.

### Claim 18 (depends on 17)

A video processing apparatus according to claim 17, wherein the said at least one processor is configured to segment the video surveillance data by carrying out semantic segmentation of the said video surveillance data.

### Claim 19 (depends on 18)

A video processing apparatus according to claim 18, wherein detecting at least one object or event of interest comprises carrying out anomaly detection.

### Claim 20 (depends on 17)

A video surveillance system comprising a video processing apparatus according to claim 17 and a client apparatus comprising a display, the client apparatus comprising at least one processor configured to determine a user's right to view the video surveillance data, the segmented data and/or at least a part thereof, the at least one processor of the client apparatus being further configured to display to the user the video surveillance data, the segmented data and/or at least a part thereof, based on that determination.
