# US 2019/0377940 A1 · People detection system with feature space enhancement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0377940 A1
- Document ID: US20190377940A1
- Status: Published application
- Kind: A1
- Application number: 16/436,008
- Filed: 2019-06-10
- Published: 2019-12-12
- Earliest claimed: 2018-06-12
- Assignees: Capillary Technologies International Pte Ltd.
- Applicants: Capillary Technologies International Pte Ltd
- Inventors: Sumandeep Banerjee; Subrat Panda; Doney Alex
- Web page: https://patentgrep.com/patent/US20190377940A1

## CPC

- G06V 10/267: {by performing operations on regions, e.g. growing, shrinking or watersheds}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 20/53: {Recognition of crowd images, e.g. recognition of crowd congestion}
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A people detection system with feature space enhancement is provided. The system includes a memory having computer-readable instructions stored therein. The system includes a processor configured to access a plurality of video frames captured using one or more overhead video cameras installed in a space and to extract one or more raw images of the space from the plurality of video frames. The processor is further configured to process the one or more raw images to generate a plurality of positive image samples and a plurality of negative image samples. The positive image samples include images having one or more persons present within the space and the negative image samples comprise images without the persons. The processor is configured to apply at least one of a crop factor and a resize factor to the positive and the negative image samples to generate curated positive and negative image samples and to detect one or more persons present in the space using a detection model trained by the curated positive and negative image samples.

## Claims

### Claim 1

A people detection system with feature space enhancement, the system comprising:
a memory having computer-readable instructions stored therein;
a processor configured to:
access a plurality of video frames captured using one or more overhead video cameras installed in a space;
extract one or more raw images of the space from the plurality of video frames;
process the one or more raw images to generate a plurality of positive image samples and a plurality of negative image samples, wherein the positive image samples comprise images having one or more persons present within the space and the negative image samples comprise images without the persons;
apply at least one of a crop factor and a resize factor to the positive and the negative image samples to generate curated positive and negative image samples; and
detect one or more persons present in the space using a detection model trained by the curated positive and negative image samples.

### Claim 2 (depends on 1)

The people detection system of claim 1, wherein the processor is further configured to execute the computer-readable instructions to:
annotate the raw images to identify one or more persons in the images;
extract annotated portions of raw images and to rotate the extracted portions at a plurality of rotation angles to generate rotated image samples; and
combine the rotated image samples with a plurality of backgrounds to generate the plurality of positive image samples.

### Claim 3 (depends on 2)

The people detection system of claim 2, wherein the processor is further configured to execute the computer-readable instructions to utilize pyramid blending for generating the positive image samples, wherein pyramid blending substantially minimizes effects of boundary artifacts.

### Claim 4 (depends on 1)

The people detection system of claim 1, wherein the processor is further configured to execute the computer-readable instructions to:
generate a plurality of sub-images using the negative image samples;
select sub-images having a resolution substantially similar to resolution of the positive image samples.

### Claim 5 (depends on 1)

The people detection system of claim 1, wherein the processor is further configured to execute the computer-readable instructions to:
extract a plurality of channel features using the positive and the negative image samples; and
determine feature vectors using the extracted channel features.

### Claim 6 (depends on 1)

The people detection system of claim 1, wherein the processor is further configured to:
iteratively train the detection model using the feature vectors corresponding to the curated positive and negative image samples, wherein the detection model comprises a plurality of classifiers;
perform negative sample mining using intermediate partially trained model, and
adjust the crop factor and/or resize factor.

### Claim 7 (depends on 6)

The people detection system of claim 6, wherein the detection model comprises an AdaBoost classifier for classification and detection of one or more persons present in the space.

### Claim 8 (depends on 7)

The people detection system of claim 7, wherein the processor is further configured to execute the computer-readable instructions to utilize a set of weak classifiers to separate the negative image samples based on a pre-determined set of boundaries.

### Claim 9 (depends on 1)

The people detection system of claim 1, wherein the processor is further configured to execute the computer-readable instructions to:
determine the crop factor to prevent detection of two persons present in close proximity as one person; and
crop the image samples in accordance with the crop factor.

### Claim 10 (depends on 9)

The people detection system of claim 9, wherein the processor is further configured to execute the computer-readable instructions to select the crop factor to remove portions corresponding to shoulders of the two persons present in close proximity.

### Claim 11 (depends on 1)

The people detection system of claim 1, wherein the processor is further configured to execute the computer-readable instructions to determine the resize factor to obtain a desired detection accuracy at a pre-determined frame rate.

### Claim 12 (depends on 1)

The people detection system of claim 1, wherein the processor is further configured to execute the computer-readable instructions to detect location and size of the detected persons present in the space.

### Claim 13

A people detection system with feature space enhancement, the system comprising:
a plurality of overhead video cameras installed within a space, each of the plurality of overhead video cameras configured to capture real-time video of the space;
a detector communicatively coupled to the plurality of overhead video cameras and configured to detect one or more persons present in the space, wherein the detector comprises:
an image processing module configured to process video frames of the captured video to generate a plurality of positive image samples and a plurality of negative image samples, wherein the positive image samples comprise images having one or more persons present within the space and the negative image samples comprise images without the persons;
a feature space estimation module configured to determine one or more feature space parameters, wherein the feature space parameters comprise a crop factor, a resize factor, or combinations thereof; and
a detector module configured to apply the one or more feature space parameters to the positive and the negative image samples to generate curated positive and negative image samples and to detect one or more persons present in the space using a detection model trained by the curated positive and negative image samples.

### Claim 14 (depends on 13)

The people detection system of claim 13, wherein the image processing module is further configured to:
extract raw images from the video frames;
annotate the raw images to identify one or more persons in the images;
extract annotated portions of raw images and to rotate the extracted portions at a plurality of rotation angles to generate rotated image samples; and
combine the rotated image samples with a plurality of backgrounds to generate the plurality of positive image samples.

### Claim 15 (depends on 13)

The people detection system of claim 13, wherein the detection module is further configured to:
iteratively train a detection model using feature vectors corresponding to the curated positive and negative image samples, wherein the detection model comprises a plurality of classifiers;
perform negative sample mining using the intermediate partially trained model, and
adjust the crop factor and/or resize factor.

### Claim 16 (depends on 15)

The people detection system of claim 15, wherein the detection model comprises AdaBoost classifier for classification and detection of one or more persons present in the space.

### Claim 17 (depends on 16)

The people detection system of claim 16, wherein the detection model is further configured to estimate a detection accuracy and a classification accuracy of the model.

### Claim 18

A computer-implemented method for detecting persons in a space, the method comprising:
accessing a plurality of video frames captured using one or more overhead video cameras installed in a space;
extracting one or more raw images of the space from the plurality of video frames;
processing the one or more raw images to generate a plurality of positive image samples and a plurality of negative image samples, wherein the positive image samples comprise images having one or more persons present within the space and the negative image samples comprise images without the persons;
applying at least one of a crop factor and a resize factor to the positive and the negative image samples to generate curated positive and negative image samples; and
detecting one or more persons present in the space using the curated positive and negative image samples.

### Claim 19 (depends on 18)

The computer implemented method of claim 18, further comprising tracking number of persons entering or exiting the space using Kalman correction and Hungarian Assignment solver.

### Claim 20 (depends on 18)

The computer implemented method of claim 18, further comprising:
iteratively training a detection model using the curated positive and negative image samples, wherein the detection model comprises a plurality of classifiers;
performing negative sample mining using the intermediate partially trained model, and
adjusting the crop factor and/or resize factor.
