# US 2017/0286779 A1 · Object detection in crowded scenes using context-driven label propagation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0286779 A1
- Document ID: US20170286779A1
- Status: Published application
- Kind: A1
- Application number: 15/087,236
- Filed: 2016-03-31
- Published: 2017-10-05
- Earliest claimed: 2016-03-31
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Quanfu Fan; Jingjing Liu; Sharathchandra U. Pankanti
- Web page: https://patentgrep.com/patent/US20170286779A1

## CPC

- G06V 10/768: {using context analysis, e.g. recognition aided by known co-occurring patterns}
- G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06V 10/764: using classification, e.g. of video objects
- G06V 20/53: {Recognition of crowd images, e.g. recognition of crowd congestion}
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A computer implemented method for detecting an object in a crowded scene utilizing an image capturing device. The method includes receiving an image of a predetermined area. From the image, the existence of selected portions as representing an entity of a selected class is determined. Each selected portion is assigned an initial confidence value that the selected portion is an entity representative of a selected class. Each selected portion is evaluated with each other to determine a context confidence value. The context confidence value and initial confidence value are utilized to determine which of the one or more selected portions are entities of a selected class.

## Claims

### Claim 1

A computer implemented method for detecting an object in a scene, the method comprising:
receiving, using a processor system, image data representing objects of a predetermined area;
making an initial determination, using the processor system, that selected portions of the image data represent one or more entities of a selected class;
determining an initial confidence value of the selected portions of the image data, wherein the initial confidence value identifies a likelihood that the selected portions of the image data represent one or more entities of a selected class;
determining a context confidence value of the selected portions of the image data, wherein the context confidence value identifies a contextual relationship between the selected portions of the image data; and
utilizing said context confidence value and said initial confidence value to determine that one of the selected portions of the image data represents an entity of a selected class.

### Claim 2 (depends on 1)

The computer implemented method of claim 1 wherein said step of making an initial determination that selected portions of the image data represent an entity of a selected task includes utilizing facial recognition attributes and assigning a confidence value based upon said facial recognition attributes that said entity is an individual.

### Claim 3 (depends on 2)

The computer implemented method of claim 2 wherein said initial confidence value includes a threshold value being assigned correlating to the selected portions of the image data representing one or more selected entities and wherein if said initial confidence value is beneath said threshold value said selected portion is deemed not being an entity of a selected class and is not included in the evaluation of other selected portions of the image data.

### Claim 4 (depends on 1)

The computer implemented method of claim 1 wherein a selected portion of image data is compared to another selected portion of image data via a scale context and said scale context compares the height of a first identified object with a second identified object.

### Claim 5 (depends on 1)

The computer implemented method of claim 1 wherein a selected portion of image data is compared to another selected portion of image data via a spatial context wherein the spatial distance between two selected portions selected portions is calculated.

### Claim 6 (depends on 1)

The computer implemented method of claim 1 wherein a selected portion of image data is compared to another selected portion of image data via a social context wherein the social context measures the relationship of pose and body orientations to determine if a selected portion of image data is facing or following another selected portion of image data.

### Claim 7 (depends on 1)

The computer implemented method of claim 1 wherein an image boundary of a selected portion is compared to an image boundary of an adjacent selected portion for determining if an overlap context exists such that the two selected portions are determined to be representative of the same individual.

### Claim 8 (depends on 1)

The computer implemented method of claim 1 wherein an identified object is compared to another identified object via a scale context and said scale context compares the height of a first identified object with a second identified object, via a spatial context wherein the spatial distance between two identified object is calculated, via a social context wherein the social context measures the relationship of pose and body orientations to determine if a identified object is facing or following another identified object and providing an aggregate context confidence value.

### Claim 9 (depends on 8)

The computer implemented method of claim 8 wherein said spatial, scale, overlap and social contexts include an attraction component and said spatial, overlap and scale contexts include a repulsion component, said aggregate context confidence value being comprised of the spatial, scale, overlap and social attraction components and said spatial overlap and scale repulsion components with respect to a respective selected portion in relation to additional selected portions within an image.

### Claim 10 (depends on 9)

The computer implemented method of claim 9 wherein a selected portion of an image is deemed not to be an individual if said aggregate context confidence value is less than a threshold.

### Claim 11 (depends on 9)

The computer implemented method of claim 9 further including moving a camera to capture an additional image based upon the detection of an individual at an outer boundary of said initial image capture.

### Claim 12 (depends on 11)

The computer implemented method of claim 11 further including moving a camera to zoom into a specific selected portion of the image if a respective context indicates unusual behavior.

### Claim 13

A computer program product for detecting individuals in a crowded scene, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions readable by a processing circuit to cause the processing circuit to perform a method comprising:
receiving image data representing objects of a predetermined area;
making an initial determination, that selected portions of the image data represent one or more entities of a selected class;
determining an initial confidence value of selected portions of the image data, wherein the initial confidence value identifies a likelihood that the selected portions of the image data represent one or more entities of a selected class;
utilizing said context confidence value and said initial confidence value to determine that one of the selected portions of the image data represents an entity of a selected class.

### Claim 14 (depends on 13)

The computer program product of claim 13 wherein said step of making an initial determination that selected portions of the image data represent one or more entities of a selected class includes utilizing facial recognition attributes and assigning a confidence value based upon said facial recognition attributes that said entity is an individual.

### Claim 15 (depends on 14)

The computer program product of claim 14 wherein a selected portion of image data is compared to another selected portion of image data via a scale context and said scale context compares the height of a first identified object with a second identified object, via a spatial context wherein the spatial distance between two selected portions is calculated, via a social context wherein the social context measures the relationship of pose and body orientations to determine if a selected portion is facing or following another selected portion and providing an aggregate context confidence value.

### Claim 16 (depends on 14)

The computer program product of claim 14 wherein said spatial, scale, overlap and social contexts include an attraction component and said spatial, overlap and scale contexts include a repulsion component, said aggregate context confidence value being comprised of the spatial, scale, overlap and social attraction components and said spatial overlap and scale repulsion components with respect to a respective selected portion of image data in relation to an additional selected portion of image data within an image.

### Claim 17 (depends on 14)

The computer program product of claim 14 wherein a selected portion of image data is deemed not to be an individual if said aggregate context confidence value is less than a threshold.

### Claim 18

An individual detection system for detecting individuals in a crowded scene utilizing an image capturing device, the detection system comprising:
a processor in communication with one or more types of memory, the processor configured to:
receive an image data representing objects of a predetermined area;
making an initial determination that selected portions of the image data represent one or more individuals;
determining an initial confidence value of the selected portions of the image data, wherein the initial confidence value identifies a likelihood that the selected portions of the image data represents one or more individuals;
determining a context confidence value of the selected portions of the image data, wherein the context confidence value identifies a contextual relationship between the selected portions of the image data; and
utilizing said context confidence value and said initial confidence value to determine that one of the selected portions of the image data represents a selected portions individual.

### Claim 19 (depends on 18)

The detection system of claim 18 wherein said processor is further configured to determine an overall context confidence value by comparing a identified object with another identified object via a scale context and said scale context compares the height of a first identified object with a second identified object, via a spatial context wherein the spatial distance between two identified object is calculated, via a social context wherein the social context measures the relationship of pose and body orientations to determine if a identified object is facing or following another identified object and providing an aggregate context confidence value.

### Claim 20 (depends on 19)

The detection system of claim 19 wherein said processor is further configured to manipulate a camera depending on the value of a certain context confidence value with respect to a respective selected portion.
