# US 2022/0044046 A1 · Device, system and method for object recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044046 A1
- Document ID: US20220044046A1
- Status: Published application
- Kind: A1
- Application number: 17/415,061
- Filed: 2019-12-16
- Published: 2022-02-10
- Earliest claimed: 2018-12-17
- Assignees: Koninklijke Philips N.V.
- Applicants: Koninklijke Philips N.V.
- Inventors: SHAKITH DEVINDA FERNANDO; LU ZHANG; ESTHER MARJAN VAN DER HEIDE; THOMAS MARIA FALCK
- Web page: https://patentgrep.com/patent/US20220044046A1

## CPC

- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)}
- G06V 10/30: Noise filtering
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

The present invention relates to a device, system and method for object recognition. To improve reliability and robustness of the recognition, the device comprises an input unit (21) configured to obtain a depth image (40) of a scene, a computation unit (22) that computes, from the depth image, a noise variance map (42) by computing pixel noise variances at object boundaries of one or more objects in the depth image, a depth confidence map (43) by filtering depth values based on their distance to the depth camera, and a motion confidence map (44) by filtering out variances caused by motion of a person in the scene. Further, from the noise variance map, the depth confidence map and the motion confidence map, one or more candidate regions (45) and their confidence in the depth image are computed, and the one or more candidate regions having the highest confidence are selected as final region of interest (41) representing the object to be recognized.

## Claims

### Claim 1

A device for object recognition, said device comprising:
an input for obtaining a depth image of a scene via a depth camera, the depth image comprising depth information representing a distance between the depth camera and elements of the scene depicted in the depth image,
a processor for:
computing, from the depth image,
a noise variance map by computing pixel noise variances at object boundaries of one or more objects in the depth image,
a depth confidence map by filtering depth values based on their distance to the depth camera, and
a motion confidence map by filtering out variances caused by motion of a person in the scene,
computing, from the noise variance map, the depth confidence map and the motion confidence map, one or more candidate regions and their confidence in the depth image, a candidate region being a region potentially representing an object or a part of the object, and
selecting the one or more candidate regions having the highest confidence as final region of interest representing the object to be recognized; and
an output for generating a representation of a final region of interest (ROT) to be utilized by an end user.

### Claim 2 (depends on 1)

The device as claimed in claim 1, wherein the object to be recognized is a bed.

### Claim 3 (depends on 1)

The device claimed in claim 1,
wherein the processor is configured to compute the noise variance map by computing pixel noise variances at boundaries of the object to be recognized and of one or more other objects occluding one or more parts of the object to be recognized in the depth image.

### Claim 4 (depends on 1)

The device as claimed in claim 1, wherein the processor is configured to compute the noise variance map by use of a noise model that models one or more noise factors.

### Claim 5 (depends on 4)

The device as claimed in claim 4,
wherein the processor is configured to compute the noise variance map by use of a noise model that models at least one noise factor of noise factors including absorptivity or reflectivity of the material of an object, reflections of light from different objects reaching the same pixel, temporal variations depending on when a reflected light reaches the same pixel over time, or more pixels having a zero pixel value when no light reaches a pixel or light that would reach a pixel is compensated by other light.

### Claim 6 (depends on 1)

The device as claimed in claim 1, wherein the processor is configured to compute the depth confidence map by filtering out depth values of pixels lying outside a depth range assigned to the object to be recognized.

### Claim 7 (depends on 6)

The device as claimed in claim 6, wherein the processor is configured to apply an adaptive filter that adaptively changes the depth range applied for filtering.

### Claim 8 (depends on 6)

The device as claimed in claim 6, wherein the processor is configured to compute the depth confidence map by use of an object model which models the depth of the object to be recognized.

### Claim 9 (depends on 1)

The device as claimed in claim 1, wherein the processor is configured to compute the motion confidence map by using the time duration to induce pixel variations to differentiate between pixel variations caused by motion and pixel variations caused by noise.

### Claim 10 (depends on 1)

The device as claimed in claim 1,
wherein the processor is configured to compute the one or more candidate regions by computing a joint confidence map from the noise variance map, the depth confidence map and the motion confidence map and to apply contour detection on the joint confidence map to detect contours in the depth image, said contours indicating the one or more candidate regions.

### Claim 11 (depends on 10)

The device as claimed in claim 10,
wherein the processor is configured to compute the confidence of the one or more candidate regions by use of a Gaussian distribution on the respective candidate region and multiplying it by the joint confidence map to obtain a region confidence map and to select the one or more candidate regions having the highest confidence in the joint confidence map as final region of interest representing the object to be recognized.

### Claim 12 (depends on 1)

The device as claimed in claim 1, wherein the processor is configured to:
rank the one or more candidate regions according to their confidence,
iteratively combine candidate regions according to their rank,
compute the sum of their confidence at every iteration,
stop the iteration when the computed sum of the confidence converges, and
select the candidate regions combined up to stop of the iteration as final region of interest representing the object to be recognized.

### Claim 13

A system for object recognition, said system comprising:
a depth camera for acquiring a depth image of a scene, the depth image comprising depth information representing the distance between the depth camera and elements of the scene depicted in the depth image;
and
a device for object recognition based on the acquired depth image, said device comprising:
an input for obtaining a depth image of a scene via a depth camera, the depth image comprising depth information representing a distance between the depth camera and elements of the scene depicted in the depth image,
a processor for:
computing, from the depth image,
a noise variance map by computing pixel noise variances at object boundaries of one or more objects in the depth image,
a depth confidence map by filtering depth value based on their distance to the depth camera, and
a motion confidence map by filtering out variances caused by motion of a person in the scene,
computing, from the noise variance map, the depth confidence map and the motion confidence map, one or more candidate regions and their confidence in the depth image, a candidate region being a region potentially representing an object or a part of the object, and
selecting the one or more candidate regions baying the highest confidence as final region of interest representing the object to be recognized; and
an output for generating a representation of a final region of interest (ROT) to be utilized by an end user.

### Claim 14

A method for object recognition, said method comprising:
obtaining a depth image of a scene, the depth image comprising depth information representing the distance between a depth camera and elements of the scene depicted in the depth image,
computing, from the depth image,
a noise variance map by computing pixel noise variances at object boundaries of one or more objects in the depth image,
a depth confidence map by filtering depth values based on their distance to the depth camera, and
a motion confidence map by filtering out variances caused by motion of a person in the scene,
computing, from the noise variance map, the depth confidence map and the motion confidence map, one or more candidate regions and their confidence in the depth image, a candidate region being a region potentially representing an object or a part of the object, and
selecting the one or more candidate regions having the highest confidence as final region of interest representing the object to be recognized.

### Claim 15 (depends on 14)

A non-transitory computer-readable medium that stores therein a computer program product, which, when executed on a processor, causes the processor to carry out the steps of the method as claimed in claim 14.

### Claim 16 (depends on 5)

The device of claim 5, wherein the object model is a Gaussian object model.

### Claim 17 (depends on 8)

The device of claim 8, wherein the object model is a Gaussian object model.
