# US 12,217,477 B2 · Object recognition method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,477 B2
- Document ID: US12217477B2
- Status: Granted
- Kind: B2
- Application number: 17/680,668
- Filed: 2022-02-25
- Published: 2022-06-09
- Granted: 2025-02-04
- Earliest claimed: 2019-08-30
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Gang Pan; Qianhui Liu; Lei Jiang; Jie Cheng; Haibo Ruan; Dong Xing; Huajin Tang; De Ma
- Web page: https://patentgrep.com/patent/US12217477B2

## CPC

- G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
- G06N 3/045: Combinations of networks
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06N 3/0499: Feedforward networks
- G06N 3/09: Supervised learning
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/77: Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

In an object recognition method, an object recognition device obtains AER data of a to-be-recognized object, wherein the AER data includes a plurality of AER events of the to-be-recognized object, each AER event comprising a timestamp and address information. The object recognition device extracts a plurality of feature maps of the AER data. Each feature map including partial spatial information and partial temporal information of the to-be-recognized object, and the partial spatial information and the partial temporal information are obtained based on the timestamp and the address information of each AER event. The object recognition device then recognizes the to-be-recognized object based on the plurality of feature maps of the AER data.

## Claims

### Claim 1

An object recognition method comprising:
obtaining address event representation (AER) data of a to-be-recognized object, wherein the AER data comprises a plurality of AER events of the to-be-recognized object, and each AER event comprises a timestamp and address information;
extracting a plurality of feature maps of the AER data, wherein each feature map comprises partial spatial information and partial temporal information of the to-be-recognized object, and the partial spatial information and the partial temporal information are obtained based on the timestamp and the address information of each AER event; and
recognizing the to-be-recognized object based on the plurality of feature maps of the AER data,
wherein the step of extracting the plurality of feature maps of the AER data comprises:
processing address information of the plurality of AER events, by using a plurality of filters, to obtain a plurality of first feature maps; and
attenuating, as a function of time, feature values in the plurality of first feature maps based on timestamps of the plurality of AER events, to obtain the plurality of feature maps of the AER data.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the step of processing address information of the plurality of AER events comprises:
performing convolution processing on spatial information of the plurality of AER events by using convolution kernels of a plurality of Gabor filters to obtain the plurality of first feature maps.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the step of recognizing the to-be-recognized object comprises:
encoding the plurality of feature maps of the AER data to obtain a plurality of spike trains, wherein each spike train comprises a plurality of spikes, each spike carries partial temporal information and partial spatial information of the to-be-recognized object, and a plurality of spikes belonging to a same spike train are obtained based on feature values at same locations in feature maps corresponding to different filters that are in a same specified direction; and
processing the plurality of spike trains by using a spiking neural network, to recognize the to-be-recognized object.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the step of encoding the plurality of feature maps of the AER data to obtain the plurality of spike trains comprises:
encoding the plurality of feature maps of the AER data by using a target coding function to obtain the plurality of spike trains, wherein the target coding function is an inverse linear function or an inverse logarithmic function.

### Claim 5

An object recognition apparatus comprising:
an interface for receiving address event representation (AER) data from an AER sensor;
a processor configured to perform operations of:
obtaining, from the AER sensor via the interface, AER data of a to-be-recognized object, wherein the AER data comprises a plurality of AER events of the to-be-recognized object, and each AER event comprises a timestamp and address information;
extracting a plurality of feature maps of the AER data, wherein each feature map comprises partial spatial information and partial temporal information of the to-be-recognized object, and the partial spatial information and the partial temporal information are obtained based on the timestamp and the address information of each AER event; and
recognizing the to-be-recognized object based on the plurality of feature maps of the AER data,
wherein the operation of extracting the plurality of feature comprises:
processing address information of the plurality of AER events by using a plurality of filters, to obtain a plurality of first feature maps; and
attenuating, as a function of time, feature values in the plurality of first feature maps based on timestamps of the plurality of AER events, to obtain the plurality of feature maps of the AER data.

### Claim 6 (depends on 5)

The apparatus according to claim 5, wherein the operation of processing the address information of the plurality of AER events comprises:
performing convolution processing on spatial information of the plurality of AER events by using convolution kernels of a plurality of Gabor filters, to obtain the plurality of first feature maps.

### Claim 7 (depends on 5)

The apparatus according to claim 5, wherein the operation of recognizing the to-be-recognized object comprises:
encoding the plurality of feature maps of the AER data to obtain a plurality of spike trains, wherein each spike train comprises a plurality of spikes, each spike carries partial temporal information and partial spatial information of the to-be-recognized object, and a plurality of spikes belonging to a same spike train are obtained based on feature values at same locations in feature maps corresponding to different filters that are in a same specified direction; and
processing the plurality of spike trains by using a spiking neural network, to recognize the to-be-recognized object.

### Claim 8 (depends on 7)

The apparatus according to claim 7, wherein the operation of encoding the plurality of feature maps of the AER data to obtain the plurality of spike trains comprises:
encoding the plurality of feature maps of the AER data by using a target coding function, to obtain the plurality of spike trains, wherein the target coding function is an inverse linear function or an inverse logarithmic function.