# US 11,907,046 B2 · Distributed object identification system and method, and edge computing device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,907,046 B2
- Document ID: US11907046B2
- Status: Granted
- Kind: B2
- Application number: 17/709,768
- Filed: 2022-03-31
- Published: 2022-07-14
- Granted: 2024-02-20
- Earliest claimed: 2020-04-09
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Wenxing Lai
- Web page: https://patentgrep.com/patent/US11907046B2

## CPC

- G06F 11/0709: {in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems}
- G06F 11/0751: {Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06)}
- G06F 11/0793: {Remedial or corrective actions (recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40)}
- G06F 11/2033: {switching over of hardware resources}
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 20/48: {Matching video sequences}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/172: {Classification, e.g. identification}
- G06F 11/3006: {where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50)}
- G06F 11/3055: {Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available (error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction of the data by redundancy in operations G06F11/14; error detection or correction by redundancy in hardware G06F11/16)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An edge computing device is provided including at least one memory configured to store computer program code and at least one processor configured to access said computer program code and operate as instructed by said computer program code. The edge computing device is included in a distributed object identification system, which includes a plurality of edge computing devices, and the edge computing device is determined as a central control device based on election from the plurality of edge computing devices. The computer program code includes first capturing code configured to cause the at least one processor to capture a video stream of an environment and first obtaining code configured to cause the at least one processor to obtain identity information of an object in the video stream by performing object identification on the video stream.

## Claims

### Claim 1

An edge computing device, included in a distributed object identification system comprising a plurality of edge computing devices, wherein the edge computing device is determined as a central control device based on election from the plurality of edge computing devices, the edge computing device comprising:
at least one memory configured to store computer program code; and
at least one processor configured to access said computer program code and operate as instructed by said computer program code, said computer program code comprising:
first capturing code configured to cause the at least one processor to capture a video stream of an environment;
first obtaining code configured to cause the at least one processor to obtain identity information of an object in the video stream by performing object identification on the video stream;
first monitoring code configured to cause the at least one processor to monitor an operating status of each of the plurality of edge computing devices;
first performing code configured to cause the at least one processor to perform a fault response operation on one edge computing device of the plurality of edge computing devices in a fault status based on the one edge computing device in the fault status being monitored;
first encrypting code configured to cause the at least one processor to encrypt the captured video stream based on a key possessed by the edge computing device; and
first determining code configured to cause the at least one processor to determine an interactive device, which acquires keys possessed by the plurality of edge computing devices, from the plurality of edge computing devices based on a set interaction condition and device information of the plurality of edge computing devices, and monitor an operating status of the interactive device.

### Claim 2 (depends on 1)

The edge computing device of claim 1, wherein the computer program code further comprises:
first acquiring code configured to cause the at least one processor to acquire the device information.

### Claim 3 (depends on 2)

The edge computing device of claim 2, wherein the computer program code further comprises:
second performing code configured to cause the at least one processor to perform, based on the interactive device in the fault status being monitored, warning based on the interactive device being in the fault status, and
second determining code configured to cause the at least one processor to determine a new interactive device from the plurality of edge computing devices in a non-fault status based on an interaction condition and the device information to replace the interactive device in the fault status.

### Claim 4 (depends on 1)

The edge computing device of claim 1, wherein types of the plurality of edge computing devices comprise first edge computing devices and second edge computing devices,
wherein the first edge computing devices are configured to be connected to external cameras,
wherein the external cameras are integrated in the second edge computing devices,
wherein the computer program code further comprises:
second acquiring code configured to cause the at least one processor to acquire device information of the first edge computing devices, and
first allocating code configured to cause the at least one processor to allocate, based on a set allocation condition, the device information, and the first edge computing devices to which the external cameras are connectable, the external cameras to one of the first edge computing devices to which the external cameras are connectable.

### Claim 5 (depends on 4)

The edge computing device of claim 4, wherein the computer program code further comprises:
third performing code configured to cause the at least one processor to perform, based on one of the first edge computing devices in the fault status being monitored, warning based on the one of the first edge computing devices being in the fault status;
third determining code configured to cause the at least one processor to determine an external camera allocated to the first edge computing devices in the fault status;
second allocating code configured to cause the at least one processor to allocate the determined external camera to one of the first edge computing devices in a non-fault status based on an allocation condition, the device information, and the first edge computing devices in the non-fault status to which the external camera is connectable; and
fourth performing code configured to cause the at least one processor to perform warning based on one of the second edge computing devices being in the fault status based on the one of the second edge computing devices in the fault status being monitored.

### Claim 6 (depends on 1)

The edge computing device of claim 1, wherein the computer program code further comprises:
second monitoring code configured to cause the at least one processor to monitor an operating status of the central control device;
first transmitting code configured to cause the at least one processor to transmit a verification request to other edge computing devices of the plurality of edge computing devices based on detection that the central control device is in the fault status, the verification request being used for verifying a network connection relationship with the central control device; and
fourth determining code configured to cause the at least one processor to determine a new central control device by performing, based on a proportion of the plurality of edge computing devices failing to be connected to the central control device in the plurality of edge computing devices being greater than a proportion threshold, election with other edge computing devices connectable to the central control device,
wherein the new central control device is configured to perform warning based on the central control device being in the fault status.

### Claim 7 (depends on 1)

The edge computing device of claim 1, wherein the computer program code further comprises:
third acquiring code configured to cause the at least one processor to acquire device information of the plurality of edge computing devices;
first pairing code configured to cause the at least one processor to pair each edge computing device of the plurality of edge computing devices with other edge computing devices of the plurality of edge computing devices based on a set forwarding condition and the device information; and
second transmitting code configured to cause the at least one processor to transmit information to the paired edge computing devices based on information failing to be transmitted to a monitor of the environment, such that the paired edge computing devices transmit the information to the monitor.

### Claim 8 (depends on 1)

The edge computing device of claim 1, wherein the computer program code further comprises:
first extracting code configured to cause the at least one processor to extract an image from the captured video stream,
fifth performing code configured to cause the at least one processor to perform feature extraction on the image to obtain an object feature;
first matching code configured to cause the at least one processor to match the object feature with a plurality of object feature samples in a database;
first using code configured to cause the at least one processor to use, as the identity information of the object in the video stream, information corresponding to the matched plurality of object feature samples; and
third transmitting code configured to cause the at least one processor to transmit the identity information of the object in the video stream to a monitor of the environment.

### Claim 9 (depends on 8)

The edge computing device of claim 8, wherein the computer program code further comprises:
third monitoring code configured to cause the at least one processor to monitor the operating status of the plurality of edge computing devices;
sixth performing code configured to cause the at least one processor to perform first real-time warning according to fault information causing the fault status based on detecting that the operating status of one of the plurality of edge computing devices is the fault status;
fourth transmitting code configured to cause the at least one processor to transmit identity information to an archive server; and
seventh performing code configured to cause the at least one processor to perform second real-time warning according to the identity information based on the identity information being a suspicious identity.

### Claim 10

An edge computing method, performed by an edge computing device, the edge computing device included in a distributed object identification system comprising a plurality of edge computing devices, wherein the edge computing device is determined as a central control device based on election from the plurality of edge computing devices, the method comprising:
capturing a video stream of an environment;
obtaining identity information of an object in the video stream by performing object identification on the video stream;
monitoring an operating status of each of the plurality of edge computing devices;
performing a fault response operation on one edge computing device of the plurality of edge computing devices in a fault status based on the one edge computing device in the fault status being monitored;
encrypting the captured video stream based on a key possessed by the edge computing device;
determining an interactive device, which acquires keys possessed by the plurality of edge computing devices, from the plurality of edge computing devices based on a set interaction condition and device information of the plurality of edge computing device; and
monitoring an operation status of the interactive device.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
acquiring the device informations, which is of the plurality of edge computing devices.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
performing, based on the interactive device in the fault status being monitored, warning based on the interactive device being in the fault status, and
determining a new interactive device from the plurality of edge computing devices in a non-fault status based on an interaction condition and the device information to replace the interactive device in the fault status.

### Claim 13 (depends on 10)

The method of claim 10, wherein types of the plurality of edge computing devices comprise first edge computing devices and second edge computing devices,
wherein the first edge computing devices are configured to be connected to external cameras,
wherein the external cameras are integrated in the second edge computing devices,
wherein the method further comprises:
acquiring device information of the first edge computing devices, and
allocating, based on a set allocation condition, the device information, and the first edge computing devices to which the external cameras are connectable, the external cameras to one of the first edge computing devices to which the external cameras are connectable.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
performing, based on one of the first edge computing devices in the fault status being monitored, warning based on the one of the first edge computing devices being in the fault status;
determining, with the central control device, an external camera allocated to the first edge computing devices in the fault status;
allocating, with the central control device, the determined external camera to one of the first edge computing devices in a non-fault status based on an allocation condition, the device information, and the first edge computing devices in the non-fault status to which the external camera is connectable; and
performing, with the central control device, warning based on one of the second edge computing devices being in the fault status based on the one of the second edge computing devices in the fault status being monitored.

### Claim 15 (depends on 10)

The method of claim 10, further comprising:
monitoring an operating status of the central control device;
transmitting a verification request to other edge computing devices of the plurality of edge computing devices based on detection that the central control device is in the fault status, the verification request being used for verifying a network connection relationship with the central control device; and
determining a new central control device by performing, based on a proportion of the plurality of edge computing devices failing to be connected to the central control device in the plurality of edge computing devices being greater than a proportion threshold, election with other edge computing devices connectable to the central control device,
wherein the new central control device is configured to perform warning based on the central control device being in the fault status.

### Claim 16 (depends on 10)

The method of claim 10, further comprising:
acquiring device information of the plurality of edge computing devices;
pairing each edge computing device of the plurality of edge computing devices with other edge computing devices of the plurality of edge computing devices based on a set forwarding condition and the device information; and
transmitting information to the paired edge computing devices based on information failing to be transmitted to a monitor of the environment, such that the paired edge computing devices transmit the information to the monitor.

### Claim 17 (depends on 10)

The method of claim 10, further comprising:
extracting an image from the captured video stream;
performing feature extraction on the image to obtain an object feature;
matching object feature with a plurality of object feature samples in a database;
using, as the identity information of the object in the video stream, information corresponding to the matched plurality of object feature samples; and
transmitting the identity information of the object in the video stream to a monitor of the environment.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
monitoring the operating status of the plurality of edge computing devices;
performing first real-time warning according to fault information causing the fault status based on detecting that the operating status of one of the plurality of edge computing devices is the fault status;
transmitting the identity information to an archive server; and
performing second real-time warning according to the identity information based on the identity information being a suspicious identity.

### Claim 19

A non-transitory computer-readable storage medium storing computer instructions that, when executed by at least one processor of a device, the device comprising an edge computing device included in a distributed object identification system comprising a plurality of edge computing devices, wherein the edge computing device is determined as a central control device based on election from the plurality of edge computing devices, cause the at least one processor to:
capture a video stream of an environment;
obtain identity information of an object in the video stream by performing object identification on the video stream;
monitor an operating status of each of the plurality of edge computing devices;
perform a fault response operation on one edge computing device of the plurality of edge computing devices in a fault status based on the one edge computing device in the fault status being monitored;
encrypt the captured video stream based on a key possessed by the edge computing device;
determine an interactive device, which acquires keys possessed by the plurality of edge computing devices, from the plurality of edge computing devices based on a set interaction condition and device information of the plurality of edge computing device; and
monitor an operation status of the interactive device.

### Claim 20 (depends on 19)

The storage medium of claim 19, wherein the instructions, when executed, further cause the at least one processor to:
acquire the device information.