# US 11,442,930 B2 · Method, apparatus, device and storage medium for data aggregation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,442,930 B2
- Document ID: US11442930B2
- Status: Granted
- Kind: B2
- Application number: 17/021,001
- Filed: 2020-09-15
- Published: 2021-06-24
- Granted: 2022-09-13
- Earliest claimed: 2020-01-20
- Assignees: Beijing Baidu Netcom Science Technology Co., Ltd.
- Applicants: Beijing Baidu Netcom Science Technology Co., Ltd.
- Inventors: Yanyan Li; Jianguo Duan; Hui Xiong
- Web page: https://patentgrep.com/patent/US11442930B2

## CPC

- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06F 16/215: Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
- G06F 16/24556: {Aggregation; Duplicate elimination}
- G06Q 30/0201: Market modelling; Market analysis; Collecting market data
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

The present application discloses a method for data aggregation, the method includes: acquiring original data to be aggregated and dividing the original data into at least one first data set; determining whether each of the at least one first data set has a corresponding historical aggregation record; when there is at least one second data set with a historical aggregation record in the at least one first data set, acquiring a historical aggregation result corresponding to each second data set to obtain at least one first aggregation result; performing aggregation on each third data set without a historical aggregation record to obtain at least one second aggregation result; and determining a third aggregation result of the original data according to the at least one first aggregation result and the at least one second aggregation result, and determining a data tag of the original data according to the third aggregation result.

## Claims

### Claim 1

A method for data aggregation, comprising:
acquiring original data to be aggregated and dividing the original data into at least one first data set;
determining whether each of the at least one first data set has a corresponding historical aggregation record;
when there is at least one second data set with a historical aggregation record in the at least one first data set, acquiring a historical aggregation result corresponding to each second data set to obtain at least one first aggregation result;
performing aggregation on each third data set without a historical aggregation record to obtain at least one second aggregation result; and
determining a third aggregation result of the original data according to the at least one first aggregation result and the at least one second aggregation result, and determining a data tag of the original data according to the third aggregation result;
wherein the dividing the original data into at least one first data set comprises:
establishing a first plan tree corresponding to the original data, the first plan tree comprising at least one first connection node and at least one first aggregation node, one leaf node of the first plan tree constituting one first aggregation node, and data represented by one first aggregation node constituting one first data set;
wherein the determining whether each of the at least one first data set has a corresponding historical aggregation record comprises:
determining whether a data attribute index of an aggregation record file is completely consistent with that of each of the at least one first aggregation node in a preset file path, wherein the data attribute index comprising a time attribute, a location attribute, a volume attribute, and a format attribute;
determining that the first aggregation node has a corresponding historical aggregation record when the data attribute index of the aggregation record file is completely consistent with that of the first aggregation node, to obtain at least one second aggregation node with a historical aggregation record in the first plan tree, data represented by one second aggregation node constituting one second data set; and
determining that the first aggregation node does not have a corresponding historical aggregation record when the data attribute index of the aggregation record file is not completely consistent with that of each of the at least one first aggregation node.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the establishing a first plan tree corresponding to the original data comprises:
acquiring a user-specified data metric for each aggregation in batch aggregation on the original data to constitute a data metric threshold for each of the at least one first data set;
determining a number of node levels of the first plan tree and the at least one first aggregation node according to the data metric threshold for each first data set; and
establishing the first plan tree according to the number of node levels and the at least one first aggregation node.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the performing aggregation on each third data set without a historical aggregation record to obtain at least one second aggregation result comprises:
performing trimming on the first plan tree according to the at least one second aggregation node of the first plan tree to obtain a second plan tree; and
performing aggregation on each of the at least one third data set according to the second plan tree to obtain at least one second aggregation result.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the performing trimming on the first plan tree according to the at least one second aggregation node of the first plan tree to obtain a second plan tree comprises:
according to a file record corresponding to each second aggregation node and the at least one first connection node of the first plan tree, determining, in the file record, a highest-level first connection node corresponding to each second aggregation node to obtain at least one second connection node corresponding to the at least one second aggregation node; and
deleting, in the first plan tree, a child node branch corresponding to each of the at least one second connection node to obtain the second plan tree.

### Claim 5

An apparatus for data aggregation, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor;
wherein the memory is stored with instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:
acquire original data to be aggregated and divide the original data into at least one first data set;
determine whether each of the at least one first data set has a corresponding historical aggregation record;
when there is at least one second data set with a historical aggregation record in the at least one first data set, acquire a historical aggregation result corresponding to each second data set to obtain at least one first aggregation result;
perform aggregation on each third data set without a historical aggregation record to obtain at least one second aggregation result; and
determine a third aggregation result of the original data according to the at least one first aggregation result and the at least one second aggregation result, and determine a data tag of the original data according to the third aggregation result;
wherein the at least one processor is further configured to:
establish a first plan tree corresponding to the original data, the first plan tree comprising at least one first connection node and at least one first aggregation node, one leaf node of the first plan tree constituting one first aggregation node, and data represented by one first aggregation node constituting one first data set;
wherein the at least one processor is further configured to:
determine whether a data attribute index of an aggregation record file is completely consistent with that of each of the at least one first aggregation node in a preset file path, wherein the data attribute index comprising a time attribute, a location attribute, a volume attribute, and a format attribute;
determine that the first aggregation node has a corresponding historical aggregation record when the data attribute index of the aggregation record file is completely consistent with that of each of the at least one first aggregation node, to obtain at least one second aggregation node with a historical aggregation record in the first plan tree, data represented by one second aggregation node constituting one second data set; and
determine that the first aggregation node does not have a corresponding historical aggregation record when the data attribute index of the aggregation record file is not completely consistent with that of each of the at least one first aggregation node.

### Claim 6 (depends on 5)

The apparatus according to claim 5, wherein the at least one processor is further configured to:
acquire a user-specified data metric for each aggregation in batch aggregation on the original data to constitute a data metric threshold for each of the at least one first data set;
determine a number of node levels of the first plan tree and the at least one first aggregation node according to the data metric threshold for each first data set; and
establish the first plan tree according to the number of node levels and the at least one first aggregation node.

### Claim 7 (depends on 5)

The apparatus according to claim 5, wherein the at least one processor is further configured to:
perform trimming on the first plan tree according to the at least one second aggregation node of the first plan tree to obtain a second plan tree; and
perform aggregation on each of the at least one third data set according to the second plan tree to obtain at least one second aggregation result.

### Claim 8 (depends on 7)

The apparatus according to claim 7, wherein the at least one processor is further configured to:
according to a file record corresponding to each second aggregation node and the at least one first connection node of the first plan tree, determine, in the file record, a highest-level first connection node corresponding to each second aggregation node to obtain at least one second connection node corresponding to the at least one second aggregation node; and
delete, in the first plan tree, a child node branch corresponding to each of the at least one second connection node to obtain the second plan tree.

### Claim 9

A non-transitory computer readable storage medium stored with computer instructions, wherein the computer instructions are configured to enable a computer to execute the following steps:
acquiring original data to be aggregated and dividing the original data into at least one first data set;
determining whether each of the at least one first data set has a corresponding historical aggregation record;
when there is at least one second data set with a historical aggregation record in the at least one first data set, acquiring a historical aggregation result corresponding to each second data set to obtain at least one first aggregation result;
performing aggregation on each third data set without a historical aggregation record to obtain at least one second aggregation result; and
determining a third aggregation result of the original data according to the at least one first aggregation result and the at least one second aggregation result, and determining a data tag of the original data according to the third aggregation result;
wherein the dividing the original data into at least one first data set comprises:
establishing a first plan tree corresponding to the original data, the first plan tree comprising at least one first connection node and at least one first aggregation node, one leaf node of the first plan tree constituting one first aggregation node, and data represented by one first aggregation node constituting one first data set;
wherein the determining whether each of the at least one first data set has a corresponding historical aggregation record comprises:
determining whether a data attribute index of an aggregation record file is completely consistent with that of each of the at least one first aggregation node in a preset file path, wherein the data attribute index comprising a time attribute, a location attribute, a volume attribute, and a format attribute;
determining that the first aggregation node has a corresponding historical aggregation record when the data attribute index of the aggregation record file is completely consistent with that of the first aggregation node, to obtain at least one second aggregation node with a historical aggregation record in the first plan tree, data represented by one second aggregation node constituting one second data set; and
determining that the first aggregation node does not have a corresponding historical aggregation record when the data attribute index of the aggregation record file is not completely consistent with that of each of the at least one first aggregation node.

### Claim 10 (depends on 9)

The non-transitory computer readable storage medium according to claim 9, wherein the computer instructions are further configured to enable the computer to execute the following steps:
acquiring a user-specified data metric for each aggregation in batch aggregation on the original data to constitute a data metric threshold for each of the at least one first data set;
determining a number of node levels of the first plan tree and the at least one first aggregation node according to the data metric threshold for each first data set; and
establishing the first plan tree according to the number of node levels and the at least one first aggregation node.

### Claim 11 (depends on 9)

The non-transitory computer readable storage medium according to claim 9, wherein the computer instructions are further configured to enable the computer to execute the following steps:
performing trimming on the first plan tree according to the at least one second aggregation node of the first plan tree to obtain a second plan tree; and
performing aggregation on each of the at least one third data set according to the second plan tree to obtain at least one second aggregation result.

### Claim 12 (depends on 11)

The non-transitory computer readable storage medium according to claim 11, wherein the computer instructions are further configured to enable the computer to execute the following steps:
according to a file record corresponding to each second aggregation node and the at least one first connection node of the first plan tree, determining, in the file record, a highest-level first connection node corresponding to each second aggregation node to obtain at least one second connection node corresponding to the at least one second aggregation node; and
deleting, in the first plan tree, a child node branch corresponding to each of the at least one second connection node to obtain the second plan tree.