# US 11,907,188 B2 · Method, device, and program product for managing data pattern

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,907,188 B2
- Document ID: US11907188B2
- Status: Granted
- Kind: B2
- Application number: 17/239,950
- Filed: 2021-04-26
- Published: 2022-07-21
- Granted: 2024-02-20
- Earliest claimed: 2021-01-20
- Assignees: EMC IP Holding Company LLC
- Applicants: EMC IP Holding Company LLC
- Inventors: Weilan Pu; Jian Kang; Chi Chen; Wen Chen
- Web page: https://patentgrep.com/patent/US11907188B2

## CPC

- G06F 16/215: Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
- G06F 16/285: {Clustering or classification}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Techniques for managing data patterns involve: acquiring multiple sets of data patterns respectively associated with multiple collection devices, wherein a set of data patterns in the multiple sets of data patterns represent patterns of duplicate data in data from one of the multiple collection devices; dividing the multiple collection devices into multiple groups based on clusters of the multiple sets of data patterns; and determining, based on sets of data patterns associated with collection devices in a group in the multiple groups, a set of shared data patterns for sharing among the collection devices in the group. Accordingly, data patterns that can be shared among multiple collection devices can be determined in a more accurate and effective manner, thereby facilitating the removal of duplicate data from the multiple collection devices.

## Claims

### Claim 1

A method for managing data patterns, including:
acquiring multiple sets of data patterns respectively associated with multiple collection devices, wherein the multiple collection devices are located in an edge network in an application environment, and wherein a set of data patterns in the multiple sets of data patterns represent patterns of duplicate data in data from one of the multiple collection devices;
generating, based on the multiple sets of data patterns, multiple pattern features, wherein each one of the pattern features is generated for a respective set of data patterns in the multiple sets of data patterns, and wherein each pattern feature includes a number of occurrences of each individual data pattern in the respective set of data patterns;
dividing the multiple collection devices into multiple groups based on the pattern features;
determining, based on the numbers of occurrences of data patterns included in sets of data patterns associated with collection devices in a group in the multiple groups, a set of shared data patterns for sharing among the collection devices in the group;
distributing the set of shared data patterns to an edge computing device in the edge network, wherein the edge computing device is connected to a target collection device in the multiple collection devices in the group;
instructing the edge computing device to generate de-duplicated data of target data from the target collection device based on the set of shared data patterns, wherein the de-duplicated data is smaller than the target data;
instructing the edge computing device to transmit the de-duplicated data to a server device that is used to process the target data; and
whereby transmission of the de-duplicated data to the server device reduces overhead of storage resources involved in data storage by the server device.

### Claim 2 (depends on 1)

The method according to claim 1, wherein dividing the multiple collection devices into the multiple groups includes:
converting respectively the multiple pattern features to multiple low-dimensional features, wherein dimensions of the multiple low-dimensional features are lower than those of the multiple pattern features; and
determining the multiple groups based on clusters of the multiple low-dimensional features.

### Claim 3 (depends on 1)

The method according to claim 1, wherein determining the set of shared data patterns further includes: determining the set of shared data patterns based on an intersection of the sets of data patterns.

### Claim 4 (depends on 1)

The method according to claim 1, wherein acquiring the multiple sets of data patterns includes:
acquiring a set of initial data patterns associated with the collection device;
ranking the set of initial data patterns based on the numbers of occurrences of the set of initial data patterns in the data; and
selecting the set of data patterns associated with the collection device based on the ranked set of initial data patterns.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the server device is located in a core network separate from the edge network; and
whereby transmission of the de-duplicated data to the server device further reduces bandwidth requirements between the edge network and the core network.

### Claim 6

An electronic device, including:
at least one processor;
a volatile memory; and
a memory coupled to the at least one processor, wherein the memory has instructions stored therein that, when executed by the at least one processor, cause the device to perform a method for managing data patterns, the method including:
acquiring multiple sets of data patterns respectively associated with multiple collection devices, wherein the multiple collection devices are located in an edge network in an application environment, and wherein a set of data patterns in the multiple sets of data patterns represent patterns of duplicate data in data from one of the multiple collection devices;
generating, based on the multiple sets of data patterns, multiple pattern features, wherein each one of the pattern features is generated for a respective set of data patterns in the multiple sets of data patterns, and wherein each pattern feature includes a number of occurrences of each individual data pattern in the respective set of data patterns;
dividing the multiple collection devices into multiple groups based on the pattern features;
determining, based on the numbers of occurrences of data patterns included in sets of data patterns associated with collection devices in a group in the multiple groups, a set of shared data patterns for sharing among the collection devices in the group;
distributing the set of shared data patterns to an edge computing device in the edge network, wherein the edge computing device is connected to a target collection device in the multiple collection devices in the group;
instructing the edge computing device to generate de-duplicated data of target data from the target collection device based on the set of shared data patterns, wherein the de-duplicated data is smaller than the target data;
instructing the edge computing device to transmit the de-duplicated data to a server device that is used to process the target data; and
whereby transmission of the de-duplicated data to the server device reduces overhead of storage resources involved in data storage by the server device.

### Claim 7 (depends on 6)

The device according to claim 6, wherein dividing the multiple collection devices into the multiple groups includes:
converting respectively the multiple pattern features to multiple low-dimensional features, wherein dimensions of the multiple low-dimensional features are lower than those of the multiple pattern features; and
determining the multiple groups based on clusters of the multiple low-dimensional features.

### Claim 8 (depends on 6)

The device according to claim 6, wherein determining the set of shared data patterns further includes: determining the set of shared data patterns based on an intersection of the sets of data patterns.

### Claim 9 (depends on 6)

The device according to claim 6, wherein acquiring the multiple sets of data patterns includes:
acquiring a set of initial data patterns associated with the collection device;
ranking the set of initial data patterns based on the numbers of occurrences of the set of initial data patterns in the data; and
selecting the set of data patterns associated with the collection device based on the ranked set of initial data patterns.

### Claim 10 (depends on 6)

The electronic device according to claim 6, wherein the server device is located in a core network separate from the edge network; and
whereby transmission of the de-duplicated data to the server device further reduces bandwidth requirements between the edge network and the core network.

### Claim 11

A computer program product having a non-transitory computer readable medium which stores a set of instructions to manage data patterns; the set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of:
acquiring multiple sets of data patterns respectively associated with multiple collection devices, wherein the multiple collection devices are located in an edge network in an application environment, and wherein a set of data patterns in the multiple sets of data patterns represent patterns of duplicate data in data from one of the multiple collection devices;
generating, based on the multiple sets of data patterns, multiple pattern features, wherein each one of the pattern features is generated for a respective set of data patterns in the multiple sets of data patterns, and wherein each pattern feature includes a number of occurrences of each individual data pattern in the respective set of data patterns;
dividing the multiple collection devices into multiple groups based on the pattern features;
determining, based on the numbers of occurrences of data patterns included in sets of data patterns associated with collection devices in a group in the multiple groups, a set of shared data patterns for sharing among the collection devices in the group;
distributing the set of shared data patterns to an edge computing device in the edge network, wherein the edge computing device is connected to a target collection device in the multiple collection devices in the group;
instructing the edge computing device to generate de-duplicated data of target data from the target collection device based on the set of shared data patterns, wherein the de-duplicated data is smaller than the target data;
instructing the edge computing device to transmit the de-duplicated data to a server device that is used to process the target data; and
whereby transmission of the de-duplicated data to the server device reduces overhead of storage resources involved in data storage by the server device.