System, method, and computer program for in-memory data deduplication
AssigneeAmdocs Development Limited
InventorLucas Dario Majerowicz
Life of this patent
Estimating the term…
Abstract
As described herein, a system, method, and computer program are provided for deduplicating data in memory. A map of key and value pairs for each unique data item of a plurality of unique data items in a dataset is maintained. A duplicate of a unique data item of the plurality of unique data items is detected. The duplicate of the unique data item is replaced with a reference to one of the key and value pairs included in the map for the unique data item.
Claims13
A non-transitory computer-readable media storing computer instructions which when executed by one or more processors of a device cause the device to prevent storage of duplicate data items in a memory to reduce an amount of the memory required to store a dataset, by causing the device to:
- maintain a map of key and value pairs for unique data items existing in the dataset stored in the memory, wherein each of the key and value pairs corresponds with one of the unique data items and includes a key and a value that are identical to one another and that are of a same data type as the unique data item for which the key and value pair corresponds;
- detect a request to write a new data item to the dataset;
- determine that the new data item is a duplicate of one of the unique data items by matching the new data item to one of the key and value pairs included in the map;
- instead of storing the new data item in the dataset, store a reference to the one of the key and value pairs included in the map for the one of the unique data items, wherein storing the reference to the one of the key and value pairs included in the map prevents storage of duplicate data items in the memory to reduce an amount of the memory required to store the dataset.
A method to prevent storage of duplicate data items in a memory to reduce an amount of the memory required to store a dataset, the method comprising:
- at a computer system:
- maintaining a map of key and value pairs for unique data items existing in the dataset stored in the memory, wherein each of the key and value pairs corresponds with one of the unique data items and includes a key and a value that are identical to one another and that are of a same data type as the unique data item for which the key and value pair corresponds;
- detecting a request to write a new data item to the dataset;
- determining that the new data item is a duplicate of one of the unique data items by matching the new data item to one of the key and value pairs included in the map;
- instead of storing the new data item in the dataset, storing a reference to the one of the key and value pairs included in the map for the one of the unique data items, wherein storing the reference to the one of the key and value pairs included in the map prevents storage of duplicate data items in the memory to reduce an amount of the memory required to store the dataset.
A system, comprising:
- a non-transitory memory storing instructions;
- and
- one or more processors in communication with the non-transitory memory that execute the instructions to prevent storage of duplicate data items in a memory to reduce an amount of the memory required to store a dataset, by executing the instructions to:
- maintain a map of key and value pairs for unique data items existing in the dataset stored in the memory, wherein each of the key and value pairs corresponds with one of the unique data items and includes a key and a value that are identical to one another and that are of a same data type as the unique data item for which the key and value pair corresponds;
- detect a request to write a new data item to the dataset;
- determine that the new data item is a duplicate of one of the unique data items by matching the new data item to one of the key and value pairs included in the map;
- instead of storing the new data item in the dataset, store a reference to the one of the key and value pairs included in the map for the one of the unique data items, wherein storing the reference to the one of the key and value pairs included in the map prevents storage of duplicate data items in the memory to reduce an amount of the memory required to store the dataset.
Dependencies are inferred from the claim text.