Distributed Storage Method and Device
AssigneeHuawei Technologies Co., Ltd.
InventorsYong Zhang, Yicong Cai, Ajith Raj Shetty S
Life of this patent
Estimating the term…
Abstract
A distributed storage method includes offline merging, by a first thread service of a distributed storage system, M small files in a file system; generating, by the first thread service, M pieces of metadata in the offline merging process; loading, by a second thread service of the distributed storage system, the M pieces of metadata into a metadata set; searching, by the second thread service, the metadata set for metadata of a first small file when the second thread service receives a first instruction; and performing, by the second thread service, the operation corresponding to the first instruction when the second thread service finds the metadata of the first small file in the metadata set.
Claims20
A method for small files optimization, comprising:
- merging, by a first thread service, M small files in a file system into one first file, wherein a size of each of the M small files is less than a predefined byte size, and wherein M is an integer that is greater than 1;
- generating, by the first thread service, M pieces of metadata, wherein each of the M pieces of metadata indicates a location of a corresponding one of the M small files in the first file;
- receiving, by a second thread service, a first instruction that instructs the second thread service to perform an operation on a first small file, wherein the first small file is one of the M small files, and wherein the first thread service and the second thread service are two different and separate thread services;
- obtaining, by the second thread service, metadata of the first small file from a metadata set that comprises metadata describing a plurality of small files including the M small files;
- and
- performing, by the second thread service, the operation on the first small file.
A server, comprising:
- a memory configured to store instructions;
- and
- one or more processors coupled to the memory and configured to execute the instructions to implement a first thread service and a second thread service and to cause the server to be configured to:
- merge, by the first thread service, M small files in a file system into one first file, wherein a size of each of the M small files is less than a predefined byte size, and wherein M is an integer that is greater than 1;
- generate, by the first thread service, M pieces of metadata, wherein each of the M pieces of metadata indicates a location of a corresponding one of the M small files in the first file;
- receive, by the second thread service, a first instruction that instructs the second thread service to perform an operation on a first small file, wherein the first small file is one of the M small files, and wherein the first thread service and the second thread service are two different and separate thread services;
- obtain, by the second thread service, metadata of a first small file from a metadata set that comprises metadata describing a plurality of small files including the M small files;
- and
- perform, by the second thread service, the operation on the first small file.
A computer program product comprising instructions that are stored on a computer-readable medium and that, when executed by a processor, cause a server to:
- merge, by a first thread service of the server, M small files in a file system into one first file, wherein a size of each of the M small files is less than a predefined byte size, and wherein M is an integer that is greater than 1;
- generate, by the first thread service, M pieces of metadata, wherein each of the M pieces of metadata indicates a location of a corresponding one of the M small files in the first file;
- receive, by a second thread service of the server, a first instruction that instructs the second thread service to perform an operation on a first small file, wherein the first small file is one of the M small files, and wherein the first thread service and the second thread service are two different and separate thread services;
- obtain, by the second thread service, metadata of the first small file from a metadata set that comprises metadata describing a plurality of small files including the M small files;
- and
- performing, by the second thread service, the operation on the first small file based on the metadata of the first small file.
Dependencies are inferred from the claim text.