Metadata management method, system and medium

A metadata management method executed by a metadata management system comprising a computer system, wherein the computer system comprises a processor and a non-transitory storage medium, wherein implementation steps for updating metadata in the metadata management method comprises:

• 1) organizing, by the processor, all directories and files in a file system of the non-transitory storage medium in a tree structure, and saving, by the processor, all directories and subfiles under a single directory as a whole on a continuous address space of an underlying non-transitory storage medium in an order of creation in sequence;
• and
• 2) updating, by the processor, all subdirectories/subfiles under the single directory in a form of a log when updating the metadata of the file system, and appending, by the processor, all metadata updates to a tail of the continuous address space corresponding to the single directory in an order of occurrence so that each metadata operation triggers only one IO request to the non-transitory storage medium,
• wherein implementation steps for metadata retrieval comprise:
• S1) organizing, by the processor, all the directories and the files in the file system of the non-transitory storage medium in the tree structure, and saving, by the processor, all the directories and the subfiles under the single directory as the whole in the continuous address space of the underlying non-transitory storage medium in the order of creation in sequence;
• and
• S2) when the processor performs file system metadata retrieval, the processor parses a directory path of the non-transitory storage medium layer by layer from a first component to a last component in sequence and finally obtains a directory or a file to be accessed in the non-transitory storage medium;
• and when the processor parses a subdirectory under a parent directory, the processor reads all data on the continuous address space of the non-transitory storage medium corresponding to the parent directory, and then searches a target subdirectory or a subfile on read data of the non-transitory storage medium until a match is successful,
• wherein the metadata retrieval is optimized according to a principle of locality,
• wherein parsing the subdirectory under the parent directory comprises following detailed steps:
• S2.1) the processor receives a metadata retrieval request from an application, and the metadata retrieval request being to retrieve an existence of the target directory or file from the parent directory;
• S2.2) the processor obtains related information of all the subdirectories/subfiles under the parent directory into the non-transitory storage medium through one read operation, and going to step S2.3);
• S2.3) the processor determines a size of the parent directory to be parsed, and the processor executes step S2.4) if the total number of the subdirectories/subfiles in the parent directory is less than a preset first threshold TH1;
• or the processor executes step S2.5) if the total number of the subdirectories/subfiles in the parent directory is between the preset first threshold TH1 and a preset second threshold TH2;
• or the processor executes step S2.6) if the total number of the subdirectories/subfiles in the parent directory is greater than the preset second threshold TH2;
• S2.4) the processor uses multi-threading technology to search the target subdirectory or subfile in the parent directory, and going to step S2.7);
• S2.5) the processor uses a vector instruction to search the target subdirectory or subfile in the parent directory, and going to step S2.7);
• S2.6) the processor uses many-core acceleration technology to search the target subdirectory or subfile in the parent directory, and going to step S2.7);
• and
• S2.7) the processor succeeds in the retrieval if the target subdirectory or subfile is found in the parent directory of the non-transitory storage medium;
• or the processor fails in the retrieval, and returns a retrieval result to an upper application,
• wherein a linear search process is performed when searching the target subdirectory or the subfile in the parent directory because no index is established for all the subdirectories/subfiles under the same parent directory to save IO operation, wherein the multi-threading technology, the vector instruction and the many-core acceleration technology shorten a computation time of the linear search process.