Data storage method, and apparatus enabling selective and efficient data reading

A data reading method, wherein the method comprises:

• obtaining a plurality of metadata blocks from a second storage area, wherein the plurality of metadata blocks correspond to a plurality of data blocks, each data block of the plurality of data blocks comprises a part of processed event data, and each metadata block of the plurality of metadata blocks comprises a start time, an end time, a quantity, and a compression manner of event data in a data block corresponding to the metadata block, and a storage address of the data block;
• finding, in the plurality of metadata blocks, at least one metadata block that comprises a start time and an end time between which any time point is in a target time interval;
• and
• obtaining, from a first storage area based on a storage address in the at least one found metadata block, at least one data block corresponding to the at least one found metadata block, and processing the data block based on a compression manner in a found metadata block corresponding to each data block, to obtain to-be-read event data,
• wherein the processing the data block based on the compression manner in the found metadata block corresponding to each data block, to obtain to-be-read event data comprises:
• performing decompression processing on the data block based on the compression manner in the found metadata block corresponding to each data block, to obtain a plurality of decompressed sequences, wherein the plurality of sequences comprise time information, space information, and event content information of event data in an event chunk corresponding to the data block, a quantity of elements in each sequence is equal to a quantity of pieces of event data in the event chunk, an element value in each sequence is an integer greater than or equal to 0, and a quantity of elements having element values greater than a first threshold value is greater than a quantity of elements having element values less than or equal to the first threshold value;
• processing the plurality of decompressed sequences obtained through decompression processing performed on each data block, to obtain the event chunk corresponding to each data block;
• and
• using, as the read event data, event data that is in event data comprised in all event chunks and whose time information is in the target time interval.

A data storage apparatus, comprising:

• a processor;
• and
• a memory, configured to store instructions executable by the processor, wherein
• execution of the instructions by the processor causes the processor to:
• obtain event data;
• process the event data, to obtain a plurality of data blocks and a plurality of corresponding metadata blocks, wherein each data block of the plurality of data blocks comprises a part of processed event data, and each metadata block of the plurality of metadata blocks comprises a start time, an end time, a quantity, and a compression manner of event data in a data block corresponding to the metadata block, and a storage address of the data block;
• and
• store the plurality of data blocks into a first storage area, and store the plurality of metadata blocks into a second storage area,
• wherein execution of the instructions to process the event data, to obtain the plurality of data blocks and the plurality of corresponding metadata blocks causes the processor to:
• perform chunking on the event data based on time information and/or space information in the event data, to obtain a plurality of event chunks;
• process each event chunk of the plurality of event chunks, to obtain a plurality of sequences, wherein the plurality of sequences comprise time information, space information, and event content information of event data in the event chunk, a quantity of elements in each sequence is equal to a quantity of pieces of event data in the event chunk, an element value in each sequence of the plurality of sequences is an integer greater than or equal to 0, and a quantity of elements having element values greater than a first threshold value is greater than a quantity of elements having element values less than or equal to the first threshold value;
• for each sequence, of the plurality of sequences select, from a preset compression manner, a compression manner matching the sequence, and compress the sequence based on the selected compression manner, to obtain compressed data;
• obtain one data block based on a plurality of pieces of compressed data obtained based on the plurality of sequences corresponding to each event chunk;
• and
• obtain, based on time information in event data comprised in each event chunk, a quantity of pieces of event data, a selected compression manner, and a storage address of the data block, to obtain one metadata block corresponding to the data block.

A data reading apparatus, comprising:

• a processor;
• and
• a memory, configured to store instructions executable by the processor, wherein
• execution of the instructions by the processor causes the processor to:
• obtain a plurality of metadata blocks from a second storage area, wherein the plurality of metadata blocks correspond to a plurality of data blocks, each data block of the plurality of data blocks comprises a part of processed event data, and each metadata block of the plurality of metadata blocks comprises a start time, an end time, a quantity, and a compression manner of event data in a data block corresponding to the metadata block, and a storage address of the data block;
• find, in the plurality of metadata blocks, at least one metadata block that comprises a start time and an end time between which any time point is in a target time interval;
• and
• obtain, from a first storage area based on a storage address in the at least one found metadata block, at least one data block corresponding to the at least one found metadata block, and process the data block based on a compression manner in a found metadata block corresponding to each data block, to obtain to-be-read event data,
• wherein execution of the instructions to process the data block based on the compression manner in the found metadata block corresponding to each data block, to obtain to-be-read event data causes the processor to:
• perform decompression processing on the data block based on the compression manner in the found metadata block corresponding to each data block, to obtain a plurality of decompressed sequences, wherein the plurality of sequences comprise time information, space information, and event content information of event data in an event chunk corresponding to the data block, a quantity of elements in each sequence is equal to a quantity of pieces of event data in the event chunk, an element value in each sequence is an integer greater than or equal to 0, and a quantity of elements having element values greater than a first threshold value is greater than a quantity of elements having element values less than or equal to the first threshold value;
• process the plurality of decompressed sequences obtained through decompression processing performed on each data block, to obtain the event chunk corresponding to each data block;
• and
• use, as the read event data, event data that is in event data comprised in all event chunks and whose time information is in the target time interval.