Pattern-aware prefetching using parallel log-structured file system

A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform the following steps in response to at least one read request for at least a first portion of a shared file:

• detecting, using at least one processing device of a pattern-aware prefetching file system, at least one pattern in at least two consecutive non-sequential accesses of said shared file, wherein said at least one pattern is detected by analyzing distances between consecutive logical offsets in said at least one read request for said shared file;
• and
• prefetching at least a second portion of said shared file from at least one storage device, wherein said second portion is predicted to be accessed based at least in part on said detected at least one pattern,
• wherein said shared file is created by a plurality of processes, wherein each of said plurality of processes has a unique physical logfile for storing data written by a corresponding process and a unique physical index file for storing metadata for the corresponding process, wherein, for a given write operation of a data chunk to the shared file by a given one of said plurality of processes, (i) said data chunk is stored in said unique physical logfile corresponding to said given process, and (ii) an index entry is created for said unique physical index file corresponding to said given process, that maps said data chunk within a logical representation of said shared file to a location within said unique physical logfile, wherein a plurality of said index entries is stored in said unique physical index file corresponding to said given process in a compressed format using said detected at least one pattern and wherein said prefetching of said at least second portion of said shared file is performed using at least one of said unique physical index files, comprising said compressed format, for said plurality of processes.

A pattern-aware prefetching file system, comprising:

• a pattern detector of a pattern-aware prefetching file system for detecting, in response to at least one read request for at least a first portion of a shared file, using at least one processing device of said pattern-aware prefetching file system, at least one pattern in at least two consecutive non-sequential accesses of said shared file, wherein said at least one pattern is detected by analyzing distances between consecutive logical offsets in said at least one read request for said shared file;
• and
• a prefetch manager for prefetching, in response to said at least one read request for said at least first portion of said shared file, at least a second portion of said shared file from at least one storage device, wherein said second portion is predicted to be accessed based at least in part on said detected at least one pattern;
• and
• a prefetch cache for storing at least said portion of said shared file,
• wherein said shared file is created by a plurality of processes, wherein each of said plurality of processes has a unique physical logfile for storing data written by a corresponding process and a unique physical index file for storing metadata for the corresponding process, wherein, for a given write operation of a data chunk to the shared file by a given one of said plurality of processes, (i) said data chunk is stored in said unique physical logfile corresponding to said given process, and (ii) an index entry is created for said unique physical index file corresponding to said given process, that maps said data chunk within a logical representation of said shared file to a location within said unique physical logfile, wherein a plurality of said index entries is stored in said unique physical index file corresponding to said given process in a compressed format using said detected at least one pattern and wherein said prefetching of said at least second portion of said shared file is performed using at least one of said unique physical index files, comprising said compressed format, for said plurality of processes.