Selective inline compression using data processing unit on a parallel file system

A method for compressing files, comprising:

• generating, at a file system, a file heatmap, wherein the file heatmap is based on read/write cycles of stored data files stored in a remote storage system;
• receiving, at a first file system client of a host, a request to write a first data file of data files to the remote storage system;
• receiving, at a file system, a request for the file heatmap from the first file system client;
• receiving, based on the request, the file heatmap at the host from a metadata server of the file system, wherein the file heatmap comprises a forecasted compressibility setting for each file of the data files;
• making a first determination, while executing a data transfer operation of the data files to the remote storage system, that the first data file is suitable for compression based on the file heatmap;
• compressing, based on the first determination, the first data file using a compression accelerator on a data processing unit (DPU) of the host to obtain a first compressed data file;
• sending the first compressed data file to the remote storage system;
• in response to sending the first compressed data file:
• sending metadata on updated data files from the first file system client, wherein the metadata includes information that the first data file is compressed by the first file system client, and wherein the updated data files are associated with the stored data files;
• committing, at the file system, a first layout mapping of the first data file to the metadata server;
• and
• updating the file heatmap based on the first layout mapping of the first data file.