Avoiding client timeouts in a distributed filesystem

A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for reducing client timeouts for a distributed filesystem, the method comprising:

• collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises:
• storing the data for the distributed filesystem in a cloud storage system, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage system, wherein each cloud controller includes a local storage space;
• and
• caching data for the distributed filesystem in the local storage space of each cloud controller, wherein due to space constraints each cloud controller cannot cache the entirety of the distributed filesystem in its local storage space, wherein the cloud controllers are configured to mask the latency of accessing the cloud storage system to provide clients with an abstraction that all of the data in the distributed filesystem is available locally;
• receiving at a cloud controller a request from a client system for a target file in a set of less-frequently accessed files in the distributed filesystem, wherein the cloud controller caches one or more initial blocks for each file in the set to ensure that at least one block of data can be returned to a client that requests to access a file in the set without incurring the latency of contacting the cloud storage system;
• determining that a first set of data blocks for the target file are currently cached in the local storage space of the cloud controller and that subsequent data blocks for the target file are not yet being cached by the cloud controller and are only available via the cloud storage system;
• requesting the subsequent data blocks from the cloud storage system, wherein a local network connection between the client system and the cloud controller is less resource-constrained than a wide-area network connection between the cloud controller and the cloud storage system, wherein requesting a subsequent data block comprises estimating the time needed for the cloud storage system to receive the request and return the subsequent data block to the cloud controller, wherein the client is configured with a timeout interval and will abort the file request if any requested data block is not received within the timeout interval;
• determining that the cached first block can be sent to the client within the timeout interval but that requesting a subsequent requested data block in the target file from the cloud storage system will exceed that subsequent request's associated client timeout interval;
• and
• delaying transmitting one or more cached data blocks for the target file that are already available at the cloud controller to the client system to ensure that the subsequent requested data blocks requested from the cloud storage system are received by the cloud controller from the cloud storage system and can be sent to the client system before their respective timeout intervals on the client system are exceeded, wherein the spacing of the transmission of the cached data blocks is determined based on network characteristics for the wide-area network connection, wherein the cloud controller delays the transmission of the cached data block but ensures that this delay does not result in exceeding the client timeout interval associated with the request for the cached data block.

A cloud controller that reduces client timeouts for a distributed filesystem, comprising:

• a processor;
• a storage mechanism that stores metadata for the distributed filesystem;
• and
• a storage management mechanism that includes a local storage space that caches distributed filesystem data, wherein due to space constraints each cloud controller cannot cache the entirety of the distributed filesystem in its local storage space, wherein the cloud controllers are configured to mask the latency of accessing the cloud storage system to provide clients with an abstraction that all of the data in the distributed filesystem is available locally;
• wherein two or more cloud controllers collectively manage the data of the distributed filesystem, wherein the cloud controllers cache and ensure data consistency for data stored in a cloud storage system;
• wherein the cloud controller is configured to:
• receive a request from a client system for a target file in a set of less-frequently accessed files in the distributed filesystem, wherein the cloud controller caches one or more initial blocks for each file in the set to ensure that at least one block of data can be returned to a client that requests to access a file in the set without incurring the latency of contacting the cloud storage system;
• determine that a first set of data blocks for the target file are currently cached in the local storage space of the cloud controller and that subsequent data blocks for the target file are not yet being cached by a cloud controller and are only available via the cloud storage system;
• request the subsequent data blocks from the cloud storage system, wherein a local network connection between the client system and the cloud controller is less resource-constrained than a wide-area network connection between the cloud controller and the cloud storage system, wherein requesting a subsequent data block comprises estimating the time needed for the cloud storage system to receive the request and return the subsequent data block to the cloud controller, wherein the client is configured with a timeout interval and will abort the file request if any requested data block is not received within the timeout interval;
• determine that the cached first block can be sent to the client within the timeout interval but that requesting a subsequent requested data block in the target file from the cloud storage system will exceed that subsequent request's associated client timeout interval;
• and
• delay transmitting one or more cached blocks for the target file that are already available at the cloud controller to the client system to ensure that the subsequent requested data blocks are received by the cloud controller from the cloud storage system and can be sent to the client system before their respective timeout intervals on the client system are exceeded, wherein the spacing of the transmission of the cached data blocks is determined based on network characteristics for the wide-area network connection, wherein the cloud controller delays the transmission of the cached data block but ensures that this delay does not result in exceeding the client timeout interval associated with the request for the cached data block.