Method and apparatus to improve buffer cache hit rate

A method of operating a network caching device which includes a main memory and a secondary storage facility, the method comprising:

• receiving a request for a first object from a client over a network;
• determining whether the first object is cached by the network caching device;
• if the first object is cached by the network caching device, then in response to the first request,
• identifying a first metafile entry as being associated with the first object, the first metafile entry containing metadata for a plurality of related objects, including the first object, stored in a single file in the secondary storage facility,
• using the metadata in the first metafile entry to locate and read the plurality of related objects, including the first object, from the secondary storage facility into the main memory, and
• providing the first object to the client;
• receiving a request for a second object from a client over the network;
• determining whether the second object is cached by the network caching device;
• and
• if the second object is not cached by the network caching device, then in response to the request for the second object,
• retrieving the second object from an origin server,
• providing the second object to the requesting client in response to the second request,
• determining whether the second object is related to the first object, wherein determining whether the second object is related to the first object comprises
• determining whether a source of the request for the second object matches a source of the request for the first object;
• and
• determining whether a predetermined length of time has elapsed from the request for the first object to the request for the second object, and
• if the second object is determined to be related to the first object and a predetermined size condition is met, then
• writing the second object to said single file in the secondary storage facility, and
• writing metadata for the second object into the first metafile entry.

A method of operating a network caching device which includes random access memory (RAM) and a mass storage facility, the method comprising:

• receiving a request for a first object over a network;
• determining whether the first object is cached by the network caching device;
• and
• if the first object is cached by the network caching device, then in response to the first request,
• identifying a first metafile entry as being associated with the first object, the first metafile entry containing metadata for a plurality of related objects, including the first object, stored in a single file in the mass storage facility,
• using the metadata in the first metafile entry to locate and read the plurality of related objects, including the first object, from the mass storage facility into the RAM, and
• providing the first object to the client;
• receiving a request for a second object over the network;
• determining whether the second object is cached by the network caching device;
• and
• if the second object is not cached by the network caching device, then in response to the request for the second object,
• retrieving the second object from an origin server,
• providing the second object to the requesting client in response to the second request,
• determining whether the second object is related to the first object, wherein determining whether the second object is related to the first object comprises
• determining whether a source of the request for the second object matches a source of the request for the first object;
• and
• determining whether a predetermined length of time has elapsed from the request for the first object to the request for the second object, and
• if the second object is determined to be related to the first object and a predetermined size condition is met, then
• writing the second object to the first file in the mass storage facility, and
• writing metadata for the second object into the first metafile entry, wherein the predetermined size condition comprises the conditions that
• writing metadata for the second object into the first metafile entry will not cause the first metafile entry to exceed a first predetermined size, and
• writing the second object to the first file would cause the first file to exceed a second predetermined size.

A machine-readable program storage medium for use in a network-connected processing system which includes a primary storage and a secondary storage, the storage medium containing instructions which, when executed by the processing system, cause the processing system to perform a process comprising:

• creating a set of metafile entries, each of the metafile entries containing metadata for a logical grouping of related objects stored by the processing system;
• and
• using the set of metafile entries to reduce accesses to the secondary storage in response to requests for the objects, including:
• receiving a request for an object from a client over a network;
• determining whether the object is cached by the network device;
• and
• if the object is not cached by the network device, then in response to the request for the object,
• determining whether the object is related to a previously requested object, wherein the object is determined to be related to the previously requested object if the object and the previously requested object are determined to have been requested by the same client within a specified period of time, and
• if the object is determined to be related to the previously requested object and a predetermined condition is met, then
• writing the object to a file in the secondary storage, and
• writing metadata for the object into a metafile entry which contains metadata for both the object and the previously requested object.

A network caching device comprising:

• a processor;
• a network interface coupled to the processor to connect the network caching device to a network;
• a main memory coupled to the processor;
• a mass storage facility coupled to the processor;
• and
• a set of instructions executable by the processor to cause the network caching device to perform a process which includes
• receiving a request for a first object from a client over the network;
• determining whether the first object is cached by the network caching device;
• if the first object is cached by the network caching device, then in response to the first request,
• identifying a first metafile entry as being associated with the first object, the first metafile entry containing metadata for a plurality of related objects, including the first object, stored in the mass storage facility,
• using the metadata in the first metafile entry to locate and read the plurality of related objects, including the first object, from the mass storage facility into the main memory, and
• providing the first object to the client;
• receiving a request for a second object from a client over the network;
• determining whether the second object is cached by the network caching device;
• and
• if the second object is not cached by the network caching device, then in response to the request for the second object,
• retrieving the second object from an origin server,
• providing the second object to the requesting client in response to the second request,
• determining whether the second object is related to the first object, and
• if the second object is determined to be related to the first object and a predetermined size condition is met, then
• writing the second object to the first file in the mass storage facility, and
• writing metadata for the second object into the first metafile entry.

A network device comprising:

• a primary storage;
• a secondary storage;
• means for creating a set of metafile entries, each of the metafile entries containing metadata for a logical grouping of related objects cached by the network device;
• and
• means for using the set of metafile entries to reduce accesses to the secondary storage in response to requests for the objects, by
• receiving a request for an object from a client over a network;
• determining whether the object is cached by the network device;
• and
• if the object is not cached by the network device, then in response to the request for the object,
• determining whether the object is related to a previously requested object, wherein the object is determined to be related to the previously requested object if the object and the previously requested object are determined to have been requested by the same client within a specified period of time, and
• if the object is determined to be related to the previously requested object and a predetermined condition is met, then
• writing the object to a file in the secondary storage, and
• writing metadata for the object into a metafile entry which contains metadata for both the object and the previously requested object.