Bounding delays and reducing threading overheads in caching

A method for retrieving information from a server for multiple clients, comprising the steps of:

• creating a plurality of threads associated with clients, wherein at least one of the threads can operate as a producer thread that stores information in a memory and as a consumer thread that transmits information from the memory;
• determining when a first one of the plurality of threads requires additional information;
• requesting the additional information from a server using the first one of the plurality of threads as a producer thread on behalf of the other of the plurality of threads while restricting the other of the plurality of threads from requesting the additional information from the server;
• and
• transmitting the additional information from the memory using the first one of the plurality of threads as a consumer thread.

A method for sending information to multiple clients from a server, comprising the steps of:

• creating respective threads for requests for information from the multiple clients, wherein at least one of the threads can operate as a producer thread that stores information received from the server into a memory and as a consumer thread that retrieves information from the memory;
• determining when one of the threads requires additional information for one of the multiple clients;
• requesting the additional information from a server using one of the threads as a producer thread on behalf of the other of the threads while restricting the other of the threads from requesting the additional information from the server;
• receiving the additional information from the server in response to the request by the one of the threads that is operating as the producer thread;
• storing the additional information in the memory using the one of the threads as the producer thread;
• for each client, retrieving the additional information from the memory using respective threads associated with each client as consumer threads.

A method of retrieving information from a server for multiple clients, comprising the steps of:

• creating a plurality of respective threads each associated with a client and operative to function as a consumer that retrieves information from a cache and a producer that stores information in the cache;
• requesting a data block of information from the server using a first one of the plurality of threads;
• storing the data block received from the server in a cache using the first one of the plurality of threads as a producer;
• transmitting the data block of information from the cache to a first client using the first one of the plurality of threads as a consumer;
• requesting a second data block of information from the server using a second one of the plurality of threads;
• storing the second data block of information from the server in the cache using the second one of the plurality of threads as a producer;
• determining when one of the first one and second one of the plurality of threads requires an additional data block of information from the server based on a status of the cache;
• and
• requesting the additional data block of information using the one of the first one and second one of the plurality of threads as a producer on behalf of the other of the plurality of threads while restricting the other of the first one and second one of the plurality of threads from requesting the additional data block of information.

A data processing system for retrieving information from a server for multiple client requests, comprising:

• a memory containing:
• a cache entry for storing information, and
• a thread processing component that
• creates a plurality of respective threads each associated with a client, wherein each thread is operative to function as a producer thread that stores information in the memory and a consumer thread that retrieves information from the memory;
• assigns a first one of the plurality of the threads as a producer thread;
• requests information from a server using the first one of the plurality of the threads as the producer thread, and
• transmits information received from the server to a client using the first one of the plurality of threads as a consumer thread;
• determines when the first one of the plurality of threads requires additional information from the server;
• requests the additional information from the server using the first one of the plurality of threads as the producer thread on behalf of the other of the plurality of threads while restricting the other of the plurality of threads from requesting the additional information from the server;
• and
• a processor for running the thread processing component.

A computer readable medium containing computer-executable instructions for receiving information from a server for a client to perform a method, comprising the steps of:

• creating a first thread associated with a request by the client for information from the server, wherein the first thread stores information in a memory and transmits information from the memory to the client;
• using the first thread, storing a first data block of the requested information from the server into the memory;
• transmitting the first data block of information to the client using the first thread;
• transmitting a second data block of information to the client using the first thread, wherein the second data block was stored into the memory using a second thread, the second thread being associated with another client and being operative to store information in the memory and transmit information from the memory to the another client;
• determining when one of the first and second threads requires an additional data block of information;
• and
• requesting the additional data block from the server using the one of the first and second threads on behalf of the other of the first and second threads while restricting the other of the first and second threads from requesting the additional data block of information.

An apparatus for receiving information from a server for a client, comprising:

• means for creating a first thread associated with a request by the client for information from the server, wherein the first thread stores information in a memory and transmits information from the memory to the client;
• means for storing a first data block of the requested information from the server into the memory;
• means for transmitting the first data block of information to the client;
• means for transmitting a second data block of information to the client using the first thread, wherein the second data block was stored into the memory using a second thread, the second thread being associated with another client and being operative to store information in the memory and transmit information from the memory to the another client;
• means for determining when one of the first and second threads requires an additional data block of information;
• and
• means for requesting the additional data block from the server using the one of the first and second threads on behalf of the other of the first and second threads while restricting the other of the first and second threads from requesting the additional data block of information.