Performing useful computations while waiting for a line in a system with a software implemented cache

An apparatus for handling software cache misses, comprising:

• a processor;
• and
• a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to:
• process a software cache lookup instruction in a program;
• perform a software cache lookup operation;
• determine if the software cache lookup operation results in a cache miss;
• initiate, by a cache miss handler, a retrieval of data from a backing store in response to results of the software cache lookup operation resulting in a cache miss;
• and
• branch, by the cache miss handler, execution to an independent subroutine that executes while execution of the program waits for retrieval of the data from the backing store, wherein:
• the independent subroutine is compiled into a separate subroutine that is stored in a known address location,
• the known address location is passed into the cache miss handler as a parameter to thereby cause the cache miss handler to execute the independent subroutine at the known address location after initiating retrieval of data from the backing store,
• initiating a retrieval of data from a backing store is performed in a first thread of execution and branching execution to an independent subroutine is performed in a second thread of execution,
• useful work is performed during cache miss handling by performing a thread switch operation between the first and second threads, and
• the compiler specified independent subroutine comprises a code sequence that does not cause another cache miss due to the independent subroutine being a portion of code that does not reference a portion of code associated with the software cache lookup instruction, does not reference variables that are common between the independent subroutine and the portion of code associated with the software cache lookup instruction, and does not reference aliases of variables used in other portions of code.