Scheduling and execution of dag-structured computation on rdma-connected clusters

A method for accessing data stored in a cache of a server, comprising:

• storing, in memory of a server, a metadata hash map that is associated with the cache of the server, wherein the metadata hash map includes one or more entries associated with data records stored in the cache of the server, wherein each of the entries are associated with a key for a corresponding data record in the cache of the server, wherein each of the entries store metadata for the corresponding data record stored in the cache of the server, and wherein the metadata comprises a server-side remote pointer that references the corresponding data record stored in the cache of the server, wherein one or more of the data records comprises a directed acyclic graph (DAG), wherein the directed acyclic graph is comprised of a collection of one or more nodes connected by one or more edges, wherein each of the nodes represents one or more tasks ordered into a sequence, and wherein each of the edges connected to a node represents one or more constraints on the task represented by the node;
• wherein the server provides the metadata hash map to a client, wherein the client accesses a selected data record stored in the cache on the server using a provided key by:
• (1) identifying one or more potentially matching entries in the metadata hash map using a hash code for the provided key;
• (2) accessing one or more data records stored in the cache on the server using the server-side remote pointers from the potentially matching entries in the metadata hash map;
• and
• (3) determining whether the accessed data records match the selected data record using the provided key;
• and
• wherein the client executes the tasks associated with the accessed data record using the constraints associated with the data record.

A computer program product for accessing data stored in a cache of a server, said computer program product comprising:

• a non-transitory computer readable storage medium having stored therein program instructions executable by one or more computers that cause the computers to perform the steps of:
• storing, in memory of a server, a metadata hash map that is associated with the cache of the server, wherein the metadata hash map includes one or more entries associated with data records stored in the cache of the server, wherein each of the entries are associated with a key for a corresponding data record in the cache on the server, wherein each of the entries store metadata for the corresponding data record stored in the cache on the server, and wherein the metadata comprises a server-side remote pointer that references the corresponding data record stored in the cache on the server, wherein one or more of the data records comprises a directed acyclic graph (DAG), wherein the directed acyclic graph is comprised of a collection of one or more nodes connected by one or more edges, wherein each of the nodes represents one or more tasks ordered into a sequence, and wherein each of the edges connected to a node represents one or more constraints on the task represented by the node;
• wherein the server provides the metadata hash map to a client, wherein the client accesses a selected data record stored in the cache on the server using a provided key by:
• (1) identifying one or more potentially matching entries in the metadata hash map using a hash code for the provided key;
• (2) accessing one or more data records stored in the cache on the server using the server-side remote pointers from the potentially matching entries in the metadata hash map;
• and
• (3) determining whether the accessed data records match the selected data record using the provided key;
• and
• wherein the client executes the tasks associated with the accessed data record using the constraints associated with the data record.