Background movement of data between nodes in a storage cluster

A computer-readable storage medium having computer-executable instructions stored thereon which, when executed by a computer, cause the computer to:

• generate a positive exception table for a first node in a storage cluster, the positive exception table comprising a bitmap identifying one or more areas on the first node that need to be synchronized from the first node to a destination node in the storage cluster;
• generate a negative exception table for the destination node, the negative exception table comprising a bitmap identifying one or more areas on the destination node that need to be synchronized from other nodes in the storage cluster to the destination node;
• execute a background synchronization thread to synchronize the contents of the first node from the first node to the destination node in the storage cluster based upon the data contained in the positive exception table, wherein the background synchronization thread is operative to
• identify based upon the contents of the positive exception table an area of the first node that needs to be migrated to the destination node,
• lock the area of the first node containing the data to be migrated,
• copy the data to be migrated to a buffer,
• write the data from the buffer to an area on the destination node, and
• following completion of the write, alter the positive exception table in the first node to indicate that the area of the first node does not need to be migrated to the destination node and alter the negative exception table in the destination node to indicate that the area of the destination node does not need to be migrated from the first node;
• receive an input/output (I/O) operation directed toward an area of the destination node identified in the negative exception table as requiring synchronization from another node in the storage cluster;
• in response to receiving the I/O operation, redirect the I/O operation to the other node in the storage cluster for fielding;
• receive an input/output (I/O) operation directed toward an area of the first node identified in the positive exception table as requiring synchronization to the destination node in the storage cluster;
• and
• fielding the I/O operation locally in response to receiving the I/O operation directed toward an area of the first node identified in the positive exception table as requiring synchronization to the destination node.

A computer system comprising two or more storage server computers, each storage server computer operating as a storage node, and each storage node configured to generate a first bitmap identifying storage areas for migration to a destination node in the computer system in response to determining that another of the storage nodes has failed, and to execute a background process for migrating the contents of the areas of the storage nodes to the destination node according to the contents of the first bitmap, wherein the destination node contains a second bitmap identifying one or more areas on the destination node that need to be migrated from the storage nodes in the computer system to the destination node and the background process is configured to:

• identify based upon the contents of the first bitmap an area of a source storage node that needs to be migrated to the destination node;
• lock the area of the source storage node containing the data to be migrated;
• copy the data to be migrated to a buffer;
• perform a write operation to write the data from the buffer to an area on the destination node;
• and
• following completion of the write operation, alter the first bitmap on the source storage node to indicate that the area of the source storage node does not need to be migrated to the destination node and alter the second bitmap on the destination node to indicate that the area of the destination node does not need to be migrated from the source storage node.