Scalable transaction-based data repository service

A system, comprising:

• one or more processors;
• and
• one or more memories that include instructions executable by the one or more processors to cause the system to at least:
• receive a plurality of candidate transactions from a plurality of data producers, the plurality of candidate transactions being associated with an election transaction for uploading to a candidate dataset to a data repository, the election transaction being one of a plurality of election transactions, wherein identifiers for the plurality of data producers are used to verify ownership of lock keys and consistency of the data repository is validated before the election transaction is begun;
• maintain dataset metadata comprising a location of a plurality of candidate datasets corresponding to the plurality of candidate transactions and a commit state of the plurality of candidate datasets;
• assign a first exclusive lock key for a candidate transaction of the plurality of candidate transactions, the first exclusive lock key being assigned such that no more than one candidate transaction of the plurality of candidate transactions of the election transaction starts at any given time;
• modify the first exclusive lock key to a shared lock key of a plurality of shared lock keys, the plurality of shared lock keys being held by the plurality of candidate transactions of the election transaction, the plurality of shared lock key being utilized to enforce that at least two candidate transactions of the plurality of candidate transactions of the election transaction execute at least in part concurrently;
• in accordance with at least one other candidate transaction being associated with one of the plurality of shared lock keys, modify the shared lock key to a second exclusive lock key to enforce that no more than a single candidate transaction of the election transaction commits to a candidate dataset at a time, wherein the first exclusive lock key, the plurality of shared lock keys, and the second exclusive lock key are provided to enforce that candidate transactions of other election transactions cannot commit;
• and
• update the commit state of the candidate dataset in the dataset metadata based at least in part on completion of the plurality of candidate transactions.

A non-transitory computer-readable storage medium having stored thereon executable instructions that, when executed by one or more processors of a computer system, cause the computer system to at least:

• receive a plurality of candidate transactions from a plurality of data producers, the plurality of candidate transaction being associated with an election transaction for uploading to a data repository, wherein identifiers for the plurality of data producers are used to verify ownership of lock keys and consistency of the data repository is validated before the election transaction is begun;
• maintain dataset metadata comprising locations of a plurality of candidate datasets and commit states of the plurality of candidate datasets;
• provide a first exclusive lock key for a candidate transaction of the plurality of candidate transactions, the first exclusive lock key being provided to enforce that no two candidate transactions of the plurality of candidate transactions of the election transaction start at any given time;
• modify the first exclusive lock key to a shared lock key of a plurality of shared lock keys, the plurality of shared lock keys being held by the plurality of candidate transactions of the election transaction, the plurality of shared lock key being utilized to enforce that at least two candidate transactions of the plurality of candidate transactions of the election transaction execute at least in part concurrently;
• in accordance with at least one other candidate transaction being associated with one of the plurality of shared lock keys, modify the shared lock key to a second exclusive lock key to enforce that no more than a single candidate transaction of the election transaction commits to a candidate dataset at a time, wherein the first exclusive lock key, the plurality of shared lock keys, and the second exclusive lock key are provided to enforce that candidate transactions of other election transactions cannot commit;
• and
• update a commit state of the candidate dataset in the dataset metadata based at least in part on completion of the plurality of candidate transactions.