Load balancing and conflict processing in workflow with task dependencies

A system comprising:

• one or more data processors;
• and
• a non-transitory computer readable storage medium containing instructions which when executed on the one or more data processors, cause the one or more data processors to perform actions including:
• receiving a plurality of reads, each read of the plurality of reads being associated with a client;
• performing, for each read of the plurality of reads, an alignment process using the read and a reference data set to generate an alignment result, the alignment result identifying one or more portions of the reference data set to which the read is to be aligned, the reference data set including, at each position of a plurality of positions, a reference-data identifier corresponding to the position, wherein each read of the plurality of reads includes a set of identifiers, wherein each of the reference-data identifiers in the reference data set and each identifier of the set of identifiers represents a base;
• identifying a first pre-identified portion of the reference data set, the first pre-identified portion including a pseudogene that corresponds to a second pre-identified portion of the reference data set, and each of the first pre-identified portion and the second pre-identified portion corresponding to a subset of the plurality of positions;
• for each position of one or more positions of the first pre-identified portion:
• identifying a subset of the plurality of reads, wherein reads in the subset have one or more identifiers aligned to the position;
• for each identifier of at least one of the one or more identifiers:
• determining that the identifier does not match a reference-data identifier in the reference data set at the position;
• in response to determining that the identifier does not match the reference-data identifier in the reference data set at the position:
• defining a sparse indicator that represents a variant indicative of a difference between the identifier and the reference-data identifier;
• and
• assigning the sparse indicator to a bucket representative of a state-transition likelihood attributable to the sparse indicator;
• determining that a data verification condition is satisfied based on the bucket assignments;
• and
• in response to determining that the data verification condition is satisfied, transmitting a communication to a computing system, the communication identifying the client.

A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform actions including:

• receiving a plurality of reads, each read of the plurality of reads being associated with a client;
• performing, for each read of the plurality of reads, an alignment process using the read and a reference data set to generate an alignment result, the alignment result identifying one or more portions of the reference data set to which the read is to be aligned, the reference data set including, at each position of a plurality of positions, a reference-data identifier corresponding to the position, wherein each read of the plurality of reads includes a set of identifiers, wherein each of the reference-data identifiers in the reference data set and each value identifier of the set of identifiers represents a base;
• identifying a first pre-identified portion of the reference data set, the first pre-identified portion including a pseudogene that corresponds to a second pre-identified portion of the reference data set, and each of the first pre-identified portion and the second pre-identified portion corresponding to a subset of the plurality of positions;
• for each position of one or more positions of the first pre-identified portion:
• identifying a subset of the plurality of reads, wherein reads in the subset have one or more identifiers aligned to the position;
• for each identifier of at least one of the one or more identifiers:
• determining that the identifier does not match a reference-data identifier in the reference data set at the position;
• in response to determining that the identifier does not match the reference-data identifier in the reference data set at the position:
• defining a sparse indicator that represents a variant indicative of a difference between the identifier and the reference-data identifier;
• and
• assigning the sparse indicator to a bucket representative of a state-transition likelihood attributable to the sparse indicator;
• determining that a data verification condition is satisfied based on the bucket assignments;
• and
• in response to determining that the data verification condition is satisfied, transmitting a communication to a data generation system that identifies the client.