Online base call compression

A method comprising performing, by an inference circuit:

• receiving raw data from a sensor chip including a plurality of cells, the raw data including a plurality of measurements for each position of a respective nucleic acid molecule of at least 100,000 nucleic acid molecules, wherein at least a portion of the at least 100,000 nucleic acid molecules include clusters of nucleic acid molecules, wherein the nucleic acid molecules of a cluster correspond to a same template nucleic acid molecule;
• for each position of the respective nucleic acid molecule:
• determining, using the raw data, a nucleotide at the position, thereby generating a sequence read:
• for each sequence read for the at least 100,000 nucleic acid molecules:
• identifying a particular cluster corresponding to the sequence read;
• incrementing a counter for the particular cluster;
• determining that a first counter for a first cluster is greater than a threshold;
• and
• in response to determining that the first counter is greater than the threshold, discarding sequence reads corresponding to the first cluster.

A system comprising:

• a sensor chip including a plurality of sequencing cells, the plurality of sequencing cells including at least 100,000 sequencing cells;
• and
• one or more processors configured to perform:
• receiving a first stream of raw data from the sensor chip including the plurality of sequencing cells, the raw data including a plurality of measurements for each position of a respective nucleic acid molecule of at least 100,000 nucleic acid molecules;
• generating a second stream of read data that includes header information, basecall data, and quality scores for the at least 100,000 nucleic acid molecules;
• extracting, from the second stream, a first sub-stream of header information that identifies each of the at least 100,000 nucleic acid molecules;
• compressing, by a first thread, the first sub-stream of header information to generate compressed header information;
• extracting, from the second stream, a second sub-stream of basecall data that provides a basecall at each position of each of the at least 100,000 nucleic acid molecules;
• compressing, by a second thread, the second sub-stream of basecall data to generate compressed basecall data;
• extracting, from the second stream, a third sub-stream of quality score data that provides a quality score for each basecall at each position of each of the at least 100,000 nucleic acid molecules;
• compressing, by a third thread, the third sub-stream of quality score data to generate compressed quality score data;
• and
• outputting the compressed header information, the compressed basecall data, and the compressed quality score data.