# US 2023/0054765 A1 · Base calling using three-dimentional (3d) convolution

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0054765 A1
- Document ID: US20230054765A1
- Status: Published application
- Kind: A1
- Application number: 17/969,613
- Filed: 2022-10-19
- Published: 2023-02-23
- Earliest claimed: 2019-05-16
- Assignees: Illumina, Inc.
- Applicants: Illumina, Inc.
- Inventors: Emrah KOSTEM
- Web page: https://patentgrep.com/patent/US20230054765A1

## CPC

- G16B 40/10: Signal processing, e.g. from mass spectrometry [MS] or from PCR
- G01N 21/6428: {Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes" (in vivo A61B5/00; immunoassay G01N33/53)}
- G01N 21/6454: {using an integrated detector array}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/048: Activation functions
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching}
- G06V 10/758: {Involving statistics of pixels or of feature values, e.g. histogram matching}
- G06V 10/76: {based on eigen-space representations, e.g. from pose or different illumination conditions; Shape manifolds}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G16B 30/00: ICT specially adapted for sequence analysis involving nucleotides or amino acids
- C12Q 1/6869: Methods for sequencing
- G06N 3/045: Combinations of networks
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N 5/048: Fuzzy inferencing
- G16B: BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- C12Q: MEASURING OR TESTING PROCESSES INVOLVING ENZYMES, NUCLEIC ACIDS OR MICROORGANISMS (immunoassay G01N33/53); COMPOSITIONS OR TEST PAPERS THEREFOR; PROCESSES OF PREPARING SUCH COMPOSITIONS; CONDITION-RESPONSIVE CONTROL IN MICROBIOLOGICAL OR ENZYMOLOGICAL PROCESSES

## Abstract

We propose a neural network-implemented method for base calling analytes. The method includes accessing a sequence of per-cycle image patches for a series of sequencing cycles, where pixels in the image patches contain intensity data for associated analytes, and applying three-dimensional (3D) convolutions on the image patches on a sliding convolution window basis such that, in a convolution window, a 3D convolution filter convolves over a plurality of the image patches and produces at least one output feature. The method further includes beginning with output features produced by the 3D convolutions as starting input, applying further convolutions and producing final output features and processing the final output features through an output layer and producing base calls for one or more of the associated analytes to be base called at each of the sequencing cycles.

## Claims

### Claim 1

A neural network-implemented method of base calling analytes, the method comprising:
accessing a sequence of per-cycle image patches generated for a series of sequencing cycles of a sequencing run, wherein pixels in the sequence of per-cycle image patches contain intensity data for associated analytes in one or more imaged channels;
applying three-dimensional (3D) convolutions on the sequence of per-cycle image patches on a sliding convolution window basis such that, in a convolution window, a 3D convolution filter convolves over a plurality of the per-cycle image patches and produces at least one output feature as a result of convolving over the sequence of per-cycle image patches on the sliding convolution window basis;
beginning with output features produced by the 3D convolutions as starting input, applying further convolutions and producing final output features; and
processing the final output features through an output layer and producing base calls for one or more of the associated analytes to be base called at each of the sequencing cycles.

### Claim 2 (depends on 1)

The neural network-implemented method of claim 1, wherein the 3D convolution filter convolves over
a plurality of the per-cycle image patches along a temporal dimension and detects and accounts for phasing and prephasing effects between successive ones of the sequencing cycles caused by asynchronous readout of sequence copies of an associated analyte,
a plurality of pixels in each of the per-cycle image patches along spatial dimensions and detects and accounts for spatial crosstalk between adjacent analytes caused by detection of emissions from a non-associated analyte by a corresponding light sensor of an associated analyte, and
each of the imaged channels along a depth dimension and detects and accounts for emission overlap between the imaged channels caused by overlap of dye emission spectra.

### Claim 3 (depends on 2)

The neural network-implemented method of claim 2, further comprising:
supplementing the output features produced as a result of a plurality of 3D convolution filters convolving over the sequence of per-cycle image patches with
imaged channel-specific and cross-cycle intensity data features of one or more of the pixels that contain the intensity data for one or more of the associated analytes to be base called;
beginning with the output features supplemented with the intensity data features as starting input, applying a cascade of one-dimensional (1D) convolutions and producing further output features, the cascade using 1D convolutions with different receptive fields and detecting varying degrees of the asynchronous readout caused by the phasing and prephasing effects; and
applying pointwise convolutions on the further output features and producing final output features.

### Claim 4 (depends on 1)

The neural network-implemented method of claim 1, further comprising:
for an associated analyte to be base called,
producing a final output feature for each of the imaged channels;
normalizing unnormalized per-cycle values in final output features of the imaged channels;
converting normalized per-cycle values into per-cycle binary values based on a threshold; and
base calling the associated analyte at each of the sequencing cycles based on the per-cycle binary values at corresponding positions in the final output features.

### Claim 5 (depends on 4)

The neural network-implemented method of claim 4, wherein the output layer comprises a sigmoid function that squashes the unnormalized per-cycle values in the final output features between zero and one; and
assigning those squashed per-cycle values that are below the threshold a zero value and assigning those squashed per-cycle values that are above the threshold a one value.

### Claim 6 (depends on 1)

The neural network-implemented method of claim 1, wherein the output layer comprises a softmax function that produces an exponentially normalized probability distribution of a base incorporated at a sequencing cycle in an associated analyte to be base called being A, C, T, and G; and
classifying the base as A, C, T, or G based on the distribution.

### Claim 7 (depends on 3)

The neural network-implemented method of claim 3, further comprising:
the 3D convolutions separately applying a respective convolution kernel on each of the imaged channels and producing at least one intermediate output feature for each of the imaged channels;
the 3D convolutions further combining intermediate output features of the imaged channels and producing output features, wherein the output features represent information mixed between the imaged channels; and
beginning with the output features supplemented with the intensity data features as starting input, applying the cascade of 1D convolutions.

### Claim 8 (depends on 3)

The neural network-implemented method of claim 3, further comprising:
the 3D convolutions separately applying a respective convolution kernel on each of the imaged channels and producing at least one intermediate output feature for each of the imaged channels;
the 3D convolutions further combining intermediate output features of the imaged channels and producing output features, wherein the output features represent information mixed between the imaged channels; and
beginning with the output features supplemented with the intensity data features as starting input, applying a plurality of cascade of 1D convolutions such that each cascade in the plurality corresponds to one of the imaged channels and operates on input independent of another cascade.

### Claim 9 (depends on 3)

The neural network-implemented method of claim 3, further comprising:
the 3D convolutions separately applying a respective convolution kernel on each of the imaged channels and producing at least one intermediate output feature for each of the imaged channels;
the 3D convolutions not combining intermediate output features of the imaged channels and instead making them available as imaged channel-specific output features;
supplementing the imaged channel-specific output features with cross-cycle intensity data features from a corresponding imaged channel of one or more of the pixels that contain the intensity data for one or more of the associated analytes to be base called; and
beginning with the imaged channel-specific output features supplemented with the intensity data features as starting input, applying the cascade of 1D convolutions.

### Claim 10 (depends on 3)

The neural network-implemented method of claim 3, further comprising:
the 3D convolutions separately applying a respective convolution kernel on each of the imaged channels and producing at least one intermediate output feature for each of the imaged channels;
the 3D convolutions not combining intermediate output features of the imaged channels and instead making them available as imaged channel-specific output features;
supplementing the imaged channel-specific output features with cross-cycle intensity data features from a corresponding imaged channel of one or more of the pixels that contain the intensity data for one or more of the associated analytes to be base called; and
beginning with the imaged channel-specific output features supplemented with the intensity data features as starting input, applying a plurality of cascade of 1D convolutions such that each cascade in the plurality corresponds to one of the imaged channels and operates on input independent of another cascade.

### Claim 11 (depends on 10)

The neural network-implemented method of claim 10, further comprising:
1D convolutions mixing information between respective per-cycle elements of each of the output features and the intensity data features on a sliding window basis and producing at least one intermediate output feature for each of the output features and the intensity data features; and
1D convolutions accumulating information across intermediate output features of the output features on a per-cycle element basis and producing further output features.

### Claim 12 (depends on 11)

The neural network-implemented method of claim 11, wherein size of the sliding window is based on a receptive field of the 1D convolutions and varies in the cascade.

### Claim 13 (depends on 11)

The neural network-implemented method of claim 11, further comprising:
applying a combination of the 1D convolutions and transposed convolutions instead of the cascade of 1D convolutions, wherein the combination alternates between application of the 1D convolutions and the transposed convolutions.

### Claim 14 (depends on 11)

The neural network-implemented method of claim 11, further comprising:
pointwise convolutions respectively convolving over further output features on a per-cycle element basis and producing at least one intermediate output feature for each of the further output features; and
pointwise convolutions accumulating information across intermediate output features of the further output features on a per-cycle element basis and producing at least one final output feature.

### Claim 15 (depends on 4)

The neural network-implemented method of claim 4, further comprising:
using the normalized per-cycle values in the final output features of the imaged channels to assign quality scores to base call predictions emitted by the output layer based on a quality score mapping, wherein the quality score mapping is determined by:
calculating predicted error rates for base call predictions made on training data and determining corresponding predicted quality scores;
determining a fit between the predicted quality scores and empirical quality scores determined from empirical base calling error rates derived from test data; and
correlating the predicted quality scores to the empirical quality scores based on the fit.

### Claim 16 (depends on 14)

The neural network-implemented method of claim 14, further comprising:
learning kernel weights of convolution filters applied by the 3D convolutions, the 1D convolutions, and the pointwise convolutions using a backpropagation-based gradient update technique that progressively matches the base call predictions emitted by the output layer with ground truth.

### Claim 17 (depends on 16)

The neural network-implemented method of claim 16, wherein the ground truth includes per-cycle binary values for each of the imaged channels, further comprising:
the backpropagation-based gradient update technique computing an error between the per-cycle binary values in the ground truth and corresponding per-cycle binary values in the final output features of the imaged channels.

### Claim 18 (depends on 16)

The neural network-implemented method of claim 16, wherein the ground truth includes a one-hot encoding identifying a correct base, further comprising:
the backpropagation-based gradient update technique computing an error between the one-hot encoding in the ground truth and exponentially normalized probability distribution produced by a softmax function.

### Claim 19 (depends on 16)

The neural network-implemented method of claim 16, further comprising varying a learning rate of the learning.

### Claim 20 (depends on 1)

The neural network-implemented method of claim 1, further comprising extracting the per-cycle image patches from respective per-cycle images of a tile of a flow cell on which the analytes are disposed.

### Claim 21 (depends on 20)

The neural network-implemented method of claim 20, further comprising base calling analytes disposed throughout the tile by extracting per-cycle image patches from overlapping regions of the tile such that the extracted per-cycle image patches have overlapping pixels.

### Claim 22

A non-transitory computer readable storage medium impressed with computer program instructions to base call analytes, the instructions, when executed on a processor, implement a method comprising:
accessing a sequence of per-cycle image patches generated for a series of sequencing cycles of a sequencing run, wherein pixels in the per-cycle image patches contain intensity data for associated analytes in one or more imaged channels;
applying three-dimensional (3D) convolutions on the sequence of per-cycle image patches on a sliding convolution window basis such that, in a convolution window, a 3D convolution filter convolves over a plurality of the per-cycle image patches and produces at least one output feature as a result of convolving over the sequence of per-cycle image patches on the sliding convolution window basis;
beginning with output features produced by the 3D convolutions as starting input, applying further convolutions and producing final output features; and
processing the final output features through an output layer and producing base calls for one or more of the associated analytes to be base called at each of the sequencing cycles.

### Claim 23

A system including one or more processors coupled to memory, the memory loaded with computer instructions to base call analytes, the instructions, when executed on the processors, implement actions comprising:
accessing a sequence of per-cycle image patches generated for a series of sequencing cycles of a sequencing run, wherein pixels in the per-cycle image patches contain intensity data for associated analytes in one or more imaged channels;
applying three-dimensional (3D) convolutions on the sequence of per-cycle image patches on a sliding convolution window basis such that, in a convolution window, a 3D convolution filter convolves over a plurality of the per-cycle image patches and produces at least one output feature as a result of convolving over the sequence of per-cycle image patches on the sliding convolution window basis;
beginning with output features produced by the 3D convolutions as starting input, applying further convolutions and producing final output features; and
processing the final output features through an output layer and producing base calls for one or more of the associated analytes to be base called at each of the sequencing cycles.