Rate of penetration forecasting while drilling using a transformer-based deep learning model

A system for forecasting a rate of penetration (ROP) while drilling a subterranean wellbore, the system comprising:

• a downhole tool configured for deployment in a drill string, the downhole tool including a trained transformer-based machine learning model, wherein the trained transformer-based machine learning model is trained with historical drilling data obtained from a plurality of drilled wells to establish relationships between measured drilling parameters and ROP, the training comprising:
• evaluating the historical drilling data to identify seasonal patterns in the historical drilling data;
• and
• dividing the historical drilling data into a plurality of sub-sequences based on the identified seasonal patterns;
• a processor configured to:
• receive short context drilling data from a plurality of downhole sensors while drilling the subterranean wellbore, the short context drilling data including a plurality of measured drilling parameters and a corresponding ROP;
• evaluate the short context drilling data using the trained transformer-based machine learning model to update the relationships between the measured drilling parameters and the ROP;
• forecast a future ROP using the short context drilling data and the updated relationships;
• and
• deploy the trained transformer-based machine learning model in the downhole tool configured for deployment in the drill string.

A method for forecasting a rate of penetration (ROP) while drilling a subterranean wellbore, the method comprising:

• training a transformer-based machine learning model with historical drilling data obtained from a plurality of drilled wells to establish relationships between measured drilling parameters and ROP, the training comprising:
• evaluating the historical drilling data to identify seasonal patterns in the historical drilling data;
• dividing the historical drilling data into a plurality of sub-sequences based on the identified seasonal patterns;
• and
• encoding the sub-sequences as tokens using a transformer-based architecture to capture the relationships between the measured drilling parameters and the ROP and thereby obtain a trained transformer-based machine learning model;
• acquiring short context drilling data while drilling the subterranean wellbore, the short context drilling data including a plurality of measured drilling parameters and a corresponding ROP;
• evaluating the short context drilling data using the trained transformer-based machine learning model to update the relationships between the measured drilling parameters and the ROP;
• forecasting a future ROP using the short context drilling data and the updated relationships;
• and
• deploying the trained transformer-based machine learning model in a downhole tool configured for deployment in a drill string.