System and method of selective fine-tuning for custom training of a natural language to logical form model

A system comprising:

• one or more processors;
• and
• one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the system to perform operations for fine-tuning a machine learning model comprising plurality of layers, the operations comprising:
• identifying an initial set of configurations, each configuration establishing a subset of the plurality of layers to be frozen and a subset of the plurality of layers to be fine-tuned;
• determining a configuration, of the initial set of configurations, that is optimized with respect to one or more parameters, thereby establishing a set of fine-tuning layers of the machine learning model and a set of frozen layers of the machine learning model;
• providing, to a remote system, an input for the machine learning model;
• obtaining, from the remote system, an output of the set of frozen layers of the machine learning model given the provided input;
• locally storing the obtained output of the set of frozen layers;
• loading, from the remote system, the set of fine-tuning layers of the machine learning model;
• and
• for a plurality of iterations, fine-tuning the set of fine-tuning layers of the machine learning model by:
• retrieving the locally stored output of the set of frozen layers, and
• updating weights associated with the set of fine-tuning layers of the machine learning model, thereby generating a customized machine learning model.

One or more non-transitory computer-readable media storing instructions for fine-tuning a machine learning model comprising plurality of layers which, when executed by one or more processors, cause a system to perform operations comprising:

• identifying an initial set of configurations, each configuration establishing a subset of the plurality of layers to be frozen and a subset of the plurality of layers to be fine-tuned;
• determining a configuration, of the initial set of configurations, that is optimized with respect to one or more parameters, thereby establishing a set of fine-tuning layers of the machine learning model and a set of frozen layers of the machine learning model;
• providing, to a remote system, an input for the machine learning model;
• obtaining, from the remote system, an output of the set of frozen layers of the machine learning model given the provided input;
• locally storing the obtained output of the set of frozen layers;
• loading, from the remote system, the set of fine-tuning layers of the machine learning model;
• and
• for a plurality of iterations, fine-tuning the set of fine-tuning layers of the machine learning model by:
• retrieving the locally stored output of the set of frozen layers, and
• updating weights associated with the set of fine-tuning layers of the machine learning model, thereby generating a customized machine learning model.