Method and system for fine-tuning neural conditional language models using constraints
AssigneeNaver Corporation
InventorsTomasz Korbak, Hady Elsahar, German Kruszewski, Marc Dymetman
Life of this patent
Estimating the term…
Abstract
A processor-implemented method for fine-tuning a pre-trained neural conditional language model to perform a downstream task. A pre-trained conditional language model and at least one target constraint for satisfying a task-related control objective are received. A neural model is trained to approximate a target conditional model that optimally reconciles a distance from the pre-trained conditional language model and the control objective across multiple contexts.
Claims32
A method implemented by a processor for fine-tuning a pre-trained neural conditional language model to perform a downstream task, the method comprising:
- receiving the pre-trained conditional language model having attributes with existing probability distributions conditioned on contexts;
- receiving at least one target constraint for satisfying a task-related control objective associated with the downstream task, the target constraint specifying an expectation of a target attribute;
- using the processor, training a neural model to approximate a target conditional model that reconciles a distance from the pre-trained conditional language model and the control objective across multiple contexts;
- and
- outputting the trained neural model representing the pre-trained neural conditional language model fine-tuned to perform the downstream task;
- wherein said training the neural model comprises:
- for each of one or more iterations:
- sampling contexts from a distribution of contexts;
- computing an unconditional energy-based model (EBM) for each sampled context using the pre-trained conditional language model and the target constraint;
- and
- updating the neural model over the sampled contexts using the computed unconditional EBMs.
A non-transitory computer-readable medium having executable instructions stored thereon for causing a processor and a memory to implement a method for fine-tuning a pre-trained neural conditional language model to perform a downstream task, the method comprising:
- receiving the pre-trained conditional language model having attributes with existing probability distributions conditioned on contexts;
- receiving at least one target constraint for satisfying a task-related control objective associated with the downstream task, the target constraint specifying an expectation of a target attribute;
- training a neural model using the processor to approximate a target conditional model that reconciles a distance from the pre-trained conditional language model and the control objective across multiple contexts;
- and
- outputting the trained neural model;
- wherein said training the neural model comprises:
- for each of one or more iterations:
- sampling contexts from a distribution of contexts;
- computing an unconditional energy-based model (EBM) for each sampled context using the pre-trained conditional language model and the target constraint;
- and
- updating the neural model over the sampled contexts using the computed unconditional EBMs.
A processor-based system for fine-tuning a pre-trained neural conditional language model to perform a downstream task, comprising:
- an energy-based model (EBM) computing block configured to compute using one or more processors an energy-based model based on a received pre-trained conditional language model having attributes with existing probability distributions conditioned on contexts and at least one received target constraint for satisfying a task-related control objective, the target constraint specifying an expectation of a target attribute;
- a context sampling block configured to sample using one or more processors a plurality of contexts;
- a model output sampling block configured to sample using one or more processors a plurality of model outputs associated with each sampled context;
- a normalization estimating block configured to estimate using one or more processors a normalization based on said computed energy-based model and said sampled plurality of model outputs;
- and
- a model updating block for training a neural model using one or more processors to approximate using one or more processors a target conditional model that reconciles a distance from the pre-trained conditional language model and the control objective across multiple contexts based on a loss gradient determined using said estimated normalization;
- the model updating block being further configured to output the trained neural model.
Dependencies are inferred from the claim text.