Parameter Efficient Prompt Tuning for Efficient Models at Scale
AssigneeGoogle Inc.
InventorsBrian David Lester, Noah J.G. Constant, Rami Eid Sammour Al-Rfou
Life of this patent
Estimating the term…
Loading patent…
AssigneeGoogle Inc.
InventorsBrian David Lester, Noah J.G. Constant, Rami Eid Sammour Al-Rfou
Estimating the term…
Systems and methods for natural language processing can leverage trained prompts to condition a large pre-trained machine-learned model to generate an output for a specific task. For example, a subset of parameters may be trained for the particular task to then be input with a set of input data into the pre-trained machine-learned model to generate the task-specific output. During the training of the prompt, the parameters of the pre-trained machine-learned model can be frozen, which can reduce the computational resources used during training while still leveraging the previously learned data from the pre-trained machine-learned model.
A computer-implemented method, the computer-implemented method comprising:
A computing system, the computing system comprising:
One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, the operations comprising:
Dependencies are inferred from the claim text.