Topic-based watermarking for large language models
OwnerErman Ayday, Alexander Nemecek +1inventors
InventorsErman Ayday, Alexander Nemecek, Yuzhou Jiang
Life of this patent
Estimating the term…
Abstract
A computer-implemented method can include receiving, at language model, an input prompt. The method can also include determining a semantic topic associated with the input prompt. The method can also include adjusting token selection probabilities to favor tokens aligned with the semantic topic. The method can also include generating output text using the adjusted probabilities, whereby usage of tokens in the output text encodes a topic-based watermark.
Claims20
A computer-implemented method, comprising:
- receiving, at language model, an input prompt;
- determining a semantic topic associated with the input prompt;
- adjusting token selection probabilities to favor tokens aligned with the semantic topic;
- and
- generating output text using the adjusted probabilities, whereby usage of tokens in the output text encodes a topic-based watermark.
A system, comprising:
- one or more processors;
- and
- memory storing machine-readable instructions that, when executed by the one or more processors, cause the one or more processors to at least:
- receive an input prompt;
- generate, based on the input prompt, logits for candidate tokens using a large language model;
- adjust the logits to bias token selection toward a topic-based subset of tokens;
- and
- generate output text using the adjusted logits, whereby usage of tokens in the output text encodes a topic-based watermark.
A computer-implemented method of detecting watermarking in text, comprising:
- receiving a text sequence;
- computing a score representing a frequency of tokens for the text sequence associated with at least one topic of a plurality of predefined semantic topics for a vocabulary of a large language model;
- comparing the score to a threshold;
- and
- determining whether the text sequence includes a watermark based on the comparison.
Dependencies are inferred from the claim text.