Secure sharing of key-value cache
AssigneesBeijing Volcano Engine Technology Co., Ltd., Southern University of Science and Technology
InventorsGuanlong WU, Yao ZHANG, Yinqian ZHANG, Zheng ZHANG, Ye WU
Life of this patent
Estimating the term…
Abstract
According to embodiments of the disclosure, a method, an apparatus, a device and a medium for securely sharing key-value cache of a language model are provided. The method includes: obtaining request content input to a language model; recognizing protected information and non-protected information in the request content; determining, for a first part in the non-protected information whose matched key-value is absent from a shared key-value cache of the language model, a key-value corresponding to the first part and adding the key-value to the shared key-value cache, where a key-value of the protected information is not added to the shared key-value cache.
Claims20
A method for securely sharing key-value cache of a language model, comprising:
- obtaining request content input to a language model;
- recognizing protected information and non-protected information in the request content;
- and
- determining, for a first part in the non-protected information whose matched key-value is absent from a shared key-value cache of the language model, a key-value corresponding to the first part and adding the key-value to the shared key-value cache,
- wherein a key-value of the protected information is not added to the shared key-value cache.
A method for security detection of a model service, comprising:
- obtaining cache information of a first model service, the cache information generated based on a first request sent to the first model service;
- and
- performing security detection on the first model service based on lifetime of the cache information, to obtain a result of security detection for cache sharing of the first model service.
An electronic device comprising a memory, a processor, and a computer program, the computer program being stored on the memory and executable on the processor, the processor, when executes the program, implementing acts comprising:
- obtaining request content input to a language model;
- recognizing protected information and non-protected information in the request content;
- determining, for a first part in the non-protected information whose matched key-value is absent from a shared key-value cache of the language model, a key-value corresponding to the first part and adding the key-value to the shared key-value cache,
- wherein a key-value of the protected information is not added to the shared key-value cache.
Dependencies are inferred from the claim text.