Loading patent…
OwnerKfir Aberman, Yuval Alaluf +2inventors
InventorsKfir Aberman, Yuval Alaluf, Daniel Cohen-Or, Sergey Tulyakov
Estimating the term…
Described herein are techniques for personalizing vision-language models (VLMs) to understand user-specific concepts, without modifying original model weights. Pre-trained VLMs are augmented with external concept heads that identify user-specific concepts in input images. A concept embedding vector is computed to represent the user-specific concept within an intermediate feature space of the VLM through iterative optimization. Then, when processing an input image and the concept is detected, the concept embedding is appended to image features extracted by a vision encoder of the VLM. Personalized textual outputs incorporating the user-specific concept are generated in response to input images and language instructions. Regularization techniques balance attention between the appended concept embedding and original image features, maintaining alignment between outputs and inputs. This approach enables VLMs to recognize and reason about personalized objects or individuals across diverse settings while preserving general capabilities, enhancing image captioning and visual question-answering with user-specific information.
A method for personalizing a vision-language model (VLM) to understand user-specific concepts, the method comprising:
A system for personalizing a vision-language model (VLM) to understand user-specific concepts, the system comprising:
A system for personalizing a vision-language model (VLM) to understand user-specific concepts, the system comprising:
Dependencies are inferred from the claim text.