Hybrid inference for an efficient, low latency llm-based assistant
AssigneeGoogle Inc.
InventorsMatthew Sharifi, Victor Carbune
Life of this patent
Estimating the term…
Loading patent…
AssigneeGoogle Inc.
InventorsMatthew Sharifi, Victor Carbune
Estimating the term…
Implementations utilize a hybrid use of a smaller LLM and a larger LLM to generate and refine content responsive to a user query/request for content generation. In various implementations, the smaller LLM is utilized to process the user query for content generation, to generate initial content responsive to the user query for content generation. The user query for content generation and the initial content can be utilized to generate a text prompt, where the text prompt can be configured to further include a request for focused edit(s). Such a text prompt can be processed using the larger LLM, to generate focused edit(s) to the initial content that refine the initiated content, so that revised content (with improved accuracy) responsive to the user query for content generation is acquired.
A method implemented by one or more processors, the method comprising:
A method implemented by one or more processors, the method comprising:
A method implemented by one or more processors, the method comprising:
Dependencies are inferred from the claim text.