Back to results
US 12,651,033 B2Granted
Utilizing a query response to automate a task associated with a webpage
AssigneeTiny Fish Inc.
InventorsShuhao Zhang, Qi Zhai, Daniel Lawrence Schafer, Pavlo Dudka, Mingyang Li, Urvish Dipakkumar Thakker
Life of this patent
Estimating the term…
Abstract
A query and processed webpage content are received. A prompt for a large language model based on the received query and the received processed webpage content is generated. A query response is received from the large language model that is generated in response to providing the prompt to the large language model. Inference patterns are generated based on the query response. The inference patterns are stored in an inference patterns store. The query response is provided.
Claims20
A method, comprising:
- receiving, by a web browser application associated with a client device, a first instance of a query and processed webpage content, wherein the processed webpage content is processed by determining corresponding information associated with a plurality of user interface web elements associated with a webpage, wherein the corresponding information associated with the plurality of user interface web elements associated with the webpage includes one or more of a corresponding role, a corresponding name, and/or a corresponding tag for a plurality of web elements associated with the webpage;
- generating a prompt for a large language model based on the received query and the received processed webpage content;
- receiving a query response from the large language model that is generated in response to providing the prompt to the large language model, wherein the query response preserves a mapping of response nodes to corresponding user interface web elements;
- generating inference patterns based on the query response;
- storing the inference patterns in an inference patterns store;
- providing the query response to a client software development kit (SDK) included in the browser application, wherein the client SDK includes one or more application programming interface(s) (APIs) that enable object-oriented programming interfaces to be generated based on the query response and provide functionality to interact with the web elements, wherein the APIs are supported by one or more programming languages;
- receiving, by the web browser application associated with the client device, a second instance of the query and the processed webpage content;
- determining, based on the mapping of response nodes to corresponding user interface web elements, that the query and the processed webpage content have been previously received;
- in response to a determination that query and the processed webpage content have been previously received:
- retrieving the inference patterns that are based on the query response received from the large language model from the inference pattern store;
- and
- generating a second query response utilizing the inference patterns from the inference pattern store;
- and
- generating, using the one or more APIs included in the client SDK, an automated task for the webpage utilizing the second query response.
11 more elements
A system, comprising:
- a processor configured to:
- receive, by a web browser application associated with a client device, a first instance of a query and processed webpage content, wherein the processed webpage content is processed by determining corresponding information associated with a plurality of user interface web elements associated with a webpage, wherein the corresponding information associated with the plurality of user interface web elements associated with the webpage includes one or more of a corresponding role, a corresponding name, and/or a corresponding tag for a plurality of web elements associated with the webpage;
- generate a prompt for a large language model based on the received query and the received processed webpage content;
- receive a query response from the large language model that is generated in response to providing the prompt to the large language model, wherein the query response preserves a mapping of response nodes to corresponding user interface web elements;
- generate inference patterns based on the query response;
- store the inference patterns in an inference patterns store;
- provide the query response to a client software development kit (SDK) included in the browser application, wherein the client SDK includes one or more application programming interface(s) (APIs) that enable object-oriented programming interfaces to be generated based on the query response and provide functionality to interact with the web elements, wherein the APIs are supported by one or more programming languages;
- receive, by the web browser application associated with the client device, a second instance of the query and the processed webpage content;
- determine, based on the mapping of response nodes to corresponding user interface web elements, that the query and the processed webpage content have been previously received;
- in response to a determination that query and the processed webpage content have been previously received:
- retrieve the inference patterns that are based on the query response received from the large language model from the inference pattern store;
- and
- generate a second query response utilizing the inference patterns from the inference pattern store;
- and
- generating, using the one or more APIs included in the client SDK, an automated task for the webpage utilizing the second query response;
- and
- a memory coupled to the processor and configured to provide the processor with instructions.
14 more elements
A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
- receiving, by a web browser application associated with a client device, a first instance of a query and processed webpage content, wherein the processed webpage content is processed by determining corresponding information associated with a plurality of user interface web elements associated with a webpage, wherein the corresponding information associated with the plurality of user interface web elements associated with the webpage includes one or more of a corresponding role, a corresponding name, and/or a corresponding tag for a plurality of web elements associated with the webpage;
- generating a prompt for a large language model based on the received query and the received processed webpage content;
- receiving a query response from the large language model that is generated in response to providing the prompt to the large language model, wherein the query response preserves a mapping of response nodes to corresponding user interface web elements;
- generating inference patterns based on the query response;
- storing the inference patterns in an inference patterns store;
- providing the query response to a client software development kit (SDK) included in the browser application, wherein the client SDK includes one or more application programming interface(s) (APIs) that enable object-oriented programming interfaces to be generated based on the query response and provide functionality to interact with the web elements, wherein the APIs are supported by one or more programming languages;
- receiving, by the web browser application associated with the client device, a second instance of the query and the processed webpage content;
- determining, based on the mapping of response nodes to corresponding user interface web elements, that the query and the processed webpage content have been previously received;
- in response to a determination that query and the processed webpage content have been previously received:
- retrieving the inference patterns that are based on the query response received from the large language model from the inference pattern store;
- and
- generating a second query response utilizing the inference patterns from the inference pattern store;
- and
- generating, using the one or more APIs included in the client SDK, an automated task for the webpage utilizing the second query response.
11 more elements
Dependencies are inferred from the claim text.