US 2024/0205174 A1 · DEVICE SENSOR INFORMATION AS CONTEXT FOR INTERACTIVE CHATBOT

Abstract

Implementations relate to processing, utilizing a large language model (“LLM”), input that is based on sensor data, from sensor(s) of a client device, to generate LLM output—and causing output, that is based on the generated LLM output, to be rendered by an interactive chatbot. The input that is based on sensor data and that is processed by the LLM in generating the LLM output can be, or can include, non-acoustic input based on non-acoustic sensor data. For example, an instance of LLM output can be generated based on processing of non-acoustic input using the LLM and without any processing of acoustic input (that is based on acoustic sensor data) using the LLM. As another example, an instance of LLM output can be generated based on processing, using the LLM, both non-acoustic input that is based on non-acoustic data and acoustic input that is based on acoustic sensor data.

Claims22

Dependent claims

A method implemented by one or more processors, the method comprising:

receiving non-acoustic sensor data from a non-acoustic sensor of a client device;
determining whether the received non-acoustic sensor data satisfies one or more conditions;
and
in response to determining that the received non-acoustic sensor data satisfies the one or more conditions:
processing the received non-acoustic sensor data to generate a natural language description for the non-acoustic sensor data,
processing, using a large language model (LLM), the generated natural language description for the non-acoustic sensor data, as input, to generate an LLM output,
generating, based on the LLM output, a natural language statement that is responsive to the received non-acoustic sensor data, and
causing the natural language statement to be rendered at the client device via an interactive chatbot that is installed at the client device or that is otherwise accessible via the client device.

5 more elements

Dependencies are inferred from the claim text.

Device sensor information as context for interactive chatbot

Life of this patent

Device sensor information as context for interactive chatbot

Life of this patent

Abstract

Claims22