In response to a user interacting with a tangible peripheral assistant control device (e.g., depressing a button of the device), causing an automated assistant to perform one or more actions. The action(s) performed can be based on input previously provided by the user in configuring the peripheral assistant control device. The action(s) performed in response to interaction with the peripheral assistant control device can vary based on one or more conditions, such as which user is currently active, where the peripheral assistant control device is currently located (which can optionally be inferred based on which of multiple assistant computing devices the button is paired with), and/or the current state of one or more smart devices and/or other devices (e.g., as determined based on a device topology). A utility of the peripheral assistant control device can be automatically extended beyond what was specifically requested by a user during configuration.
Drawings
Claims20
A method implemented by one or more processors, the method comprising:
determining, based on data that is accessible via a computing device, whether a first user or a second user is a currently active user of a peripheral assistant control device,
wherein the peripheral assistant control device is in wireless communication with the computing device, and
wherein an automated assistant is accessible via the computing device;
receiving, from the peripheral assistant control device, other data that is generated in response to a touch interface of the peripheral assistant control device receiving a touch input,
wherein receipt of the other data that is generated in response to interaction with the touch interface causes the automated assistant to perform a function, selected from at least a first automated assistant function and a second automated assistant function, according to whether the first user or the second user is determined to be the currently active user of the peripheral assistant control device;
when the first user is determined to be the currently active user:
causing, in response to receiving the other data and determining that the first user is the currently active user, the automated assistant to execute the first automated assistant function;
and
when the second user is determined to be the currently active user:
causing, in response to receiving the other data and determining that the second user is the currently active user, the automated assistant to execute the second automated assistant function, which is different from the first automated assistant function.
7 more elements
A method implemented by one or more processors, the method comprising:
determining, based on data that is accessible via a computing device, whether a peripheral assistant control device is located at a first area or a second area,
wherein the peripheral assistant control device is in wireless communication with the computing device, and an automated assistant is accessible via the computing device;
receiving, from the peripheral assistant control device, other data that is generated in response to a touch interface of the peripheral assistant control device receiving a touch input,
wherein interaction with the touch interface causes the automated assistant to perform a function, selected from at least a first automated assistant function and a second automated assistant function, according to whether the peripheral assistant control device is determined to be located at the first area or the second area;
when the peripheral assistant control device is determined to be at the first area:
causing, in response to receiving the other data and determining that the peripheral assistant control device is determined to be at the first area, the automated assistant to execute the first automated assistant function;
and
when the peripheral assistant control device is determined to be at the second area:
causing, in response to receiving the other data and determining that the peripheral assistant control device is determined to be at the second area, the automated assistant to execute the second automated assistant function, which is different from the first automated assistant function.
6 more elements
A method implemented by one or more processors, the method comprising:
receiving audio data corresponding a spoken utterance provided by a user to an automated assistant interface of a computing device, wherein the computing device provides access to an automated assistant and is connected to a peripheral assistant control device;
determining, based on the audio data, that the spoken utterance includes a request for controlling one or more functions in response to an interaction with a touch interface of the peripheral assistant control device, wherein the peripheral assistant control device provides an output to the computing device, or another computing device, in response to the interaction with the touch interface;
determining, based on the audio data, that the spoken utterance at least partially identifies the one or more functions for correlating to the output of the peripheral assistant control device;
identifying, based on data that is associated with the one or more functions, one or more other functions that are executable via the computing device or the automated assistant, wherein the spoken utterance is void of content specifying the one or more other functions;
and
generating correlation data that correlates the output of the peripheral assistant control device with the one or more functions when at least a first condition is satisfied, and correlates the output of the peripheral assistant control device with the one or more other functions when at least a second condition is satisfied.
16/610,815Filed 1 Feb 2019 · us provisional application
Priority
7 Dec 2018
Published
US 2021/0072953 A111 Mar 2021
Granted
1 Feb 2022
Term ends
1 Oct 2039Estimated · howhide20 years from earliest known nonprovisional domestic filing plus recorded patent term adjustment, including 242 days of adjustment. Maintenance-fee lapses, terminal disclaimers and statutory extensions have not been checked. The continuity chain may be incomplete; this is not an enforceability determination.
Indexing scheme relating to G06F21/00 and subgroups addressing additional information or applications relating to security arrangements for…:…
Life of this patent
About 12 years 11 months of term left · ends 1 Oct 2039 (estimated)
Priority7 Dec 2018
Filed1 Feb 2019
Published11 Mar 2021
Granted1 Feb 2022
Today12 Oct 2026
Term ends1 Oct 2039
Priority7 Dec 2018
Granted1 Feb 2022
Term ends1 Oct 2039
Priority claimedParent application 62/776,979
Application filedUS 16/610,815
Assigned to Google LLCassignment of assignor's interest
PublishedUS 2021/0072953 A1
GrantedUS 11,237,795 B2
TodayWithin its term
Term endsEstimated, includes 242 days of adjustment
Description15,595 words
Summary6,018 words
#Humans may engage in human-to-computer dialogs with interactive software applications referred to herein as “automated assistants” (also referred to as “digital agents,” “chatbots,” “interactive personal assistants,” “intelligent personal assistants,” “assistant applications,” “conversational agents,” etc.). For example, humans (which when they interact with automated assistants may be referred to as “users”) may provide commands and/or requests to an automated assistant using spoken natural language input (i.e. utterances), which may in some cases be converted into text and then processed, and/or by providing textual (e.g., typed) natural language input. An automated assistant responds to a request by providing responsive user interface output, which can include audible and/or visual user interface output.
#In many cases, users provide spoken utterances to cause an automated assistant to perform various actions. However, extensive computational and/or network resources can be utilized in processing spoken utterances to perform automated assistant actions. For instance, an automated assistant action of turning on a smart light can be accomplished by transmitting a corresponding command to the smart light directly, or to a third-party server that processes the command, then transmits a corresponding command to the smart light. However, in performing such an action in response to a spoken utterance of “Assistant, turn on smart light X”, speech-to-text processing is performed on audio data that captures the spoken utterance, natural language processing is performed based on the resulting text, and then the corresponding command is determined. Further, where speech-to-text and/or natural language processing is performed remotely, audio data and/or other data will need to be transmitted over one or more networks.
#Implementations set forth herein relate to causing an automated assistant to perform one or more actions responsive to a user interacting with a peripheral assistant control device. The peripheral assistant control device can be paired (e.g., via Bluetooth or other communications protocol) with an assistant computing device that includes an automated assistant interface and that alone and/or through interaction with remote automated assistant components, enables performance of various automated assistant functions. In response to user interaction with the peripheral assistant control device, output can be provided by the peripheral assistant control device to the paired assistant computing device. In response to receiving the output, the paired assistant computing device can initiate the performance of one or more automated assistant actions/functions. Implementations described herein may serve to increase the utility of peripheral assistant control devices such that fewer peripheral assistant control devices may be required to enable performance of a given number of automated assistant functions. In addition, some implementations may facilitate/reduce the complexity of configuration of the peripheral assistant control devices. Furthermore, since the peripheral assistant control devices may reduce the amount of speech recognition processing (and associated bandwidth usage etc.) that is required by the automated assistant system, increasing the utility/functionality of the peripheral assistant control devices may serve to further reduce the processing, bandwidth etc. that is utilized by the automated assistant system as a whole.
Brief description of the drawings5 figures
#FIG. 1A and FIG. 1B illustrate views of a peripheral assistant control device having functions that are separately assigned according to whether a particular user is determined to be present in an area that includes the peripheral assistant control device.
#FIG. 2A and FIG. 2B illustrate views of control of one or more functions being assigned to a peripheral assistant control device based on a location of the peripheral assistant control device.
#FIG. 3 illustrates a system for assigning control, over one or more executable assistant functions, to a peripheral assistant control device according to whether one or more different conditions are satisfied.
Detailed description9,434 words
#FIG. 1A and FIG. 1B illustrate a view 100 and a view 130 of a peripheral assistant control device 112 having functions that are separately assigned according to whether a particular user is determined to be present in an area that includes the peripheral assistant control device 112. Specifically, FIG. 1A illustrates a view 100 of a peripheral assistant control device 112 being assigned a first assistant function when a first user 106 is determined to be present in a location of the peripheral assistant control device 112, and/or otherwise determined to be proximate to the peripheral assistant control device 112. For example, when a first user 106 is determined to be located within an area 128 of their home 126 (e.g., a kitchen of the home), the peripheral assistant control device 112 can reassigned control of a first assistant function (e.g., turn on/off kitchen lights). However, when the user is determined to have relocated away from the area 128 and/or the home 126, as illustrated in FIG. 1B, and/or a second user 108 is determined to the present within the area 128 of the home 126, and/or otherwise proximate to the peripheral assistant control device 112, the peripheral assistant control device 112 can be assigned to control of a second assistant function (e.g., play music at a first client device 110).
#For instance, the first user 106 can provide a request to an application accessible via a computing device for requesting that the peripheral assistant control device 112 control lights in their kitchen when the first user 106, specifically, is interacting with the peripheral assistant control device 112. Additionally, the second user 108 can also use an application for requesting that the peripheral assistant control device 112 control music rendered by the first client device 110 when the second user 108, specifically, is interacting with the peripheral assistant control device 112. Thereafter, when the first user 106 is determined to by present and interacting with the peripheral assistant control device 112, the peripheral assistant control device 112 can be assigned control over the lights in the kitchen. Therefore, any interaction with between the first user 106 and the peripheral assistant control device 112 at that time will result in operations of the kitchen lights being modified. Furthermore, when the first user 106 is no longer present, but the second user 108 is present and interacting with the peripheral assistant control device 112, any interactions between the second user 108 and the peripheral assistant control device 112 will result in causing the first client device 110 to render music based on the interactions.
#In some implementations, a server device 120 and/or a client device can consider either the first user 106 or the second user 108 “active” with respect to their ability to influence assignment of controls for the peripheral assistant control device 112. Furthermore, whether a particular user is “active” with respect to the peripheral assistant control device 112 can be based on a voice identification of the first user 106 and/or the second user 108. For instance, and with previous permission from the first user 106, the first client device 110 and/or the server device 120 can perform speech processing to determine whether the first user 106 has provided a recent spoken utterance to the first client device 110. If the first user 106 has been identified, based on voice characteristics of the first user 106, to have been the user who most recently provided the spoken utterance, the peripheral assistant control device 112 can be assigned control over one or more particular functions (e.g., controlling the kitchen lights) as preferred by the first user 106. Furthermore, the second user 108 can subsequently interact with the first client device 110 by providing a spoken utterance. The first client device 110 and/or the server device 120 can identify the second user 108 as the user who provided the subsequent spoken utterance, and cause the peripheral assistant control device 112 to be assigned control over one or more other particular functions (e.g., controlling music rendered by the first client device 110) as preferred by the second user 108.