# US 12,010,262 B2 · Auto-activating smart responses based on activities from remote devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,010,262 B2 - Document ID: US12010262B2 - Status: Granted - Kind: B2 - Application number: 16/998,450 - Filed: 2020-08-20 - Published: 2020-12-03 - Granted: 2024-06-11 - Earliest claimed: 2013-08-06 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Varun A. Vora; Swapnil R. Dave - Web page: https://patentgrep.com/patent/US12010262B2 ## CPC - H04M 3/42042: {Notifying the called party of information on the calling party (details within substation equipment H04M1/57, signalling details H04Q3/72)} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - H04M 1/64: Automatic arrangements for answering calls; Automatic arrangements for recording messages for absent subscribers; Arrangements for recording conversations (centralised dictation systems H04M11/10) - H04M 1/663: Preventing unauthorised calls to a telephone set - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An electronic device with one or more processors and memory includes a procedure for using a digital assistant to automatically respond to incoming communications. In some embodiments, the device obtains a speech input from a user, and, in response to obtaining the speech input, the device determines whether the speech input includes instructions for performing a specified action in response to receipt of a subsequent incoming communication from one or more specified senders. After storing the instructions, the device obtains an incoming communication from a respective sender. The device determines whether the respective sender is one of the one or more specified senders, and, upon determining that the respective sender is one of the one or more specified senders, the device performs the specified action in accordance with the instructions and thereafter automatically deleting the instructions. ## Claims ### Claim 1 A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs comprising instructions for: in response to obtaining a speech input from a user, determining whether the speech input includes instructions for performing a specified action in response to receipt of one or more specified subsequent incoming communication types from one or more specified senders; in response to receiving an incoming communication from a respective sender: determining whether the respective sender is one of the one or more specified senders; and determining whether the incoming communication is one of the one or more specified subsequent incoming communication types; and upon determining that the respective sender is one of the one or more specified senders and that the incoming communication is one of the one or more specified subsequent incoming communication types, performing the specified action in accordance with the instructions. ### Claim 2 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein a respective specified subsequent incoming communication type of the one or more specified subsequent incoming communication types is one of a phone call, voice message, voicemail, video call, SMS, MMS, VOIP call, or email. ### Claim 3 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein the incoming communication is a phone call, and wherein the specified action is sending a voice message. ### Claim 4 (depends on 3) The non-transitory computer-readable storage medium of claim 3, wherein at least a part of the voice message is a recording of a user's voice. ### Claim 5 (depends on 3) The non-transitory computer-readable storage medium of claim 3, wherein at least a part of the voice message is a digitized or text-to-speech version of input text. ### Claim 6 (depends on 1) The non-transitory computer-readable storage medium of claim 1, the one or more programs further comprising instructions for: performing natural language processing on the speech input so as to convert the speech input into text, and wherein storing the instructions includes storing the instructions as text. ### Claim 7 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein performing the specified action includes: composing a response to the respective sender in accordance with the instructions; and communicating the response to the sender. ### Claim 8 (depends on 7) The non-transitory computer-readable storage medium of claim 7, wherein the composing includes composing the response to the respective sender in accordance with the instructions and based at least in part on contextual information. ### Claim 9 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the respective sender is one of the one or more specified senders based on a list of contacts associated with the user of the device. ### Claim 10 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the respective sender is one of the one or more specified senders based on caller ID associated with the respective sender. ### Claim 11 (depends on 1) The non-transitory computer-readable storage medium of claim 1, the one or more programs further comprising instructions for: identifying an address associated with the sender of the communication; and wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the sender is one of the one or more specified senders based on the address associated with the respective sender of the communication. ### Claim 12 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the sender is one of the one or more specified senders based on a content of the incoming communication received from the respective sender. ### Claim 13 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein the action is selected from the group consisting of: sending a voice message to the sender; sending an SMS message to the sender; sending an MMS or photograph to the sender; sending an email to the sender; sending an event invite to the sender; sharing one or more calendar entries with the sender; and sharing one or more custom attributes with the sender. ### Claim 14 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein the instructions are automatically deleted after performing the specified action. ### Claim 15 (depends on 1) The non-transitory computer-readable storage medium of claim 1, wherein determining whether the speech input includes instructions for performing a specified action in response to receipt of a subsequent incoming communication from one or more specified senders comprises: in accordance with a determination that the speech input includes incomplete instructions for performing a specified action in response to receipt of a subsequent incoming communication from one or more specified senders, obtaining additional information from the user. ### Claim 16 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein obtaining additional information from the user comprises at least one of: initiating dialogue with the user; obtaining a second speech input from the user; and obtaining a physical input from the user. ### Claim 17 A method of causing a digital assistant to automatically respond to incoming communications, the method comprising: at one or more electronic devices each having one or more processors and memory: in response to obtaining a speech input from a user, determining whether the speech input includes instructions for performing a specified action in response to receipt of one or more specified subsequent incoming communication types from one or more specified senders; in response to receiving an incoming communication from a respective sender: determining whether the respective sender is one of the one or more specified senders; and determining whether the incoming communication is one of the one or more specified subsequent incoming communication types; and upon determining that the respective sender is one of the one or more specified senders and that the incoming communication is one of the one or more specified subsequent incoming communication types, performing the specified action in accordance with the instructions. ### Claim 18 (depends on 17) The method of claim 17, wherein a respective specified subsequent incoming communication type of the one or more specified subsequent incoming communication types is one of a phone call, voice message, voicemail, video call, SMS, MIMS, VOIP call, or email. ### Claim 19 (depends on 17) The method of claim 17, wherein the incoming communication is a phone call, and wherein the specified action is sending a voice message. ### Claim 20 (depends on 19) The method of claim 19, wherein at least a part of the voice message is a recording of a user's voice. ### Claim 21 (depends on 19) The method of claim 19, wherein at least a part of the voice message is a digitized or text-to-speech version of input text. ### Claim 22 (depends on 17) The method of claim 17, further comprising: performing natural language processing on the speech input so as to convert the speech input into text, and wherein storing the instructions includes storing the instructions as text. ### Claim 23 (depends on 17) The method of claim 17, wherein performing the specified action includes: composing a response to the respective sender in accordance with the instructions; and communicating the response to the sender. ### Claim 24 (depends on 23) The method of claim 23, wherein the composing includes composing the response to the respective sender in accordance with the instructions and based at least in part on contextual information. ### Claim 25 (depends on 17) The method of claim 17, wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the respective sender is one of the one or more specified senders based on a list of contacts associated with the user of the device. ### Claim 26 (depends on 17) The method of claim 17, wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the respective sender is one of the one or more specified senders based on caller ID associated with the respective sender. ### Claim 27 (depends on 17) The method of claim 17, further comprising: identifying an address associated with the sender of the communication; and wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the sender is one of the one or more specified senders based on the address associated with the respective sender of the communication. ### Claim 28 (depends on 17) The method of claim 17, wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the sender is one of the one or more specified senders based on a content of the incoming communication received from the respective sender. ### Claim 29 (depends on 17) The method of claim 17, wherein the action is selected from the group consisting of: sending a voice message to the sender; sending an SMS message to the sender; sending an MMS or photograph to the sender; sending an email to the sender; sending an event invite to the sender; sharing one or more calendar entries with the sender; and sharing one or more custom attributes with the sender. ### Claim 30 (depends on 17) The method of claim 17, wherein the instructions are automatically deleted after performing the specified action. ### Claim 31 (depends on 17) The method of claim 17, wherein determining whether the speech input includes instructions for performing a specified action in response to receipt of a subsequent incoming communication from one or more specified senders comprises: in accordance with a determination that the speech input includes incomplete instructions for performing a specified action in response to receipt of a subsequent incoming communication from one or more specified senders, obtaining additional information from the user. ### Claim 32 (depends on 31) The method of claim 31, wherein obtaining additional information from the user comprises at least one of: initiating dialogue with the user; obtaining a second speech input from the user; and obtaining a physical input from the user. ### Claim 33 An electronic device, comprising: one or more processors; and memory storing instructions for operating a digital assistant the one or more programs comprising instructions for: in response to obtaining a speech input from a user, determining whether the speech input includes instructions for performing a specified action in response to receipt of one or more specified subsequent incoming communication types from one or more specified senders; in response to receiving an incoming communication from a respective sender: determining whether the respective sender is one of the one or more specified senders; and determining whether the incoming communication is one of the one or more specified subsequent incoming communication types; and upon determining that the respective sender is one of the one or more specified senders and that the incoming communication is one of the one or more specified subsequent incoming communication types, performing the specified action in accordance with the instructions. ### Claim 34 (depends on 33) The electronic device of claim 33, wherein a respective specified subsequent incoming communication type of the one or more specified subsequent incoming communication types is one of a phone call, voice message, voicemail, video call, SMS, MMS, VOIP call, or email. ### Claim 35 (depends on 33) The electronic device of claim 33, wherein the incoming communication is a phone call, and wherein the specified action is sending a voice message. ### Claim 36 (depends on 35) The electronic device of claim 35, wherein at least a part of the voice message is a recording of a user's voice. ### Claim 37 (depends on 35) The electronic device of claim 35, wherein at least a part of the voice message is a digitized or text-to-speech version of input text. ### Claim 38 (depends on 33) The electronic device of claim 33, the one or more programs further comprising instructions for: performing natural language processing on the speech input so as to convert the speech input into text, and wherein storing the instructions includes storing the instructions as text. ### Claim 39 (depends on 33) The electronic device of claim 33, wherein performing the specified action includes: composing a response to the respective sender in accordance with the instructions; and communicating the response to the sender. ### Claim 40 (depends on 39) The electronic device of claim 39, wherein the composing includes composing the response to the respective sender in accordance with the instructions and based at least in part on contextual information. ### Claim 41 (depends on 33) The electronic device of claim 33, wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the respective sender is one of the one or more specified senders based on a list of contacts associated with the user of the device. ### Claim 42 (depends on 33) The electronic device of claim 33, wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the respective sender is one of the one or more specified senders based on caller ID associated with the respective sender. ### Claim 43 (depends on 33) The electronic device of claim 33, the one or more programs further comprising instructions for: identifying an address associated with the sender of the communication; and wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the sender is one of the one or more specified senders based on the address associated with the respective sender of the communication. ### Claim 44 (depends on 33) The electronic device of claim 33, wherein determining whether the respective sender is one of the one or more specified senders includes ascertaining whether the sender is one of the one or more specified senders based on a content of the incoming communication received from the respective sender. ### Claim 45 (depends on 33) The electronic device of claim 33, wherein the action is selected from the group consisting of: sending a voice message to the sender; sending an SMS message to the sender; sending an MMS or photograph to the sender; sending an email to the sender; sending an event invite to the sender; sharing one or more calendar entries with the sender; and sharing one or more custom attributes with the sender. ### Claim 46 (depends on 33) The electronic device of claim 33, wherein the instructions are automatically deleted after performing the specified action. ### Claim 47 (depends on 33) The electronic device of claim 33, wherein determining whether the speech input includes instructions for performing a specified action in response to receipt of a subsequent incoming communication from one or more specified senders comprises: in accordance with a determination that the speech input includes incomplete instructions for performing a specified action in response to receipt of a subsequent incoming communication from one or more specified senders, obtaining additional information from the user. ### Claim 48 (depends on 47) The electronic device of claim 47, wherein obtaining additional information from the user comprises at least one of: initiating dialogue with the user; obtaining a second speech input from the user; and obtaining a physical input from the user.