# US 12,763,074 B2 · Electronic personal interactive device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,763,074 B2 - Document ID: US12763074B2 - Status: Granted - Kind: B2 - Application number: 17/664,469 - Filed: 2022-05-23 - Published: 2022-09-08 - Granted: 2026-10-13 - Earliest claimed: 2010-05-13 - Assignees: Poltorak Technologies LLC - Applicants: Poltorak Technologies LLC - Inventors: Alexander I. Poltorak - Web page: https://patentgrep.com/patent/US12763074B2 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 16/435: Filtering based on additional data, e.g. user or group profiles - G06T 13/205: {driven by audio data} - G06V 40/174: {Facial expression recognition} - G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)} - G10L 13/00: Speech synthesis; Text to speech systems - G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser - G10L 15/142: {Hidden Markov Models [HMMs]} - G10L 15/16: using artificial neural networks - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications - G10L 25/63: for estimating an emotional state - H04L 12/2818: {from a device located outside both the home and the home network (access arrangements H04L12/2856; for remote control or remote monitoring of applications H04L67/025; arrangements for transmitting signals characterised by the use of a wireless eletrical link G08C17/00; telephonic communication systems adapted for combination with remote control systems H04M11/007)} - G10L 2015/223: {Execution procedure of a spoken command} - H04M 1/0202: {Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets (constructional features of telephone transmitters or receivers, e.g. of speakers or microphones H04M1/03)} - H04M 2250/10: including a GPS signal receiver - H04W 84/12: WLAN [Wireless Local Area Networks] - H04W 84/18: Self-organising networks, e.g. ad-hoc networks or sensor networks - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract An interface device and method of use, comprising audio and image inputs; a processor for determining topics of interest, and receiving information of interest to the user from a remote resource; an audio-visual output for presenting an anthropomorphic object conveying the received information, having a selectively defined and adaptively alterable mood; an external communication device adapted to remotely communicate at least a voice conversation with a human user of the personal interface device. Also provided is a system and method adapted to receive logic for, synthesize, and engage in conversation dependent on received conversational logic and a personality. ## Claims ### Claim 1 A method comprising: determining a topic of interest to a user, based on a natural language input from the user; automatically determining a context of at least one of the user and an activity of the user, by analyzing with at least one automated processor (a) at least one of a plurality of user-specific data inputs, (b) a stored history of past interactions with the user, and (c) a stored user profile associated with the user; automatically communicating the determined topic of interest to the user through a communication port to a remote server; automatically receiving in real time, at least one unit of information stored in the remote server dependent on the topic of interest through the communication port without requiring an express request from the user; automatically presenting a natural language output controlled by the at least one automated processor using an automated conversational agent comprising an anthropomorphic object configured to engage in a context-appropriate interactive spoken natural language conversation relating to the topic of interest according to conversational logic, and employing a learning system that is adaptive and changes internally as a consequence of experience, conveying the received at least one unit of information to the user in a manner dependent on the determined context of the at least one of the user and the activity of the user, said automatically presenting comprising implementing an automated conversational agent through a human-machine user interface which interacts with the user according to conversational logic; and brokering an ad or an incentive in real time by communication with a report server without communication of an express request for the ad of the incentive, dependent on at least one of the determined context of the at least one of the user and the activity of the user, and the topic of interest to the user. ### Claim 2 (depends on 1) The method according to claim 1, further comprising automatically determining an emotional state of the user, wherein the presenting is further dependent on the detected emotional state of the user and a desired future emotional state of the user. ### Claim 3 (depends on 1) The method according to claim 1, further comprising receiving spoken language from the user and outputting spoken language from an automated anthropomorphic object as part of an interactive spoken natural language conversation. ### Claim 4 (depends on 1) The method according to claim 1, wherein the determining the topic of interest to the user is further based on an automated analysis of the stored history of past interactions. ### Claim 5 (depends on 1) The method according to claim 1, wherein said automatically communicating through the communication port comprises communicating information dependent on the topic of interest to a plurality of remote server databases, and said automatically receiving information stored in the remote server dependent on the topic of interest through the communication port comprises automatically receiving the information from the plurality of remote server databases. ### Claim 6 (depends on 5) The method according to claim 5, further comprising automatically selecting a subset of the received information from the plurality of remote server databases selectively dependent on the context of the at least one of the user and the activity of the user. ### Claim 7 (depends on 1) The method according to claim 1, further comprising automatically receiving sponsored or advertising content through the communication port that is brokered in real time, and automatically presenting the real time brokered sponsored or advertising content to the user through the human-machine user interface. ### Claim 8 (depends on 7) The method according to claim 7, wherein the real time brokered sponsored or advertising content is related to the topic of interest and is sensitive to the determined context. ### Claim 9 (depends on 7) The method according to claim 7, wherein the sponsored or advertising content is related to a user profile associated with the user. ### Claim 10 A conversational agent system comprising: an input port configured to receive a natural language input from a user; at least one automated processor employing an adaptive learning system that changes internally as a consequence of experience, configured to: analyze at least one of a plurality of user-specific data inputs, a stored history of past interactions with the user, and a stored user profile associated with the user, to determine a context of at least one of the user and an activity of the user; control an automated conversational agent to in dependence on the adaptive learning system, and to engage in a context-appropriate interactive natural language conversation with the user comprising the topic of interest; and broker an ad or an incentive in real time by communication with a report server without communication of an express request for the ad or an incentive, dependent on at least one of the context of at least one of the user and the activity of the user, and the topic of interest; a communication port configured to: communicate the determined topic of interest to a remote server; receive information from the remote server stored in a remote database dependent on the topic of interest; receive the brokered ad or incentive; and a human-machine user interface comprising the automated conversational agent which interacts with the user according to natural language conversational logic to present the received ad or incentive and received information to the user in a manner dependent on the context of at least one of the user and the activity of the user. ### Claim 11 (depends on 10) The system according to claim 10, wherein the presentation is further dependent on an emotional state of the user. ### Claim 12 (depends on 10) The system according to claim 10, wherein the at least one processor is further configured to use the stored history of past interactions with the user to determine the topic of interest to the user. ### Claim 13 (depends on 1) The system according to claim 1, wherein the communication port is configured to communicate information dependent on the topic of interest to a plurality of remote servers, and to receive the information from the plurality of remote servers. ### Claim 14 (depends on 13) The system according to claim 13, wherein the at least one automated processor is further configured to select a subset of the received information from the plurality of remote databases selectively dependent on the context of at least one of the user and the activity of the user. ### Claim 15 (depends on 10) The system according to claim 10, wherein the ad or incentive is synthesized within the context-appropriate interactive natural language conversation. ### Claim 16 (depends on 15) The system according to claim 15, wherein the ad or incentive is brokered dependent on the stored user profile associated with the user. ### Claim 17 A non-transitory computer readable medium, containing instructions for controlling at least one automated processor, comprising: instructions for determining a topic of interest to a user, based on a natural language input from the user; instructions for determining a context of at least one of the user and an activity of the user, by analyzing with at least one automated processor of at least one of a plurality of user-specific data inputs, a stored history of past interactions with the user, and a stored user profile associated with the user; instructions for communicating the determined topic of interest to the user through a communication port to a remote server; instructions for receiving information stored in the remote server dependent on the topic of interest through the communication port; instructions for brokering an ad or incentive in real time personalized based on the topic of interest selected in dependence on the context of at least one of the user and the activity of the user; instructions for presenting a natural language output and employing an adaptive learning system that changes internally as a consequence of experience, conveying the received information in a manner dependent on the context of at least one of the user and the activity of the user, and the ad or incentive to the user; instructions for implementing a conversational agent through a human-machine user interface which interacts with the user according to conversational logic; and, instructions for controlling the automated conversational agent to engage in a context-appropriate interactive natural language conversation with the user comprising the topic of interest. ### Claim 18 (depends on 17) The non-transitory computer readable medium according to claim 17, wherein the conversation agent comprises an anthropomorphic object having a head presented through an audio-visual output, further comprising: instructions for analyzing an intonation of the user's voice; instructions for animating the head based on a parametrically deformable 3D mesh model to represent facial movements comprising speech and facial gestures; and instructions for implementing the conversational agent which interacts with the user using speech dependent on the intonation of the user's voice.