# US 12,001,862 B1 · Disambiguating user input with memorization for improved user assistance

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,001,862 B1
- Document ID: US12001862B1
- Status: Granted
- Kind: B1
- Application number: 16/135,752
- Filed: 2018-09-19
- Granted: 2024-06-04
- Earliest claimed: 2018-04-20
- Assignees: Meta Platforms, Inc.
- Applicants: Meta Platforms, Inc.
- Inventors: Xiaohu Liu; Baiyang Liu
- Web page: https://patentgrep.com/patent/US12001862B1

## CPC

- G06Q 10/00: Administration; Management
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 7/14: Merging, i.e. combining at least two sets of record carriers each arranged in the same ordered sequence to produce a single set having the same ordered sequence
- G06F 8/31: {Programming languages or programming paradigms}
- G06F 9/44505: {Configuring for program initiating, e.g. using registry, configuration files}
- G06F 9/4451: {User profiles; Roaming}
- G06F 9/453: {Help systems}
- G06F 16/176: Support for shared access to files; File sharing support
- G06F 16/2255: {Hash tables}
- G06F 16/2365: {Ensuring data consistency and integrity}
- G06F 16/243: {Natural language query formulation}
- G06F 16/24552: {Database cache management}
- G06F 16/24575: {using context}
- G06F 16/24578: {using ranking}
- G06F 16/248: Presentation of query results
- G06F 16/3322: {using system suggestions (G06F16/3325 takes precedence)}
- G06F 16/3323: {using document space presentation or visualization, e.g. category, hierarchy or range presentation and selection}
- G06F 16/3329: Natural language query formulation
- G06F 16/3344: {using natural language analysis}
- G06F 16/338: Presentation of query results
- G06F 16/90332: {Natural language query formulation or dialogue systems}
- G06F 16/90335: {Query processing}
- G06F 16/9038: Presentation of query results
- G06F 16/904: Browsing; Visualisation therefor (for navigating the web G06F16/954; browsing optimisation for the web G06F16/957)
- G06F 16/951: Indexing; Web crawling techniques
- G06F 16/9532: Query formulation
- G06F 16/9535: Search customisation based on user profiles and personalisation
- G06F 18/2411: based on the proximity to a decision surface, e.g. support vector machines
- G06F 21/6245: {Protecting personal data, e.g. for financial or medical purposes}
- G06F 40/205: Parsing
- G06F 40/274: Converting codes to words; Guess-ahead of partial word inputs
- G06F 40/295: Named entity recognition
- G06F 40/30: Semantic analysis
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N 5/027: {Frames}
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- G06Q 10/40: Business processes related to social networking or social networking services
- G06Q 10/42: Determination of affinities or common interests between users
- G06Q 10/48: using social graphs
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/063: {Training}
- G10L 15/07: to the speaker
- G10L 15/16: using artificial neural networks
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 15/183: using context dependencies, e.g. language models
- G10L 15/187: Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 17/06: Decision making techniques; Pattern matching strategies
- G10L 17/22: Interactive procedures; Man-machine interfaces
- H04L 5/02: Channels characterised by the type of signal
- H04L 12/2816: {Controlling appliance services of a home automation network by calling their functionalities (arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station; in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00)}
- H04L 41/20: {Network management software packages}
- H04L 41/22: comprising specially adapted graphical user interfaces [GUI]
- H04L 43/0882: Utilisation of link capacity
- H04L 43/0894: Packet rate
- H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
- H04L 51/18: Commands or executable codes
- H04L 51/216: Handling conversation history, e.g. grouping of messages in sessions or threads
- H04L 51/52: for supporting social networking services
- H04L 63/102: {Entity profiles}
- H04L 67/306: User profiles
- H04L 67/535: {Tracking the activity of the user (network monitoring arrangements H04L43/00; recording of computer activity G06F11/34)}
- H04L 67/5651: Reducing the amount or size of exchanged application data
- H04L 67/75: Indicating network or usage conditions on the user display
- H04W 12/08: Access security
- G06F 2216/13: Prefetching
- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 13/04: Details of speech synthesis systems, e.g. synthesiser structure or memory management
- G10L 17/00: Speaker identification or verification techniques
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/225: {Feedback of the input speech}
- H04L 51/046: Interoperability with other network applications or services
- H04L 51/222: using geographical location information, e.g. messages transmitted or received in proximity of a certain spot or area
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- H04L 67/53: using third party service providers
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

In one embodiment, a method includes receiving a user input including an ambiguous mention by a user from the user's client system, analyzing the user input to identify intents and slots based on a natural-language understanding module, where at least one slot is an ambiguous slot corresponding to the ambiguous mention, identifying candidate entities corresponding to the ambiguous slot, ranking the candidate entities based on the user's user profile data, sending to the user's client system instructions for prompting the user to select a candidate entity from a set of candidate entities having a rank greater than a threshold rank, where the selected candidate entity is resolved to the ambiguous slot, and storing the selected candidate entity together with the ambiguous mention as a mention-entity pair as part of the user profile data responsive to receiving the selection from the user.

## Claims

### Claim 1

A method comprising, by one or more computing systems:
receiving, from a client system associated with a first user during a first dialog session, a user input by the first user, wherein the user input comprises an ambiguous mention;
analyzing, based on a natural-language understanding module, the user input to identify one or more intents and one or more slots, wherein at least one of the one or more slots is an ambiguous slot corresponding to the ambiguous mention;
determining one or more tasks based on the one or more intents associated with the user input;
identifying one or more candidate entities corresponding to the ambiguous slot;
ranking the one or more candidate entities based on user profile data associated with the first user;
sending, to the client system associated with the first user during the first dialog session, instructions for prompting the first user to select a candidate entity from a set of candidate entities having a rank greater than a threshold rank, wherein the selected candidate entity is resolved to the ambiguous slot;
receiving, from the client system during the first dialog session, a user selected candidate entity from the first user;
determining, during the first dialog session, whether to store the user selected candidate entity together with the ambiguous mention as a mention-entity pair in a dialog state associated with the first dialog session or as part of the user profile data based on whether the stored mention-entity pair is to be used for resolving the ambiguous mention during the first dialog session or across a plurality of dialog sessions; and
storing the mention-entity pair during the first dialog session, wherein the mention-entity pair is associated with the one or more tasks and a decay factor indicating a confidence for resolving the ambiguous mention based on the mention-entity pair when the mention-entity pair is stored as part of the user profile data, wherein the decay factor is determined based on a slot-type of the ambiguous slot corresponding to the ambiguous mention and the one or more intents, and wherein the mention-entity pair is discarded after the first dialog session when the mention-entity pair is stored in the dialog state associated with the first dialog session.

### Claim 2 (depends on 1)

The method of claim 1, wherein the one or more candidate entities are associated with one or more confidence scores, respectively.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
selecting the set of candidate entities from the one or more candidate entities.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
identifying one or more task states associated with the one or more tasks.

### Claim 5 (depends on 4)

The method of claim 4, wherein identifying the one or more candidate entities corresponding to the ambiguous slot is based on one or more of:
user profile data associated with the first user;
a social graph;
one or more dialog states of one or more dialog sessions associated with the user input; or
the one or more task states.

### Claim 6 (depends on 1)

The method of claim 1, further comprising assigning an expiration time to the stored mention-entity pair, wherein the expiration time is determined based on one or more of:
user profile data associated with the first user;
a social graph; or
the dialog state associated with the first dialog session.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
determining, based on the assigned expiration time, a type of memory storage for storing the mention-entity pair.

### Claim 8 (depends on 1)

The method of claim 1, further comprising assigning the decay factor to the mention-entity pair when the mention-entity pair is stored as part of the user profile data.

### Claim 9 (depends on 1)

The method of claim 1, wherein the decay factor is determined further based on one or more of:
a confidence score associated with the mention-entity pair; or
a longevity associated with the mention-entity pair.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
receiving, from the client system associated with the first user during a second dialog session, a subsequent user input, wherein the subsequent user input comprises the ambiguous mention;
accessing the user profile data associated with the first user; and
identifying, for the ambiguous mention, the selected candidate entity based on the stored mention-entity pair.

### Claim 11 (depends on 1)

The method of claim 1, wherein storing the mention-entity pair as part of the user profile data comprises:
accessing user profile associated with the first user, wherein the user profile comprises privacy settings associated with the first user; and
determining, based on the privacy settings, the mention-entity pair is allowed by the first user to be stored as part of the user profile data.

### Claim 12 (depends on 1)

The method of claim 1, wherein storing the mention-entity pair comprises:
sending, to the client system associated with the first user, instructions for prompting the first user to confirm that the first user wants to store the mention-entity pair; and
receiving, from the client system associated with the first user, a confirmation from the first user to store the entity-pair.

### Claim 13 (depends on 1)

The method of claim 1, wherein the mention-entity pair is stored as part of the user profile data, and wherein the method further comprises:
sending, to the client system associated with the first user, instruction for presenting a user interface operable to allow the first user to edit the stored mention-entity pair.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
receiving, from the client system via the user interface, a user edit by the first user to the stored mention-entity pair; and
updating, in the user profile data, the stored mention-entity pair based on the user edit.

### Claim 15 (depends on 14)

The method of claim 14, wherein the user edit comprises one or more of:
adding more information to the stored mention-entity pair;
modifying a content of the stored mention-entity pair; or
deleting the stored mention-entity pair.

### Claim 16

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
receive, from a client system associated with a first user during a first dialog session, a user input by the first user, wherein the user input comprises an ambiguous mention;
analyze, based on a natural-language understanding module, the user input to identify one or more intents and one or more slots, wherein at least one of the one or more slots is an ambiguous slot corresponding to the ambiguous mention;
identify one or more candidate entities corresponding to the ambiguous slot;
determine one or more tasks based on the one or more intents associated with the user input;
rank the one or more candidate entities based on user profile data associated with the first user;
send, to the client system associated with the first user during the first dialog session, instructions for prompting the first user to select a candidate entity from a set of candidate entities having a rank greater than a threshold rank, wherein the selected candidate entity is resolved to the ambiguous slot;
receive, from the client system during the first dialog session, a user selected candidate entity from the first user;
determine, during the first dialog session, whether to store the user selected candidate entity together with the ambiguous mention as a mention-entity pair in a dialog state associated with the first dialog session or as part of the user profile data based on whether the stored mention-entity pair is to be used for resolving the ambiguous mention during the first dialog session or across a plurality of dialog sessions; and
store, the mention-entity pair during the first dialog session, wherein the mention-entity pair is associated with the one or more tasks and a decay factor indicating a confidence for resolving the ambiguous mention based on the mention-entity pair when the mention-entity pair is stored as part of the user profile data, wherein the decay factor is determined based on a slot-type of the ambiguous slot corresponding to the ambiguous mention and the one or more intents, and wherein the mention-entity pair is discarded after the first dialog session when the mention-entity pair is stored in the dialog state associated with the first dialog session.

### Claim 17

A system comprising: one or more processors; and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
receive, from a client system associated with a first user during a first dialog session, a user input by the first user, wherein the user input comprises an ambiguous mention;
analyze, based on a natural-language understanding module, the user input to identify one or more intents and one or more slots, wherein at least one of the one or more slots is an ambiguous slot corresponding to the ambiguous mention;
identify one or more candidate entities corresponding to the ambiguous slot;
determine one or more tasks based on the one or more intents associated with the user input;
rank the one or more candidate entities based on user profile data associated with the first user;
send, to the client system associated with the first user during the first dialog session, instructions for prompting the first user to select a candidate entity from a set of candidate entities having a rank greater than a threshold rank, wherein the selected candidate entity is resolved to the ambiguous slot;
receive, from the client system during the first dialog session, a user selected candidate entity from the first user;
determine, during the first dialog session, whether to store the user selected candidate entity together with the ambiguous mention as a mention-entity pair in a dialog state associated with the first dialog session or as part of the user profile data based on whether the stored mention-entity pair is to be used for resolving the ambiguous mention during the first dialog session or across a plurality of dialog sessions; and
store, the mention-entity pair during the first dialog session, wherein the mention-entity pair is associated with the one or more tasks and a decay factor indicating a confidence for resolving the ambiguous mention based on the mention-entity pair when the mention-entity pair is stored as part of the user profile data, wherein the decay factor is determined based on a slot-type of the ambiguous slot corresponding to the ambiguous mention and the one or more intents, and wherein the mention-entity pair is discarded after the first dialog session when the mention-entity pair is stored in the dialog state associated with the first dialog session.

### Claim 18 (depends on 1)

The method of claim 1, wherein the confidence for resolving the ambiguous mention based on the mention-entity pair represents a likelihood that the ambiguous mention is correctly resolved based on the mention-entity pair.