# US 11,030,207 B1 · Updating displayed data visualizations according to identified conversation centers in natural language commands > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,030,207 B1 - Document ID: US11030207B1 - Status: Granted - Kind: B1 - Application number: 15/978,067 - Filed: 2018-05-11 - Granted: 2021-06-08 - Earliest claimed: 2016-04-12 - Assignees: Tableau Software, Inc. - Applicants: Tableau Software, Inc. - Inventors: Vidya R. Setlur; Sarah E. Battersby; Melanie K. Tory; Richard C. Gossweiler, III; Angel Xuan Chang; Isaac J. Dykeman; MD Enamul Hoque Prince - Web page: https://patentgrep.com/patent/US11030207B1 ## CPC - G06F 16/90332: {Natural language query formulation or dialogue systems} - G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials - G06F 16/243: {Natural language query formulation} - G06F 16/24522: {Translation of natural language queries to structured queries} - G06F 16/248: Presentation of query results - G06F 16/3329: Natural language query formulation - G06F 16/9038: Presentation of query results - G06F 18/22: Matching criteria, e.g. proximity measures - G06F 18/23213: with fixed number of clusters, e.g. K-means clustering - G06F 40/20: Natural language analysis (semantic analysis of natural language G06F40/30) - G06F 40/30: Semantic analysis - G06F 40/211: Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method uses natural language for visual analysis of a dataset. The method displays a data visualization based on a first dataset. The method then extracts analytic phrases from a first natural language command related to the data visualization. The method computes conversation centers associated with the first natural language command based on the analytic phrases and computes analytical functions for the conversation centers, thereby creating functional phrases. The method updates the data visualization based on the functional phrases. The method extracts new analytic phrases from a second natural language command related to the updated data visualization and computes a temporary conversation centers associated with the second natural language command based on the new analytic phrases. The method derives new conversation centers from the original conversation centers and the temporary conversation centers using transitional rules. The method then updates the data visualization based on the new conversation centers. ## Claims ### Claim 1 A method of using natural language for visual analysis of a dataset, comprising: at computer having a display, one or more processors, and memory storing one or more programs configured for execution by the one or more processors: displaying a data visualization based on a first dataset retrieved from a database using a first set of one or more queries; receiving a first user input to specify a first natural language command related to the data visualization; extracting a first set of one or more independent analytic phrases from the first natural language command; computing a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytic phrases; creating a first set of one or more functional phrases by computing a first set of analytical functions associated with the first set of one or more conversation centers; updating the data visualization based on the first set of one or more functional phrases; receiving a second user input to specify a second natural language command related to the updated data visualization; extracting a second set of one or more independent analytic phrases from the second natural language command; computing a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytic phrases; deriving a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers using one or more transitional rules; and updating the data visualization based on the second set of one or more conversation centers, including highlighting or filtering data marks whose characteristics correspond to a data attribute corresponding to the second set of one or more conversation centers. ### Claim 2 (depends on 1) The method of claim 1, wherein updating the data visualization based on the second set of one or more conversation centers comprises: determining one or more data attributes corresponding to the second set of one or more conversation centers; scanning displayed data visualizations to identify one or more of the displayed data visualizations that contain data marks whose characteristics correspond to a first data attribute in the one or more data attributes; and highlighting the data marks whose characteristics correspond to the first data attribute. ### Claim 3 (depends on 2) The method of claim 2, further comprising filtering results from the displayed data visualizations that contain data marks whose characteristics do not correspond to the one or more data attributes. ### Claim 4 (depends on 3) The method of claim 3, further comprising receiving a user input to determine whether to filter or to highlight the data marks, and filtering or highlighting the data marks on the displayed data visualizations based on the determination. ### Claim 5 (depends on 2) The method of claim 2, wherein the visualization characteristics include one or more of color, size, and shape. ### Claim 6 (depends on 2) The method of claim 2, wherein the visualization characteristics correspond to a visual encoding of data marks. ### Claim 7 (depends on 6) The method of claim 6, wherein the visual encoding is one or more of color, size, and shape. ### Claim 8 (depends on 2) The method of claim 2, further comprising: determining whether none of the displayed data visualizations contain data marks whose characteristics correspond to the first data attribute; and in accordance with the determination that none of the displayed data visualizations contain data marks whose characteristics correspond to the first data attribute: generating a specification for a new data visualization with the first data attribute; and displaying the new data visualization. ### Claim 9 (depends on 8) The method of claim 8, wherein displaying the new data visualization further comprises: determining a chart type based on the specification; and generating and displaying the chart. ### Claim 10 (depends on 9) The method of claim 9, wherein the chart is positioned using a two-dimensional grid-based layout algorithm, automatically coordinated with other data visualizations. ### Claim 11 (depends on 1) The method of claim 1, further comprising: creating a second set of one or more functional phrases by computing a second set of one or more analytical functions associated with the second set of one or more conversation centers; selecting a first functional phrase from the second set of one or more functional phrases, wherein the first functional phrase comprises a parameterized data selection criterion; selecting an initial range for values of the parameters of the parameterized data selection criterion; displaying an editable user interface control corresponding to the parameterized data selection criterion, wherein the user interface control displays the current values of the parameters; and ordering a displayed set of one or more editable user interface controls based on the order of queries in the second natural language command, wherein the order of queries is inferred while extracting the second set of one or more analytic phrases from the second natural language command. ### Claim 12 (depends on 11) The method of claim 11, further comprising: detecting a user input to edit the user interface control; and in response to detecting the user input, adjusting the first functional phrase according to the user input. ### Claim 13 (depends on 12) The method of claim 12, wherein the user interface control displays a slider, and detecting the user input comprises detecting a sliding of the slider. ### Claim 14 (depends on 11) The method of claim 11, wherein ordering the displayed set of one or more editable user interface controls further comprises performing compact placement of small word-scale visualization within text using a library. ### Claim 15 (depends on 1) The method of claim 1, further comprising: determining a first token in the second natural language command that does not correspond to any of the analytic phrases in the second set of one or more analytic phrases; searching for a correctly spelled term corresponding to the first token using a search library by comparing the first token with one or more features of the first dataset; substituting the correctly spelled term for the first token in the second natural language command to obtain a third natural language command; and extracting the second set of one or more analytic phrases from the third natural language command. ### Claim 16 (depends on 15) The method of claim 15, wherein the one or more features include data attributes, cell values, and related keywords of the first dataset. ### Claim 17 (depends on 15) The method of claim 15, further comprising: determining whether there is no correctly spelled term corresponding to the first token; and in accordance with a determination that there is no correctly spelled term corresponding to the first token: parsing the second natural language command to obtain a parse tree; pruning the parse tree to remove the portion of the tree corresponding to the first token; and extracting the second set of one or more analytic phrases based on the pruned parse tree. ### Claim 18 (depends on 17) The method of claim 17, further comprising generating a textual feedback indicating that the first token was unrecognized and accordingly removed from the second natural language command. ### Claim 19 (depends on 15) The method of claim 15, further comprising displaying the first token. ### Claim 20 (depends on 15) The method of claim 15, further comprising generating a textual feedback indicating that the correctly spelled term is substituted for the first token in the second natural language command. ### Claim 21 (depends on 20) The method of claim 20, further comprising displaying and highlighting the correctly spelled term. ### Claim 22 An electronic device, comprising: a display; one or more hardware processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more hardware processors, the one or more programs including instructions for: displaying a data visualization based on a first dataset retrieved from a database using a first set of one or more queries; receiving a first user input to specify a first natural language command related to the data visualization; extracting a first set of one or more independent analytic phrases from the first natural language command; computing a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytic phrases; creating a first set of one or more functional phrases by computing a first set of analytical functions associated with the first set of one or more conversation centers; updating the data visualization based on the first set of one or more functional phrases; receiving a second user input to specify a second natural language command related to the updated data visualization; extracting a second set of one or more independent analytic phrases from the second natural language command; computing a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytic phrases; deriving a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers using one or more transitional rules; and updating the data visualization based on the second set of one or more conversation centers, wherein the updating comprises: determining one or more data attributes corresponding to the second set of one or more conversation centers; scanning displayed data visualizations to identify one or more of the displayed data visualizations that contain data marks whose characteristics correspond to a first data attribute in the one or more data attributes; and highlighting the data marks whose characteristics correspond to the first data attribute. ### Claim 23 A non-transitory computer readable storage medium storing one or more programs configured for execution by an electronic device with a display, the one or more programs comprising instructions for: displaying a data visualization based on a first dataset retrieved from a database using a first set of one or more queries; receiving a first user input to specify a first natural language command related to the data visualization; extracting a first set of one or more independent analytic phrases from the first natural language command; computing a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytic phrases; creating a first set of one or more functional phrases by computing a first set of analytical functions associated with the first set of one or more conversation centers; updating the data visualization based on the first set of one or more functional phrases; receiving a second user input to specify a second natural language command related to the updated data visualization; extracting a second set of one or more independent analytic phrases from the second natural language command; computing a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytic phrases; deriving a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers using one or more transitional rules; and updating the data visualization based on the second set of one or more conversation centers, wherein the updating comprises: determining one or more data attributes corresponding to the second set of one or more conversation centers; scanning displayed data visualizations to identify one or more of the displayed data visualizations that contain data marks whose characteristics correspond to a first data attribute in the one or more data attributes; and highlighting the data marks whose characteristics correspond to the first data attribute.