Question answering for data visualizations
AssigneeAdobe Inc.
InventorsScott Cohen, Kushal Kafle, Brian Price
Life of this patent
Estimating the term…
Abstract
Systems and techniques are described that provide for question answering using data visualizations, such as bar graphs. Such data visualizations are often generated from collected data, and provided within image files that illustrate the underlying data and relationships between data elements. The described techniques analyze a query and a related data visualization, and identify one or more spatial regions within the data visualization in which an answer to the query may be found.
Claims20
A computer program product, the computer program product being tangibly embodied on a non-transitory computer-readable storage medium and comprising instructions that, when executed by at least one computing device, are configured to cause the at least one computing device to:
- identify a data visualization (DV);
- generate a DV feature map characterizing the data visualization, including maintaining a correspondence of spatial relationships of mapped features of the data visualization within the DV feature map with corresponding features of the data visualization;
- identify a query for an answer included within the data visualization;
- encode the query as a query feature vector;
- generate a prediction of at least one answer location within the data visualization, based on the DV feature map and the query feature vector;
- and
- determine the answer from the at least one answer location.
A computer-implemented method, the method comprising:
- receiving a query for a data visualization (DV);
- encoding the query as a query feature vector;
- generating a DV feature map characterizing at least one spatial region of the data visualization;
- generating a prediction of at least one answer location within the at least one spatial region, based on a combination of the at least one spatial region of the data visualization and the query feature vector;
- and
- determining an answer to the query from the at least one answer location.
A system comprising:
- at least one memory including instructions;
- and
- at least one processor that is operably coupled to the at least one memory and that is arranged and configured to execute instructions that, when executed, cause the at least one processor to:
- generate a visualization training dataset that includes a plurality of training data visualizations and visualization parameters, and query/answer pairs for the plurality of training data visualizations;
- train a feature map generator to generate a feature map of each of the training data visualizations;
- train a query feature vector generator to generate a query feature vector of each of the queries of the query/answer pairs;
- train an answer location generator to generate an answer location within each of the training data visualizations of an answer for a corresponding query of the query/answer pairs, based on outputs of the trained feature map generator and the trained query feature vector;
- input a new data visualization and a new query into the trained feature map generator and the trained query feature vector to obtain a new feature map and a new query feature vector;
- and
- generate a new answer location within the new data visualization for the new query, based on the new feature map and the new query feature vector.
Dependencies are inferred from the claim text.