US 11,775,756 B2Granted

Automated caption generation from a dataset

AssigneeAdobe Inc.

InventorsEunyee Koh, Xin Qian, Tak Yeon Lee, Sana Malik Lee, Ryan Anthony Rossi, Fan Du, Duy-Trung Trong Dinh

Life of this patent

Estimating the term…

Loading patent…

Abstract

A dataset captioning system is described that generates captions of text to describe insights identified from a dataset, automatically and without user intervention. To do so, given an input of a dataset the dataset captioning system determines which data insights are likely to support potential visualizations of the dataset, generates text based on these insights, orders the text, processes the ordered text for readability, and then outputs the text as a caption. These techniques also include adjustments made to the complexity of the text, globalization of the text, inclusion of links to outside sources of information, translation of the text, and so on as part of generating the caption.

Claims20

Dependent claims

A method comprising:

generating, by a processing device automatically and without user intervention, a caption that textually describes a dataset having a plurality of data entries organized as a plurality of data subsets, the generating including:
determining which datatypes are included in the plurality of data subsets, respectively;
identifying a composition including a visualization of the dataset from a plurality of compositions based on the datatypes and a set of pre-defined heuristics that detect whether the data subsets include an outlier;
determining which data insights correspond to the composition by detecting cyclic patterns in the dataset and determining which cyclic patterns are statistically significant by comparing a correlation coefficient associated with a data insight to a threshold significance value;
detecting semantic datatypes in the data subsets from a pre-defined taxonomy;
generating text, based on the data insights that correspond to the composition and based on the semantic datatypes, from the plurality of data entries of the dataset;
editing the text for readability based on a relationship between the data insights;
and
forming the caption based at least in part on the edited text.

6 more elements

A system comprising:

a dataset input module implemented at least partially in hardware of a processing device to receive a dataset having a plurality of data entries and identify a composition including a visualization of the dataset from a plurality of compositions based on datatypes of the composition and a set of pre-defined heuristics that detect whether the plurality of data entries include an outlier;
a text generation module implemented at least partially in hardware of the processing device to generate text based on a plurality of data insights from the plurality of data entries of the dataset by determining which data insights correspond to the composition by detecting cyclic patterns in the dataset and determining which cyclic patterns are statistically significant by comparing a correlation coefficient associated with a data insight to a threshold significance value;
detecting semantic datatypes in the dataset from a pre-defined taxonomy;
and
a caption formation module implemented at least partially in hardware of the processing device to generate a caption including text based on the data insights that correspond to the composition and based on the semantic datatypes, the caption formation module including a complexity adjustment module configured to adjust language complexity of the text as part of the caption.

2 more elements

Dependencies are inferred from the claim text.