US 2019/0102620 A1 · SYSTEMS AND METHODS FOR DETECTING SEMANTICS OF COLUMNS FROM TABULAR DATA

Abstract

Systems and methods for detecting the semantics for columns in tabular data automatically in electronic documents. For example, a media guidance application may receive an electronic document that may contain a table relating to television program schedules. The media guidance application may then detect data rows that form the records of the table, and determine the column type for every column in the table to extract television program schedule information.

Claims22

Dependent claims

(canceled)

A method for automatically detecting semantics of columns in a data table to determine a data concept that each column represents, the method comprising:

receiving an electronic document containing a grid;
determining whether the grid contains tabular data;
in response to determining that the grid contains tabular data, identifying a plurality of rows and a plurality of columns from the tabular data;
determining whether a portion of the plurality of rows that share a same data type exceeds a pre-defined percentage;
in response to determining that the portion of the plurality of rows that share the same data type exceeds the pre-defined percentage, determining whether the plurality of rows contains a header row by comparing a first row of the grid and other rows of the grid;
in response to determining that the first row of the grid has a different data type from the other rows of the grid, identifying the first row of the grid as the header row;
in response to determining that the plurality of rows contains the header row:
determining whether a respective data value in the header row corresponding to a respective column indicates a semantics type of the respective column,
wherein the semantics type represents a data concept that data values within the respective column correspond to;
in response to determining that the respective data value corresponding to the respective column in the header row indicates the semantics type of the respective column, identifying a pre-stored data format corresponding to the semantics type;
and
in response to determining that the plurality of rows does not contain a header row, or the respective data value in the header row corresponding to the respective column does not indicate the semantics type of the respective column:
querying a semantics table to identify one or more semantics types based at least in part on a data type corresponding to at least a subset of the plurality of data values of other data rows;
refining the one or more semantics types based on a context of the tabular data to determine a semantics type for the respective column;
and
storing data values corresponding to each respective column in the data format compliant with the semantics type.

13 more elements

A system for automatically detecting semantics of columns in a data table to determine a data concept that each column represents, the system comprising:

memory;
communication circuitry;
and
control circuitry configured to:
receive, via the communication circuitry, an electronic document containing a grid;
determine whether the grid contains tabular data;
in response to determining that the grid contains tabular data, identify a plurality of rows and a plurality of columns from the tabular data;
determine whether a portion of the plurality of rows that share a same data type exceeds a pre-defined percentage;
in response to determining that the portion of the plurality of rows that share the same data type exceeds the pre-defined percentage, determine whether the plurality of rows contains a header row by comparing a first row of the grid and other rows of the grid;
in response to determining that the first row of the grid has a different data type from the other rows of the grid, identify the first row of the grid as the header row;
in response to determining that the plurality of rows contains the header row:
determine whether a respective data value in the header row corresponding to a respective column indicates a semantics type of the respective column,
wherein the semantics type represents a data concept that data values within the respective column correspond to;
in response to determining that the respective data value corresponding to the respective column in the header row indicates the semantics type of the respective column, identify a pre-stored data format corresponding to the semantics type;
and
in response to determining that the plurality of rows does not contain a header row, or the respective data value in the header row corresponding to the respective column does not indicate the semantics type of the respective column:
query a semantics table to identify one or more semantics types based at least in part on a data type corresponding to at least a subset of the plurality of data values of other data rows;
refine the one or more semantics types based on a context of the tabular data to determine a semantics type for the respective column;
and
store, at the memory, data values corresponding to each respective column in the data format compliant with the semantics type.

17 more elements

Dependencies are inferred from the claim text.

Abstract

Claims22

Dependent claims

(canceled)

A method for automatically detecting semantics of columns in a data table to determine a data concept that each column represents, the method comprising:

receiving an electronic document containing a grid;
determining whether the grid contains tabular data;
in response to determining that the grid contains tabular data, identifying a plurality of rows and a plurality of columns from the tabular data;
determining whether a portion of the plurality of rows that share a same data type exceeds a pre-defined percentage;
in response to determining that the portion of the plurality of rows that share the same data type exceeds the pre-defined percentage, determining whether the plurality of rows contains a header row by comparing a first row of the grid and other rows of the grid;
in response to determining that the first row of the grid has a different data type from the other rows of the grid, identifying the first row of the grid as the header row;
in response to determining that the plurality of rows contains the header row:
determining whether a respective data value in the header row corresponding to a respective column indicates a semantics type of the respective column,
wherein the semantics type represents a data concept that data values within the respective column correspond to;
in response to determining that the respective data value corresponding to the respective column in the header row indicates the semantics type of the respective column, identifying a pre-stored data format corresponding to the semantics type;
and
in response to determining that the plurality of rows does not contain a header row, or the respective data value in the header row corresponding to the respective column does not indicate the semantics type of the respective column:
querying a semantics table to identify one or more semantics types based at least in part on a data type corresponding to at least a subset of the plurality of data values of other data rows;
refining the one or more semantics types based on a context of the tabular data to determine a semantics type for the respective column;
and
storing data values corresponding to each respective column in the data format compliant with the semantics type.

13 more elements

A system for automatically detecting semantics of columns in a data table to determine a data concept that each column represents, the system comprising:

memory;
communication circuitry;
and
control circuitry configured to:
receive, via the communication circuitry, an electronic document containing a grid;
determine whether the grid contains tabular data;
in response to determining that the grid contains tabular data, identify a plurality of rows and a plurality of columns from the tabular data;
determine whether a portion of the plurality of rows that share a same data type exceeds a pre-defined percentage;
in response to determining that the portion of the plurality of rows that share the same data type exceeds the pre-defined percentage, determine whether the plurality of rows contains a header row by comparing a first row of the grid and other rows of the grid;
in response to determining that the first row of the grid has a different data type from the other rows of the grid, identify the first row of the grid as the header row;
in response to determining that the plurality of rows contains the header row:
determine whether a respective data value in the header row corresponding to a respective column indicates a semantics type of the respective column,
wherein the semantics type represents a data concept that data values within the respective column correspond to;
in response to determining that the respective data value corresponding to the respective column in the header row indicates the semantics type of the respective column, identify a pre-stored data format corresponding to the semantics type;
and
in response to determining that the plurality of rows does not contain a header row, or the respective data value in the header row corresponding to the respective column does not indicate the semantics type of the respective column:
query a semantics table to identify one or more semantics types based at least in part on a data type corresponding to at least a subset of the plurality of data values of other data rows;
refine the one or more semantics types based on a context of the tabular data to determine a semantics type for the respective column;
and
store, at the memory, data values corresponding to each respective column in the data format compliant with the semantics type.

17 more elements

Dependencies are inferred from the claim text.

Systems and methods for detecting semantics of columns from tabular data

Life of this patent

Abstract

Claims22

Systems and methods for detecting semantics of columns from tabular data

Life of this patent

Abstract

Claims22