Data discovery solution for data curation
AssigneePraxi Data, Inc.
InventorAndy Tae Ahn
Life of this patent
Estimating the term…
Abstract
Disclosed are methods and systems for a data discovery solution which harnesses the power of crowdsourcing to improve automated data curation. This is done in two complimentary ways: (a) large scale collective curation through anonymized multi-tenancy, and (b) and through internet scale matching and validation gaming platform using mobile application game. The result is the most extensive library of semantic-technical mappings of the enterprise data, which are immediately at hand to provide a fast, easy and a good understanding of the enterprise data. The data discovery solution forms a gateway for governing and unlocking value from big data.
Claims24
A method of generating a set of candidate patterns that train a model comprising:
- receiving training input, including:
- transmitting, via a matching application, a first matching question to a first player of a plurality of players, wherein the first matching question is configured to solicit a response that matches a first data element of a dataset to a first tag;
- receiving, via the matching application, a first player response that matches the first data element to the first;
- and
- storing a first pattern to a database comprising a corpus of patterns, the first pattern corresponding to the first player response;
- validating the first pattern within the corpus of patterns, including:
- transmitting, via the matching application, a second matching question to the first player, wherein the second matching question comprises a validator question corresponding to at least one past answer;
- receiving, via the matching application, a second player response to the second matching question;
- and
- generating a reliability rating for the first player based on consistency of the second player response to the second matching question with the at least one past answer;
- weighting the first pattern in the corpus of patterns based on the reliability rating of the first player;
- and
- training an artificial intelligence model using the first pattern weighted based on the reliability rating of the first player.
A method of validating a plurality of unvalidated patterns using a crowdsource-based application, wherein the plurality of unvalidated patterns associate a plurality of tags to a plurality of data elements from one or more data sources comprising:
- receiving training input, including:
- transmitting, via the crowdsource-based application, a first matching question to a first player of a plurality of players, wherein the first matching question is configured to solicit a first player response that matches a first data element to a first tag;
- receiving, via the crowdsource-based application, the first player response;
- and
- storing a first pattern to a database comprising a corpus of patterns, the first pattern corresponding to the first player response;
- validating the first pattern within the corpus of patterns, including:
- transmitting, via the crowdsource-based application, a second matching question to the first player, wherein the second matching question comprises a validator question corresponding to a plurality of past answers from a corresponding plurality of other players, wherein each past answer of the plurality of past answers includes a corresponding reliability score associated with a respective one of the plurality of other players;
- receiving, via the crowdsource-based application, a second player response to the second matching question from the first player;
- matching the second player response to one or more past answers of the plurality of past answers;
- and
- generating a reliability rating for the first player based on the one or more reliability scores corresponding to the one or more past answers matching the second player response;
- weighting the first pattern in the corpus of patterns based on the reliability rating of the first player;
- training an artificial intelligence model using the first pattern weighted based on the reliability rating of the first player;
- and
- updating the weighting of the first pattern after the reliability rating of the first player changes.
A system of generating a set of candidate patterns comprising:
- a processor;
- and
- a memory including an artificial intelligence model that is trained with a corpus of patterns, the memory further including instructions that when executed, cause the processor to:
- receive training input, including:
- transmitting, via a matching application, a first matching question to a first player of a plurality of players, wherein the first matching question is configured to solicit a response that matches a first data element of a dataset to a first tag;
- receiving, via the matching application, a first player response that matches the first data element to the first tag;
- and
- storing a first pattern to a database comprising a corpus of patterns, the first pattern corresponding to the first player response;
- validate the first pattern within the corpus of patterns, including:
- transmitting, via the matching application, a second matching question to the first player, wherein the second matching question comprises a validator question corresponding to at least one past answer;
- receiving, via the matching application, a second player response to the second matching question;
- and
- generating a reliability rating for the first player based on consistency of the second player response to the second matching question with the at least one past answer;
- weight the first pattern in the corpus of patterns based on the reliability rating of the first player;
- and
- train the artificial intelligence model using the first pattern weighted based on the reliability rating of the first player.
Dependencies are inferred from the claim text.