Back to results
US 7,599,952 B2Granted
System and method for parsing unstructured data into structured data
AssigneeMicrosoft Corporation
InventorDavid J. Parkinson
Life of this patent
Estimating the term…
Abstract
The present invention uses a machine-learned statistical model to map between unstructured data and structured data. By using machine learning techniques, the present parsing engine can be very quickly and easily trained for new and different locations or domains and can also accommodate for inputs which are unseen in the training data.
Claims17
A method of convening an unstructured input into structured data for use with an application, the method comprising:
- applying, using a processor of a computer, a statistical parse to the unstructured input to statistically map portions of the unstructured input to portions of a predefined data structure, wherein the predefined data structure has slots to be filled by specific portions of the unstructured input, and wherein applying a statistical parse to statistically map portions of the unstructured input comprises:
- breaking the unstructured input into a plurality of tokens;
- identifying features of interest in the plurality of tokens;
- and
- for each token of the plurality of tokens,
- generating a feature vector associated with the token, the feature vector being indicative of the features in the token;
- and
- generating a score associated with each slot, based on the feature vector associated with the token, the score being indicative of whether the token should fill the associated slot in the predefined data structure, wherein scores are generated for the token for each slot in the predefined data structure;
- and
- outputting a relationship between the unstructured input and the predefined data structure based on the statistical parse, wherein outputting comprises filling slots of the predefined data structure with portions of the unstructured input based on the scores.
7 more elements
A computer-implemented system for generating structured data from an unstructured data input, the system comprising:
- a processor;
- a statistical parsing engine that receives the unstructured data input and, using the processor, identifies portions of the unstructured data input that correspond to portions of a predefined data structure having slots to be filled by specific portions of the unstructured data input;
- wherein the statistical parsing engine includes a tokenization component that receives the unstructured data input from an application and generates tokens based on the unstructured data input;
- wherein the statistical parsing engine identifies features in the tokens and generates a set of scores corresponding to each token, one score in the set of scores corresponding to each slot in the predefined data structure;
- and
- wherein the statistical parsing engine statistically maps the portions of the unstructured data input to portions in the predefined data structure based on the scores, wherein the statistical parsing engine generates a structured data output, based on the statistical mapping, by filling the portions of the predefined data structure with the portions of the unstructured data input.
3 more elements
A computer readable medium storing instructions which, when executed by a computer, cause the computer to perform a method comprising:
- receiving an unstructured data input from an application;
- applying a machine learned, statistical parse to the unstructured data input, using a processor of the computer, to generate a structured data output by populating a predefined data schema with portions of the unstructured data input based on the parse, wherein the predefined data schema has slots to be filled by specific portions of the unstructured data input and populating the predefined data schema comprises filling the slots in the predefined data schema with the portions of the unstructured data, wherein applying the statistical parse comprises:
- breaking the unstructured data input into tokens;
- identifying features of interest in the tokens;
- and
- generating a score for each of the slots in the predefined data schema, based on the features of interest in a selected token, the score being indicative of whether the selected token should fill the slot in the predefined data structure.
3 more elements
Dependencies are inferred from the claim text.