Transforming a data stream into structured data

A system for identifying a structure of a data stream, the system comprising:

• at least one processor;
• and
• at least one storage device storing instructions that are operable, when executed by the at least one processor, to cause the at least one processor to perform operations comprising:
• obtaining a sequence of tokens representing a data entry of the data stream, each token of the sequence of tokens being assigned an attribute value describing a respective portion of the data entry represented by that token, the attribute value being associated with at least one merging rule specifying other tokens that are mergeable with that token;
• assigning the sequence of tokens to a cluster, the cluster including a given sequence of tokens that represent a structure for a given data entry, wherein assigning the sequence of tokens to the cluster comprises:
• identifying a sequence of attribute values of the given sequence of tokens;
• determining that each attribute value, of the sequence of attribute values of the given sequence of tokens, satisfies a respective merging rule associated with a corresponding attribute value of a corresponding token of the sequence of tokens;
• in response to determining that each attribute value satisfies the respective merging rule, assigning the sequence of tokens to the cluster;
• and
• outputting a representation of the cluster comprising the given sequence of tokens that represent the structure for the data entry of the data stream.

One or more non-transitory computer readable media storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

• obtaining a sequence of tokens representing a data entry of a data stream, each token of the sequence of tokens being assigned an attribute value describing a respective portion of the data entry represented by that token, the attribute value being associated with at least one merging rule specifying other tokens that are mergeable with that token;
• assigning the sequence of tokens to a cluster, the cluster including a given sequence of tokens that represent a structure for a given data entry, wherein assigning the sequence of tokens to the cluster comprises:
• identifying a sequence of attribute values of the given sequence of tokens;
• determining that each attribute value, of the sequence of attribute values of the given sequence of tokens, satisfies a respective merging rule associated with a corresponding attribute value of a corresponding token of the sequence of tokens;
• in response to determining that each attribute value satisfies the respective merging rule, assigning the sequence of tokens to the cluster;
• and
• outputting a representation of the cluster comprising the given sequence of tokens that represent the structure for the data entry of the data stream.