# US 2014/0207792 A1 · Automatically generating regular expressions for data field extractions with natural language editing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0207792 A1
- Document ID: US20140207792A1
- Status: Published application
- Kind: A1
- Application number: 13/748,306
- Filed: 2013-01-23
- Published: 2014-07-24
- Earliest claimed: 2013-01-23
- Assignees: Splunk Inc.
- Applicants: Splunk Inc.
- Inventors: R. David Carasso; Micah James Delfino; Johnvey Hwang
- Web page: https://patentgrep.com/patent/US20140207792A1

## CPC

- G06F 40/211: Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments are directed towards automatically generating extraction rules for extracting fields from event records. An extraction rule application receives field data describing the fields to be extracted (including one or more examples) and a collection of event records that may be a representative sample set from a larger set of events records. The extraction rule application generates extraction rules based on the event records and the field data. These extraction rules may be ranked using a determined quality score. Quality scores for extraction rules may be determined based on various metrics related to the operation of the extraction rules and the resultant extracted values. Preferred extraction rules may be determined by ranking the extraction rules based on their quality scores. Also, natural language expressions may be used to create, edit, or modify extraction rules.

## Claims

### Claim 1

A computer-implemented method comprising:
gathering, using a computing device, a stream of data;
transforming the stream of data into a plurality of events, wherein each event includes a portion of the stream of data;
associating a time stamp with each event of the plurality of events;
storing the plurality of events and their associated time stamps;
displaying a first event of the plurality of events;
receiving a selection of a portion of text within the first event;
determining a field extraction rule that extracts as a value of a field the selection of the portion of text within the first event when the field extraction rule is applied to the first event;
displaying a second event of the plurality of events; and
indicating, for the second event, a value of the field for the second event that would be extracted by applying the extraction rule to the second event.

### Claim at position 2

2-8. (canceled)

### Claim 9

A network device that is operative for generating extraction rules, comprising:
a transceiver that is operative to communicate over a network;
a memory that is operative to store at least instructions; and
a processor device that is operative to execute instructions that enable actions, including:
gathering a stream of data;
transforming the stream of data into a plurality of events, wherein each event includes a portion of the stream of data;
associating a time stamp with each event of the plurality of events;
storing the plurality of events and their associated time stamps;
displaying a first event of the plurality of events;
receiving a selection of a portion of text within the first event;
determining a field extraction rule that extracts as a value of a field the selection of the portion of text within the first event when the field extraction rule is applied to the first event;
displaying a second event of the plurality of events; and
indicating, for the second event, a value of the field for the second event that would be extracted by applying the extraction rule to the second event.

### Claim at position 4

10-16. (canceled)

### Claim 17

A processor readable non-transitive storage media that includes instructions for generating extraction rules over a network, wherein execution of the instructions by a processor device enables actions, comprising:
gathering a stream of data;
transforming the stream of data into a plurality of events, wherein each event includes a portion of the stream of data;
associating a time stamp with each event of the plurality of events;
storing the plurality of events and their associated time stamps;
displaying a first event of the plurality of events;
receiving a selection of a portion of text within the first event;
determining a field extraction rule that extracts as a value of a field the selection of the portion of text within the first event when the field extraction rule is applied to the first event;
displaying a second event of the plurality of events; and
indicating, for the second event, a value of the field for the second event that would be extracted by applying the extraction rule to the second event.

### Claim at position 6

18-23. (canceled)

### Claim 24

A system that is arranged for generating extraction rules over a network, comprising:
a server device, including:
a transceiver that is operative to communicate over the network;
a memory that is operative to store at least instructions; and
a processor device that is operative to execute instructions that enable actions, including:
gathering, using a computing device, a stream of data;
transforming the stream of data into a plurality of events, wherein each event includes a portion of the stream of data;
associating a time stamp with each event of the plurality of events;
storing the plurality of events and their associated time stamps;
displaying a first event of the plurality of events;
receiving field data including a selection of a portion of text within the first event;
determining a field extraction rule that extracts as a value of a field the selection of the portion of text within the first event when the field extraction rule is applied to the first event data;
displaying at least a second event of the plurality of events; and
indicating, for the second event, a value of the field for the second event that would be extracted by applying the extraction rule to the second event.

### Claim at position 8

25-30. (canceled)

### Claim 31 (depends on 1)

The method of claim 1, wherein the first event includes machine data.

### Claim 32 (depends on 1)

The method of claim 1, wherein the first event includes unstructured data.

### Claim 33 (depends on 1)

The method of claim 1, wherein storing the plurality of events includes indexing each event of the plurality of events.

### Claim 34 (depends on 1)

The method of claim 1, further comprising:
receiving an indication that a desired value of the field for the second event does not comprise the indicated value of the field for the second event;
modifying the field extraction rule such that the modified field extraction rule would extract the selection of the portion of text when the field extraction rule is applied to the first event and would extract a value of the field for the second event that is different from the indicated value of the field for the second event when the field extraction rule is applied to the second event

### Claim 35 (depends on 1)

The method of claim 1, further comprising:
receiving a selection of a portion of text within the second event;
modifying the field extraction rule such that the modified field extraction rule would extract the selection of the portion of text within the first event when the field extraction rule is applied to the first event and would extract the selection of the portion of text within the second event when the field extraction rule is applied to the second event.

### Claim 36 (depends on 1)

The method of claim 1, wherein the field extraction rule comprises a regular expression.

### Claim 37 (depends on 1)

The method of claim 1, further comprising:
displaying natural language representing the field extraction rule;
receiving an edit to the natural language;
determining a modified field extraction rule corresponding to the edited natural language; and
determining values for the field corresponding to the modified field extraction rule.

### Claim 38 (depends on 1)

The method of claim 1, further comprising determining a data type of the selection of the portion of text within the first event, and wherein determining the field extraction rule that extracts as the value of the field the selection of the portion of text within the first event comprises determining that the field extraction rule would extract a value comprising the data type as the value of the field for at least one other event.

### Claim 39 (depends on 9)

The network device of claim 9, wherein the first event includes machine data.

### Claim 40 (depends on 9)

The network device of claim 9, wherein the first event includes unstructured data.

### Claim 41 (depends on 9)

The network device of claim 9, wherein storing the plurality of events includes indexing each event of the plurality of events.

### Claim 42 (depends on 9)

The network device of claim 9, wherein the actions further comprise:
receiving an indication that a desired value of the field for the second event does not comprise the indicated value of the field for the second event;
modifying the field extraction rule such that the modified field extraction rule would extract the selection of the portion of text when the field extraction rule is applied to the first event and would extract a value of the field for the second event that is different from the indicated value of the field for the second event when the field extraction rule is applied to the second event

### Claim 43 (depends on 9)

The network device of claim 9, wherein the actions further comprise:
receiving a selection of a portion of text within the second event;
modifying the field extraction rule such that the modified field extraction rule would extract the selection of the portion of text within the first event when the field extraction rule is applied to the first event and would extract the selection of the portion of text within the second event when the field extraction rule is applied to the second event.

### Claim 44 (depends on 9)

The network device of claim 9, wherein the field extraction rule comprises a regular expression.

### Claim 45 (depends on 9)

The network device of claim 9, wherein the actions further comprise:
displaying natural language representing the field extraction rule; receiving an edit to the natural language;
determining a modified field extraction rule corresponding to the edited natural language; and
determining values for the field corresponding to the modified field extraction rule.

### Claim 46 (depends on 9)

The network device of claim 9, wherein the actions further comprise determining a data type of the selection of the portion of text within the first event, and wherein determining the field extraction rule that extracts as the value of the field the selection of the portion of text within the first event comprises determining that the field extraction rule would extract a value comprising the data type as the value of the field for at least one other event.

### Claim 47 (depends on 17)

The media of claim 17, wherein the first event includes machine data.

### Claim 48 (depends on 17)

The media of claim 17, wherein the first event includes unstructured data.

### Claim 49 (depends on 17)

The media of claim 17, wherein storing the plurality of events includes indexing each event of the plurality of events.

### Claim 50 (depends on 17)

The media of claim 17, wherein the actions further comprise:
receiving an indication that a desired value of the field for the second event does not comprise the indicated value of the field for the second event;
modifying the field extraction rule such that the modified field extraction rule would extract the selection of the portion of text when the field extraction rule is applied to the first event and would extract a value of the field for the second event that is different from the indicated value of the field for the second event when the field extraction rule is applied to the second event

### Claim 51 (depends on 17)

The media of claim 17, wherein the actions further comprise:
receiving a selection of a portion of text within the second event;
modifying the field extraction rule such that the modified field extraction rule would extract the selection of the portion of text within the first event when the field extraction rule is applied to the first event and would extract the selection of the portion of text within the second event when the field extraction rule is applied to the second event.

### Claim 52 (depends on 17)

The media of claim 17, wherein the field extraction rule comprises a regular expression.

### Claim 53 (depends on 17)

The media of claim 17, wherein the actions further comprise:
displaying natural language representing the field extraction rule; receiving an edit to the natural language;
determining a modified field extraction rule corresponding to the edited natural language; and
determining values for the field corresponding to the modified field extraction rule.

### Claim 54 (depends on 17)

The media of claim 17, wherein the actions further comprise determining a data type of the selection of the portion of text within the first event, and wherein determining the field extraction rule that extracts as the value of the field the selection of the portion of text within the first event comprises determining that the field extraction rule would extract a value comprising the data type as the value of the field for at least one other event.

### Claim 55 (depends on 24)

The system of claim 24, wherein the first event includes machine data.

### Claim 56

(canceled)

### Claim 57 (depends on 1)

The method of claim 1, further comprising:
receiving an indication that a value is to serve as a counter example for the field; and
modifying the field extraction rule to bias against identifying the counter-example value as a value for the field.
