US 12,754,931 B1Granted

Mirrored lookup table at stream data processing and batch search systems

AssigneeCisco Technology, Inc.

InventorsRicky Burnett, Ryan Min, Zhen Peng Zhu

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods are described for generating, using, and modifying a mirrored lookup table by a stream data processing system. On receipt of a request relating to a pipeline for streaming data processing, a stream data processing system may access a lookup table of a batch search system including a set of key-value pairs. Based on the lookup table, the stream data processing system may generate a mirrored lookup table at the stream data processing system by at least replicating the set of key-value pairs. The stream data processing system may use the mirror table in performing lookup operations to enrich the streaming data input. The output of the lookup operations or the output of the pipeline may be provided to a destination, such as the batch search system.

Claims20

Dependent claims

A computer-implemented method implementing at a stream data processing system comprising at least one processor, the computer-implemented method comprising:

accessing a lookup table associated with a batch search system that performs searches, wherein the lookup table includes a set of key-value pairs, each key-value pair associating a specific key with a corresponding value;
generating a mirror of the lookup table at the stream data processing system that manipulates data and writes the data to an output data stream, the mirror replicating the set of key-value pairs of the lookup table from the batch search system that performs the searches to the stream data processing system that manipulates data and writes the data to the output data stream;
accessing a streaming data input with the stream data processing system, wherein the streaming data input includes a data item comprising one or more fields;
modifying the one or more fields of the streaming data input according to the mirror of the lookup table stored at the stream data processing system to result in a modified data item, wherein modifying the one or more fields comprises:
determining that the one or more fields include a key of a key-value pair in the set of key-value pairs,
identifying, via a lookup operation performed using the mirror of the lookup table, a value corresponding with the key, and
modifying the one or more fields to include the corresponding value for the key;
and
transmitting the data item to a destination.

6 more elements

A system comprising:

one or more processors configured to:
access a lookup table associated with a batch search system that performs searches, wherein the lookup table includes a set of key-value pairs, each key-value pair associating a specific key with a corresponding value;
generate a mirror of the lookup table at the stream data processing system that manipulates data and writes the data to an output data stream, the mirror replicating the set of key-value pairs of the lookup table from the batch search system that performs the searches to the stream data processing system that manipulates data and writes the data to the output data stream;
access a streaming data input with the stream data processing system, wherein the streaming data input includes a data item comprising one or more fields;
modify the one or more fields of the streaming data input according to the mirror of the lookup table stored at the stream data processing system to result in a modified data item, wherein modifying the one or more fields comprises;
determining that the one or more fields include a key of a key-value pair in the set of key-value pairs,
identifying, via a lookup operation performed using the mirror of the lookup table, a value corresponding with the key, and
modifying the one or more fields to include the corresponding value for the key;
and
transmit the data item to a destination.

7 more elements

One or more non-transitory computer-readable media including computer-executable instructions that, when executed by a computing system, cause the computing system to:

access a lookup table associated with a batch search system that performs searches, wherein the lookup table includes a set of key-value pairs, each key-value pair associating a specific key with a corresponding value;
generate a mirror of the lookup table at the stream data processing system that manipulates data and writes the data to an output data stream, the mirror replicating the set of key-value pairs of the lookup table from the batch search system that performs the searches to the stream data processing system that manipulates data and writes the data to the output data stream;
access a streaming data input with the stream data processing system, wherein the streaming data input includes a data item comprising one or more fields;
modify the one or more fields of the streaming data input according to the mirror of the lookup table stored at the stream data processing system to result in a modified data item, wherein modifying the one or more fields comprises:
determining that the one or more fields include a key of a key-value pair in the set of key-value pairs,
identifying, via a lookup operation performed using the mirror of the lookup table, a value corresponding with the key, and
modifying the one or more fields to include the corresponding value for the key;
and
transmit the data item to a destination.

6 more elements

Dependencies are inferred from the claim text.