Data forwarding using multiple data pipelines
AssigneeSplunk Inc.
InventorsIoannis Vlachogiannis, Panagiotis Papadomitsos
Life of this patent
Estimating the term…
Abstract
In accordance with implementations of the present disclosure, a computer-implemented method for forwarding data includes generating a backup of received live data. The backup of the received live data may be generated on non-volatile storage. The method further includes providing the received live data to a first data buffer for forwarding of the received live data, where a portion of the received live data is dropped prior to being forwarded. Based on the portion of the received live data being dropped, the portion of the received live data is recovered from the backup and the recovered portion of the received live data is provided to a second data buffer for forwarding of the recovered portion of the received live data.
Claims30
A computer-implemented method comprising:
- receiving live data at an input interface of a data forwarder, the data forwarder comprising a real-time data pipeline and a stale data pipeline;
- generating a backup of the received live data at the data forwarder while providing the received live data to a first data buffer of the real-time data pipeline for forwarding of the received live data from an output interface of the data forwarder to a destination node, wherein a portion of the received live data is dropped prior to the forwarding;
- and
- based on the portion of the received live data being dropped, recovering the portion of the received live data from the backup and providing the recovered portion of the received live data to a second data buffer of the stale data pipeline, the second data buffer for forwarding of the recovered portion of the received live data from the output interface to the destination node.
A system for forwarding data, the system comprising:
- one or more data processors;
- and
- one or more computer-readable storage media containing instructions which when executed on the one or more data processors, cause the one or more processors to perform operations including:
- receiving live data at an input interface of a data forwarder, the data forwarder comprising a real-time data pipeline and a stale data pipeline;
- generating a backup of the received live data at the data forwarder while providing the received live data to a first data buffer of the real-time data pipeline for forwarding of the received live data from an output interface of the data forwarder to a destination node, wherein a portion of the received live data is dropped prior to the forwarding;
- and
- based on the portion of the received live data being dropped, recovering the portion of the received live data from the backup and providing the recovered portion of the received live data to a second data buffer of the stale data pipeline, the second data buffer for forwarding of the recovered portion of the received live data from the output interface to the destination node.
One or more non-transitory computer-storage media storing computer-useable instructions that, when executed by a computing device, perform a method for forwarding data, the method comprising:
- receiving live data at an input interface of a data forwarder, the data forwarder comprising a real-time data pipeline and a stale data pipeline;
- generating a backup of the received live data at the data forwarder while providing the received live data to a first data buffer of the real-time data pipeline for forwarding of the received live data from an output interface of the data forwarder to a destination node, wherein a portion of the received live data is dropped prior to the forwarding;
- and
- based on the portion of the received live data being dropped, recovering the portion of the received live data from the backup and providing the recovered portion of the received live data to a second data buffer of the stale data pipeline, the second data buffer for forwarding of the recovered portion of the received live data from the output interface to the destination node.
Dependencies are inferred from the claim text.