Back to results
US 12,079,173 B2Granted
Segmented streaming data processing
AssigneeFluency Corp.
InventorsChristopher Jordan, Kun Luo
Life of this patent
Estimating the term…
Abstract
The disclosure relates generally to a method, device, and system for creating a searchable data store with an infinite retention period for streaming documents. In an embodiment, a method of storing data includes receiving an incoming stream of time-series data; storing the time-series data of the incoming data stream in a first segment; and closing the first segment in response to a resource consumption limit of the first segment being reached.
Claims20
A method of storing data, the method comprising:
- receiving, by a first node, an incoming data stream of time-series data, the first node hosting a first shard, the first shard having only two active partitions at any one time, the two active partitions of the first shard being a first normal partition and a first overflow partition;
- receiving, by a second node, the incoming data stream of time-series data, the second node hosting a second shard, the second shard having only two active partitions at any one time, the two active partitions being a second normal partition and a second overflow partition;
- storing, by the first shard, the time-series data of the incoming data stream in a first segment, the first segment being a computer-readable medium;
- storing, by the second shard, the time-series data of the incoming data stream in the first segment;
- rotating to a second segment, a third shard, and a fourth shard in response to a resource consumption limit of the first segment being reached, the first segment, the first shard, and the second shard each being closed after the rotating;
- and
- storing, by the third shard and the fourth shard, the time-series data of the incoming data stream in the second segment, the second segment being a computer-readable medium.
4 more elements
A method of storing data, the method comprising:
- receiving an incoming stream of data, by a first virtual node and a second virtual node, the incoming stream of data comprising a plurality of events, the first and second virtual nodes being virtual nodes on cloud hardware;
- storing, by the first and second virtual nodes, a supplemental timestamp with each event of the plurality of events when the respective event is detected without an original timestamp, wherein the supplemental timestamp is based on a time at which the respective event is received;
- storing, by the first and second virtual nodes, the events of the plurality of events in a first segment, the first segment being stored in a computer-readable data store;
- in response to a resource consumption limit of the first segment being reached, receiving the incoming stream of data, by a third virtual node and a fourth virtual node, the third and fourth virtual nodes being virtual nodes on cloud hardware;
- and
- storing, by the third and fourth virtual nodes, the events of the plurality of events in a second segment, the second segment being stored in a computer-readable data store.
3 more elements
A method of storing data, the method comprising:
- receiving, by a first plurality of nodes, an incoming stream of time-series data;
- storing the time-series data of the incoming data stream in a first segment, the first segment being stored in a computer-readable medium, wherein a first plurality of shards store the time-series data of the incoming data stream in the first segment, wherein each of the shards of the first plurality of shards operates in parallel, and wherein each of the shards of the first plurality of shards is hosted on a different node of the first plurality of nodes;
- in response to a resource consumption limit of the first segment being reached, receiving, by a second plurality of nodes, the incoming stream of time-series data;
- and
- storing the time-series data of the incoming data stream in a second segment, the second segment being stored in a computer-readable medium, wherein a second plurality of shards store the time-series data of the incoming data stream in the first segment, wherein each of the shards of the second plurality of shards operates in parallel, and wherein each of the shards of the second plurality of shards is hosted on a different node of the second plurality of nodes.
2 more elements
Dependencies are inferred from the claim text.