US 2025/0156401 A1Published application

Auto Computation of Counters and Last-N features across High Cardinality Dimensions in Distributed Systems

AssigneeElasticflash, Inc.

InventorsDarshan Bharatkumar Rawal, Naoki Iwakami, Pradeep Jnana Madhavarapu

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods for computing counters in a distributed system are described. One embodiment includes receiving a counter update request, that includes at least one counter entry each including a timestamp, an operation value, a first set of dimension values, and at least one counter value, writing to a counter ledger, where the counter ledger comprises an immutable table where each entry has a first timestamp, an operation, a first set of dimensions, and a first at least one counter state, each counter state restricted to numeric values, reading a latest counter snapshot from an immutable snapshot ledger, generating a new counter snapshot from the latest counter snapshot and selected counter entries in the counter ledger, where the new counter snapshot includes the second set of dimension values, a second timestamp, at least one snapshot counter value, writing the new counter snapshot to the snapshot ledger.

Claims6

Dependent claims

A counter server, comprising:

a processor;
a network interface;
non-volatile memory comprising:
a counter ledger comprising an immutable table where each entry includes values for fields of:
a first timestamp;
an operation, where the available values for the operation include at least “delta” and “set”;
a first plurality of dimensions, each dimension restricted to values having a particular data type;
and
a first at least one counter state, each counter state restricted to values that are numerical;
a counter analytics application that, when executed, instructs the processor to:
receive a counter update request, where the counter update request includes at least one counter entry that each includes a timestamp, an operation value, a first plurality of dimension values, and at least one counter value;
write the at least one counter entry to the counter ledger;
reading a latest counter snapshot from an immutable snapshot ledger, where the latest counter snapshot has the most recent timestamp of counter snapshots in the snapshot ledger;
select counter entries from the counter ledger for compaction given a second plurality of dimension values, where the counter entries are selected by having dimension values matching those of the second plurality of dimension values and having a timestamp after the time of the latest counter snapshot;
generate a new counter snapshot from the latest counter snapshot and the selected counter entries in the counter ledger at a predefined time interval, where the new counter snapshot includes:
the second plurality of dimension values;
a second timestamp taken from the most recent timestamp of the selected counter entries;
at least one snapshot counter value, where each snapshot counter value is calculated from corresponding counter state values of the selected counter entries and the latest counter snapshot;
write the new counter snapshot to the snapshot ledger.

16 more elements

A counter server, comprising:

a processor;
a network interface;
non-volatile memory comprising:
a counter ledger comprising an immutable table where each entry includes values for fields of:
a first timestamp;
an operation, where the available values for the operation include at least “delta” and “set”;
a first plurality of dimensions, each dimension restricted to values having a particular data type;
and
a first at least one counter state, each counter state restricted to values that are numerical;
a counter analytics application that, when executed, instructs the processor to:
receive a counter snapshot request from a client device, where the counter snapshot request includes at least a first plurality of dimension values;
when the counter snapshot request does not include an “on the fly” flag:
read a latest counter snapshot from an immutable snapshot ledger, where the latest counter snapshot has the most recent timestamp of counter snapshots in the snapshot ledger;
send the latest counter snapshot to the client device;
when the counter snapshot request includes an “on the fly” flag:
reading a latest counter snapshot from a snapshot ledger, where the latest counter snapshot has the most recent timestamp of counter snapshots in the snapshot ledger;
select counter entries from the counter ledger for compaction given a second plurality of dimension values, where the counter entries are selected by having dimension values matching those of the second plurality of dimension values and having a timestamp after the time of the latest counter snapshot;
generate a new counter snapshot from the latest counter snapshot and the selected counter entries in the counter ledger at a predefined time interval, where the new counter snapshot includes:
the second plurality of dimension values;
a second timestamp taken from the most recent timestamp of the selected counter entries;
at least one snapshot counter value, where each snapshot counter value is calculated from corresponding counter state values of the selected counter entries and the latest counter snapshot;
send the new counter snapshot to the client device.

19 more elements

Dependencies are inferred from the claim text.