# US 10,374,915 B1 · Metrics processing service > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,374,915 B1 - Document ID: US10374915B1 - Status: Granted - Kind: B1 - Application number: 14/754,388 - Filed: 2015-06-29 - Granted: 2019-08-06 - Earliest claimed: 2015-06-29 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Xiqiang Zhi; George Steven McPherson - Web page: https://patentgrep.com/patent/US10374915B1 ## CPC - G06F 9/5061: {Partitioning or combining of resources} - H04L 43/065: related to network devices - H04L 43/0876: Network utilisation, e.g. volume of load or congestion level - H04L 41/12: Discovery or management of network topologies - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A distributed services platform may generate a plurality of log files containing metric values. Metric values may be provided to a first level of a topology of aggregation modules. The first level of aggregation modules may provide output to a second level of the topology. Subsequent levels of the topology may perform greater amounts of aggregation while providing stream-based access to the aggregated values. State information for the aggregation topology may be saved subsequent to each cycle of values through the topology. ## Claims ### Claim 1 A system comprising: one or more computing nodes operable as a service, the service generating a plurality of log entries during operation; and one or more memories having stored thereon computer-readable instructions that, upon execution by a computing device, cause the system at least to: identify, based on a topology of aggregation modules, a first aggregation module corresponding to a first level of the topology; identify, based on the topology, a second aggregation module corresponding to a second level of the topology; provide a first value to the first aggregation module, the first value corresponding to an entry in the plurality of log entries; provide a second value to the second aggregation module, the second value obtained based at least in part on output of the first aggregation module using the first value and one or more additional values corresponding to entries in the plurality of log entries; obtain a third value based at least in part on output of the second aggregation module using the second value and one or more additional values corresponding to output of the first aggregation module; store information indicative of a location of the entry in the plurality of log entries for retrieval by a client process to provide a real-time view of the operation of the service. ### Claim 2 (depends on 1) The system of claim 1, further comprising one or more memories having stored thereon computer-readable instructions that, upon execution by the computing device, cause the system at least to: store the information indicative of the location of the entry in response to providing the first value to the first aggregation module and the second value to the second aggregation module. ### Claim 3 (depends on 1) The system of claim 1, further comprising one or more memories having stored thereon computer-readable instructions that, upon execution by the computing device, cause the system at least to: store the information indicative of the location of the entry in response to providing the first value to all aggregations modules associated with a top level of aggregation modules in the aggregation topology. ### Claim 4 (depends on 1) The system of claim 1, wherein the information indicative of the location of the entry is a timestamp. ### Claim 5 (depends on 1) The system of claim 1, further comprising one or more memories having stored thereon computer-readable instructions that, upon execution by the computing device, cause the system at least to: transmit the second value to a client process in response to the first aggregation module receiving a first set of values corresponding to a first aggregation period; and transmit the third value to the client process in response to the second aggregation module receiving a second set of values, calculated by the first aggregation module, corresponding to a second aggregation period. ### Claim 6 A method comprising: providing a first value to a first aggregation module corresponding to a first level of a topology of aggregation modules, the first value corresponding to an entry in a plurality of log entries; providing a second value to a second aggregation module corresponding to a second level of the topology, the second value obtained based at least in part on output of the first aggregation module using the first value and one or more additional values corresponding to entries in the plurality of log entries; obtaining a third value based at least in part on output of the second aggregation module using the second value and one or more additional values corresponding to output of the first aggregation module; storing information indicative of a location of the entry in the plurality of log entries; and providing the second value to a client process, wherein the second value is provided prior to obtaining the third value. ### Claim 7 (depends on 6) The method of claim 6, wherein the information indicative of the location of the entry is indicative of completed processing of the first value. ### Claim 8 (depends on 6) The method of claim 6, wherein the information indicative of the location of the entry comprises an offset into a source of the plurality of log entries, wherein data prior to the offset has been processed by the first level of the topology. ### Claim 9 (depends on 6) The method of claim 6, wherein the information indicative of the location of the entry comprises a value indicative of time. ### Claim 10 (depends on 6) The method of claim 6, further comprising: storing, in a low-latency memory, state information for the first aggregation module and the second aggregation module in response to providing the first value to all aggregations modules associated with a top level of aggregation modules in the topology; and storing, on a durable storage device, the information indicative of the location of the entry. ### Claim 11 (depends on 6) The method of claim 6, wherein the first aggregation module calculates the second value in response to receiving a set of values corresponding to a first aggregation period. ### Claim 12 (depends on 6) The method of claim 6, further comprising: transmitting the second value to a client process in response to the first aggregation module receiving a set of values corresponding to a first aggregation period; and transmitting the third value to the client process in response to the second aggregation module receiving a set of values, calculated by the first aggregation module, corresponding to a second aggregation period. ### Claim 13 (depends on 6) The method of claim 6, further comprising: resuming processing, subsequent to a system failure, of the plurality of log entries based at least in part on an interval of the first aggregation module and the information indicative of the location of the entry in the plurality of log entries. ### Claim 14 A non-transitory computer-readable storage medium having stored thereon instructions that, upon execution by one or more computing devices, cause the one or more computing devices at least to: provide a first value to a first aggregation module corresponding to a first level of a topology, the first value corresponding to an entry in a plurality of log entries; provide a second value to a second aggregation module corresponding to a second level of the topology, the second value obtained based at least in part on output of the first aggregation module using the first value and one or more additional values corresponding to entries in the plurality of log entries; obtain a third value based at least in part on output of the second aggregation module using the second value and one or more additional values corresponding to output of the first aggregation module; store a timestamp associated with the entry in the plurality of log entries; and provide, prior to obtaining the third value, the second value to a client process. ### Claim 15 (depends on 14) The non-transitory computer-readable storage medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: store, in a low-latency memory, state information for the first aggregation module and the second aggregation module in response to providing the first value to all aggregations modules associated with a top level of aggregation modules in the topology; and store the timestamp on a durable storage device. ### Claim 16 (depends on 14) The non-transitory computer-readable storage medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: store the timestamp in response to providing the first value to all aggregations modules associated with a top level of aggregation modules in the topology. ### Claim 17 (depends on 14) The non-transitory computer-readable storage medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: process the plurality of log entries based at least in part on an interval of the first aggregation module and the timestamp. ### Claim 18 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein the first aggregation module calculates the second value in response to receiving a set of values corresponding to a first aggregation period. ### Claim 19 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein the second aggregation module is associated with a second aggregation period longer than a first aggregation period. ### Claim 20 (depends on 14) The non-transitory computer-readable storage medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: transmit the second value to a client process in response to the first aggregation module receiving a set of values corresponding to a first aggregation period; and transmit the third value to the client process in response to the second aggregation module receiving a set of values, calculated by the first aggregation module, corresponding to a second aggregation period.