# US 10,122,789 B1 · Log information transmission integrity > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,122,789 B1 - Document ID: US10122789B1 - Status: Granted - Kind: B1 - Application number: 15/698,332 - Filed: 2017-09-07 - Granted: 2018-11-06 - Earliest claimed: 2014-06-20 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Daniel Vassallo; Emmanuel Pierre Devillard; Hongqi Wang; Muhammad Umer Azad - Web page: https://patentgrep.com/patent/US10122789B1 ## CPC - H04L 41/40: using virtualisation of network functions or resources, e.g. SDN or NFV entities - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - G06F 11/0778: {Dumping, i.e. gathering error/state information after a fault for later diagnosis} - G06F 11/0784: {Routing of error reports, e.g. with a specific transmission path or data flow} - G06F 11/30: Monitoring - G06F 17/40: Data acquisition and logging (for input to computer G06F3/00) - H04L 41/046: comprising network management agents or mobile agents therefor - H04L 41/069: using logs of notifications; Post-processing of notifications - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system for collecting and disseminating log information may comprise a log agent and a log service. A log agent may transmit a log stream to a log service using non-overlapping adjacent subsets of log entries. Each subset may be associated with a unique and monotonically increasing sequence number, which is not incremented until the subset is received and processed by a log service. ## Claims ### Claim 1 A system, comprising: at least one processor; at least one memory having stored thereon instructions that, upon execution by the at least one processor, cause the system at least to: store a first subset of a plurality of records, the plurality of records comprising operational information for at least one computing node; store information associating the first subset of the plurality of records with a first identifier issued by a provider of log sequence identifiers; transmit the first subset of the plurality of records and the first identifier for processing by a computing node; obtain information indicating that the first subset of the plurality of records were processed by the computing node; obtain a second identifier issued by the provider of log sequence identifiers, the second identifier unique with respect to previous identifiers of subsets of the plurality of records; free the first subset of the plurality of records from storage; and store information associating a second subset of the plurality of records with the second identifier. ### Claim 2 (depends on 1) The system of claim 1, the at least one memory having stored thereon instructions that, upon execution by the at least one processor, cause the system at least to: obtain the plurality of records from a service operated on the at least one computing node. ### Claim 3 (depends on 1) The system of claim 1, the at least one memory having stored thereon instructions that, upon execution by the at least one processor, cause the system at least to: order the plurality of records based at least in part on time. ### Claim 4 (depends on 3) The system of claim 3, wherein the plurality of records comprises log records from a plurality of hosted services. ### Claim 5 (depends on 1) The system of claim 1, the at least one memory having stored thereon instructions that, upon execution by the at least one processor, cause the system at least to: obtain a message indicative of the computing node having processed the first plurality of messages, the message comprising the second identifier. ### Claim 6 A method, comprising: storing a plurality of log records; obtaining a first identifier from a provider of log sequence identifiers; storing information associating a first subset of the plurality of log records with the first identifier; transmitting the first subset of the plurality of log records to a computing node; freeing the first subset of the plurality of log records from storage in response to receiving information indicating that the computing node has processed the first subset of the plurality of log records; and storing information associating a second subset of the plurality of log records with a second identifier received from the provider of log sequence identifiers. ### Claim 7 (depends on 6) The method of claim 6, wherein the second identifier is received in a message acknowledging that the computing node has processed the first subset of the plurality of log records. ### Claim 8 (depends on 6) The method of claim 6, further comprising: obtaining the plurality of log records from a plurality of hosted services. ### Claim 9 (depends on 8) The method of claim 8, further comprising: transmitting the plurality of log records as a stream in which the plurality of log records are ordered based on time. ### Claim 10 (depends on 6) The method of claim 6, wherein the second subset of the plurality of log records is non-overlapping with the first subset of the plurality of log records. ### Claim 11 (depends on 6) The method of claim 6, further comprising: repeating transmission of the first subset of the plurality of log records and the first identifier, based at least in part on retrieving the first subset of the plurality of log records and the first identifier from storage subsequent to recovering from a system failure. ### Claim 12 (depends on 6) The method of claim 6 wherein the plurality of log records further comprises log entries from a plurality of log files. ### Claim 13 (depends on 6) The method of claim 6, wherein the second identifier is monotonically increased with respect to previous identifiers. ### Claim 14 (depends on 6) The method of claim 6, further comprising: determining a number of entries in the first subset based on a maximum rate at which the computing node can process log records. ### Claim 15 (depends on 6) The method of claim 6, wherein the computing node comprises a service that disseminates log information to a client of a hosted service. ### Claim 16 A non-transitory computer-readable storage medium having stored thereon instructions that, upon execution by a computing device, cause the computing device at least to: store a first subset of a plurality of log entries; store an association between the first subset of the plurality of log entries and a first identifier received from a provider of log sequence identifiers; transmit the first subset of the plurality of log entries and the first identifier to a computing node comprising a service for disseminating log information; obtain information from the computing node, the received information indicative of the first subset of the plurality of log entries having been processed by the computing node, the information further comprising a second identifier unique with respect to previous identifiers including the first identifier; free the record from storage in response to the log entries having been processed by the computing node; and store an association between a second subset of the plurality of log entries a second identifier received from the provider of log sequence identifiers. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein log entries in the first subset of the plurality of log entries are ordered prior to log entries in the second subset of the plurality of log entries. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium of claim 16, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: obtain the first subset of the plurality of log entries from a hosted service. ### Claim 19 (depends on 16) The non-transitory computer-readable storage medium of claim 16, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: determine a number of log entries to include in the first subset based on a maximum rate at which the computing node can process log entries. ### Claim 20 (depends on 16) The non-transitory computer-readable storage medium of claim 16, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: recover from a system failure, wherein the system failure occurred prior to receiving confirmation that the service for disseminating log information processed the first subset of the plurality of log entries; and determine to retransmit the first subset of the plurality of log entries based at least in part on the stored association between the first subset of the plurality of log entries and the first identifier.