# US 10,771,306 B2 · Log monitoring system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,771,306 B2 - Document ID: US10771306B2 - Status: Granted - Kind: B2 - Application number: 15/788,381 - Filed: 2017-10-19 - Published: 2018-03-01 - Granted: 2020-09-08 - Earliest claimed: 2012-02-08 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Raleigh H. Upshur; Nicholas Alexander Allen; Patrick J. Ward - Web page: https://patentgrep.com/patent/US10771306B2 ## CPC - H04L 69/329: in the application layer [OSI layer 7] - G06F 11/00: Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00) - G06F 11/0778: {Dumping, i.e. gathering error/state information after a fault for later diagnosis} - G06F 11/0784: {Routing of error reports, e.g. with a specific transmission path or data flow} - G06F 11/0787: {Storage of error reports, e.g. persistent data storage, storage using memory protection} - G06F 11/3006: {where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50)} - G06F 11/3409: {for performance assessment} - G06F 11/3452: {Performance evaluation by statistical analysis} - H04L 41/0686: Additional information in the notification, e.g. enhancement of specific meta-data - H04L 41/069: using logs of notifications; Post-processing of notifications - G06F 11/3476: {Data logging (G06F11/14, G06F11/2205 take precedence)} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Disclosed are various embodiments for a log monitoring system to monitor the health of server log files. The log monitoring system may generate at least one log health signal based on an analysis of the server log content generated by at least one host application. Furthermore, the application may generate a system integrity record based on the at least one log health signal and an external signal, wherein the external signal embodies a system health metric of the at least one host application. ## Claims ### Claim 1 A non-transitory computer-readable medium having a plurality of computer instructions executable by at least one computing device, wherein, upon execution, the plurality of computer instructions cause the at least one computing device to: retrieve a plurality of log files generated by at least one host application; generate a first log health signal based at least in part on whether a data size of the plurality of log files is within a defined deviation of an expected server log data size for a time period, wherein the expected server log data size is determined at least in part as a function of a network traffic prediction, the network traffic prediction being determined based at least in part on both an expected quantity of log file data generated per client multiplied by a quantity of a plurality of clients accessing a host system within the time period, the at least one host application being executed by the host system; generate a second log health signal based at least in part on whether the plurality of log files meets an expected number of log files; generate a third log health signal based at least in part on whether the plurality of log files meets an expected server log format, wherein the expected server log format comprises a defined file format; receive an indication of an operational error detected by the host system and a time of origination of the operational error; determine at least one log file of the plurality of log files associated with the operational error by matching the time of origination of the operational error to a time of creation of the at least one log file; and determine a server log content error record based at least in part on the first log health signal, the second log health signal, the third log health signal, the at least one log file, and the operational error, the server log content error record including a server log content error associated with an interval of time, and the server log error record represents an absence or a detection of log tampering or log file corruption. ### Claim 2 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein individual ones of the plurality of log files comprise a record of a plurality of operations executed in association with the at least one host application. ### Claim 3 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the plurality of computer instructions further cause the at least one computing device to transmit the server log content error to an alarm service for transmitting a notification message to a client device according to the server log content error. ### Claim 4 A system, comprising: at least one computing device; and at least one application stored on a hardware memory executed by a hardware processor in the at least one computing device, the at least one application causing the at least one computing device to at least: receive a plurality of log files generated by a host application; generate a first log health signal based at least in part on whether a data size of the plurality of log files is within a defined deviation of an expected server log content data size for a time period, wherein the expected server log content data size is determined at least in part as a function of a network traffic prediction, the network traffic prediction being determined at least in part as a function of both an expected quantity of log file data generated per client multiplied by a quantity of a plurality of clients accessing a host system within the time period, the host application being executed by the host system; generate a second log health signal based at least in part on whether the plurality of log files meets an expected number of log files; generate a third log health signal based at least in part on whether the plurality of log files meets an expected server log format, wherein the expected server log format comprises a defined file format; receive an indication of an operational error detected by the host system and a time of origination of the operational error; determine that at least one log file of the plurality of log files is associated with the operational error by matching the time of origination of the operational error to a time of creation of the at least one log file; and determine a server log content error record based at least in part on the first log health signal, the second log health signal, the third log health signal, the at least one log file, and the operational error, the server log content error record including a log content error associated with an interval of time, and the server log content error record represents an absence or a detection of log tampering or log file corruption. ### Claim 5 (depends on 4) The system of claim 4, wherein the at least one application further causes the at least one computing device to at least generate an alarm in response to generating a plurality of consecutive system integrity errors. ### Claim 6 (depends on 4) The system of claim 4, wherein the third log health signal is generated in response to identifying a threshold number of errors in the plurality of log files. ### Claim 7 (depends on 4) The system of claim 4, wherein the third log health signal is generated in response to a threshold number of lines in at least one log file matching a particular pattern. ### Claim 8 (depends on 4) The system of claim 4, wherein the expected server log content data size is further determined from receiving an expected data size for the host application from a configuration database. ### Claim 9 (depends on 4) The system of claim 4, wherein the plurality of log files are retrieved from an archival log database based at least in part on a time offset from a generation of at least one of the plurality of log files. ### Claim 10 (depends on 4) The system of claim 4, wherein the server log content error is generated based at least in part on a weight associated with at least one log health signal. ### Claim 11 (depends on 4) The system of claim 4, wherein the at least one application further causes the at least one computing device to at least receive an intrusion detection signal transmitted from an intrusion detection system that executes concurrently with the host application, wherein the intrusion detection signal is transmitted in response to detecting an intrusion event for the host application. ### Claim 12 (depends on 4) The system of claim 4, wherein the at least one application further causes the at least one computing device to identify a previous alarm decision, from memory, by correlating at least one log health signal to a previous log health signal associated with the previous alarm decision, wherein the previous alarm decision comprises an indication of whether a respective system log error was generated in response to the previous log health signal. ### Claim 13 (depends on 12) The system of claim 12, wherein generating the server log content error is further based at least in part on the previous alarm decision. ### Claim 14 (depends on 4) The system of claim 4, wherein the at least one application further causes the at least one computing device to store, in a memory accessible to the at least one computing device, a historical analysis comprising the server log content error and a corresponding analysis of a determination for transmitting an alarm for the server log content error, wherein the corresponding analysis comprises an indication as to whether the alarm was a false alarm or a true alarm. ### Claim 15 (depends on 4) The system of claim 4, wherein the at least one application further causes the at least one computing device to transmit the server log content error to an alarm service for generating an alarm according to the server log content error. ### Claim 16 (depends on 4) The system of claim 4, wherein the at least one application further causes the at least one computing device to transmit an alarm notification to a client device in response to determining that a percent of a plurality of server log errors among a plurality of time intervals exceeds a threshold percentage. ### Claim 17 A method, comprising: receiving, in at least one computing device, a plurality of log files generated by a host application; generating, in the at least one computing device, a first log health signal based at least in part on whether a data size of the plurality of log files is within a defined deviation of an expected server log data size for a time period, wherein the expected server log data size is determined at least in part as a function of a network traffic prediction, the network traffic prediction being determined based at least in part on both an expected quantity of data generated per client multiplied by a quantity of a plurality of clients accessing a host system within the time period, the host application being executed by the host system; generating, in the at least one computing device, a second log health signal based at least in part on whether the plurality of log files meets an expected number of log files; generating, in the at least one computing device, a third log health signal based at least in part on whether the plurality of log files meets an expected server log format, wherein the expected server log format comprises a defined file format; receiving, in the at least one computing device, an indication of an operational error detected by the host system and a time of origination of the operational error; determining, in the at least one computing device, at least one log file of the plurality of log files as being associated with the operational error by matching the time of origination of the operational error to a time of creation of the at least one log file of the plurality of log files; and determining, in the at least one computing device, a system log error record based at least in part on the first log health signal, the second log health signal, the third log health signal, the at least one log file, and the operational error, the system log error record including a server log content error associated with an interval of time, wherein the system log error record represents an absence or a detection of log tampering or log file corruption. ### Claim 18 (depends on 17) The method of claim 17, further comprising storing, in a memory accessible to the at least one computing device, a historical analysis comprising the system log error record, a corresponding analysis of a determination of the system log error record, and a plurality of past alarm outcomes associated with at least one log health signal. ### Claim 19 (depends on 18) The method of claim 18, further comprising, in the at least one computing device, tuning an analysis used to generate the system log error record based at least in part on the historical analysis. ### Claim 20 (depends on 17) The method of claim 17, wherein generating the system log error record further comprises recalculating the system log error record based at least in part on a delayed log file retrieved in response to a log backfilling occurrence.