# US 11,307,941 B1 · System and method for hybrid kernel- and user-space incremental and full checkpointing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,307,941 B1 - Document ID: US11307941B1 - Status: Granted - Kind: B1 - Application number: 16/854,860 - Filed: 2020-04-21 - Granted: 2022-04-19 - Earliest claimed: 2011-04-28 - Assignees: Open Invention Network, LLC - Applicants: Open Invention Network LLC - Inventors: Keith Richard Backensto; Allan Havemose - Web page: https://patentgrep.com/patent/US11307941B1 ## CPC - G06F 11/1471: involving logging of persistent data for recovery - G06F 11/1407: {Checkpointing the instruction stream} - G06F 11/1438: {Restarting or rejuvenating} - G06F 11/1446: Point-in-time backing up or restoration of persistent data - G06F 11/1448: {Management of the data involved in backup or backup restore} - G06F 11/1482: using middleware or operating system [OS] functionalities - G06F 2201/805: Real-time - G06F 2201/82: Solving problems relating to consistency - G06F 2201/84: Using snapshots, i.e. a logical point-in-time copy of the data - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system includes a multi-process application that runs. A multi-process application runs on primary hosts and is checkpointed by a checkpointer comprised of at least one of a kernel-mode checkpointer module and one or more user-space interceptors providing at least one of barrier synchronization, checkpointing thread, resource flushing, and an application virtualization space. Checkpoints may be written to storage and the application restored from said stored checkpoint at a later time. Checkpointing may be incremental using Page Table Entry (PTE) pages and Virtual Memory Areas (VMA) information. Checkpointing is transparent to the application and requires no modification to the application, operating system, networking stack or libraries. In an alternate embodiment the kernel-mode checkpointer is built into the kernel. ## Claims ### Claim 1 A system, comprising: one or more Central Processing Units (CPUs) operatively connected to computer system memory and configured to execute one or more multi-process applications on a host with a host operating system; a kernel-space checkpointer configured to provide incremental kernel-space checkpointing of the multi-process applications; and one or more user-space interceptors preloaded into an address space of each application process, wherein said user-space interceptors comprise a barrier; wherein said kernel-space checkpointer is called while the execution of application processes are halted at said barrier; wherein said incremental checkpointing is comprised of tracking dirty pages for page table entry (PTE) pages within the address space of the one or more multi-process applications; wherein a dirty page bit is repurposed to track dirty pages for incremental checkpointing. ### Claim 2 (depends on 1) The system according to claim 1 wherein said host operating system is one of Windows, Linux, Solaris, Android, MacOS, iOS, or UNIX. ### Claim 3 (depends on 1) The system according to claim 1 wherein said kernel-space checkpointer is implemented as one of a kernel module, a loadable kernel module, or kernel loadable module. ### Claim 4 (depends on 1) The system according to claim 1 wherein said kernel-space checkpointer is compiled into the kernel. ### Claim 5 (depends on 1) The system according to claim 1 wherein said kernel-space checkpointer is implemented as one of a character device or block device. ### Claim 6 (depends on 1) The system according to claim 1, comprising at least one page table configured to map between process virtual addresses and physical memory addresses. ### Claim 7 (depends on 1) The system according to claim 1, wherein said user-space interceptors comprise creation of a per-process checkpointing thread and an application virtualization space providing a private resource name space. ### Claim 8 (depends on 1) The system according to claim 1, wherein a page table entry with page dirty set is included in an incremental checkpoint, and a page table entry with page dirty bit cleared is excluded from the incremental checkpoint. ### Claim 9 (depends on 8) The system according to claim 8, wherein the page table entry dirty bit is cleared after taking an incremental checkpoint. ### Claim 10 (depends on 1) The system according to claim 1, wherein virtual memory areas (VMAs) of the one or more multi-process applications are used to identify memory pages for multi-process applications. ### Claim 11 (depends on 10) The system according to claim 10, wherein for each virtual memory page identified by said virtual memory areas, a page table is traversed and a corresponding page table entry is identified. ### Claim 12 (depends on 10) The system according to claim 10, wherein checkpoint information for each page contains virtual memory area page information and corresponding page table entry information. ### Claim 13 (depends on 10) The system according to claim 10, wherein said memory pages are identified by traversing virtual memory areas (VMAs) for each application process, and for each virtual memory area identify pages comprising each virtual memory area. ### Claim 14 (depends on 1) The system according to claim 1, further comprising a checkpointer configured to: first taking one full checkpoint comprising memory pages in multi-process applications, and clearing all page dirty bits for said memory pages; taking one or more incremental checkpoints, and merging said incremental checkpoints into said full checkpoint; wherein pages included in said incremental checkpoints replace said pages in the full checkpoint, and other pages in said full checkpoint remain unmodified. ### Claim 15 (depends on 14) The system according to claim 14, wherein said merged checkpoints are written to one of memory, local storage, network storage, or remote storage. ### Claim 16 (depends on 15) The system according to claim 15, further comprising compressing-said checkpoints before writing to one of memory, local storage, network storage, or remote storage. ### Claim 17 (depends on 14) The system according to claim 14, wherein: a full checkpoint is comprised of global state for application processes, process hierarchy, shared state, and individual process checkpoints; and said individual process checkpoints are comprised of one or more page information blocks comprised of virtual memory areas page information and page table entry page information. ### Claim 18 (depends on 14) The system according to claim 14, wherein a page in said full checkpoint is left unmodified if said incremental checkpoints does not contain said page. ### Claim 19 (depends on 1) The system according to claim 1, wherein said incremental checkpoints are transferred from kernel-space to user-spacing using one of copy_to_user( ) or memory mapped pages. ### Claim 20 A method, comprising: executing one or more multi-process applications on a host with a host operating system; wherein one or more user-space interceptors are located in an address space of each application process, wherein the one or more user-space interceptors comprise a barrier; calling a kernel-space checkpointer while the execution of application processes are halted at said barrier; incrementally checkpointing, by the kernel-space checkpointer, the processes of the multi-process applications; wherein said incremental checkpointing is comprised of tracking dirty pages for page table entry pages within the address space of said multi-process applications; wherein a dirty page bit is repurposed to track dirty pages for incremental checkpointing.