# US 2023/0266972 A1 · System and methods for single instruction multiple request processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0266972 A1 - Document ID: US20230266972A1 - Status: Published application - Kind: A1 - Application number: 18/072,492 - Filed: 2022-11-30 - Published: 2023-08-24 - Earliest claimed: 2022-02-08 - Assignees: Purdue Research Foundation - Applicants: Purdue Research Foundation - Inventors: Timothy Glenn Rogers; Mahmoud Khairy - Web page: https://patentgrep.com/patent/US20230266972A1 ## CPC - G06F 9/3851: {from multiple instruction streams, e.g. multistreaming} - G06F 9/52: Program synchronisation; Mutual exclusion, e.g. by means of semaphores - G06F 9/3844: {using dynamic branch prediction, e.g. using branch history tables} - G06F 9/3877: {using a secondary processor, e.g. coprocessor (peripheral processor G06F13/12)} - G06F 9/3888: {controlled by a single instruction for multiple threads [SIMT] in parallel} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system may include a central processing unit (CPU) having a Simultaneous Multi-Threading (SMT) thread/execution model. The system may further include a request processing unit (RPU) having an Out-of-Order Single Instruction Multiple Thread (SIMT) execution model. The CPU may receive a plurality of requests. The CPU may group a portion of the requests in a batch. The CPU may cause the RPU to execute instructions corresponding to each request in the batch. The RPU may execute, with a plurality of threads, the instructions corresponding to the batch of requests in lockstep. ## Claims ### Claim 1 A system, comprising: a central processing unit (CPU) having a Simultaneous Multi-Threading (SMT) thread/execution model; and a request processing unit (RPU) having an Out-of-Order Single Instruction Multiple Thread (SIMT) execution model, wherein the CPU is configured to: receive a plurality of requests; group a portion of the requests in a batch; cause the RPU to execute instructions corresponding to each request in the batch, and wherein the RPU is configured to: execute, with a plurality of threads, the instructions corresponding to the batch of requests in lockstep. ### Claim 2 (depends on 1) The system of claim 1, wherein the CPU and RPU are configured to execute instructions of a same instruction set architecture. ### Claim 3 (depends on 1) The system of claim 1, wherein to group a portion of the requests in the batch, the CPU is further configured to: group the requests in response to the requests invoking a same procedure. ### Claim 4 (depends on 1) The system of claim 1, wherein to group a portion of the requests in the batch, the CPU is further configured to: group the requests based on the number of arguments in the requests, respectively. ### Claim 5 (depends on 1) The system of claim 1, wherein the request is received at a data center over a communications network. ### Claim 6 (depends on 1) The system of claim 1, wherein the request is received according to a communications protocol. ### Claim 7 (depends on 6) The system of claim 6, wherein the communications protocol is Hypertext Transfer Protocol (HTTP) or Remote Procedure Call (RPC). ### Claim 8 (depends on 1) The system of claim 1, wherein the CPU is further configured to assign a warp of threads to the requests of the batch, wherein each thread corresponds to a request. ### Claim 9 (depends on 7) The system of claim 7, wherein the CPU is further configured to: coalesce stack segments of the threads in the physical address space to minimize memory divergence. ### Claim 10 (depends on 7) The system of claim 7, wherein the RPU is further configured to: optimize control flow reconvergence at Immediate Post-Dominator (IPDOM) points; wherein the RPU is further configured to: execute the warp of threads according to the control flow. ### Claim 11 (depends on 10) The system of claim 10, wherein the control flow comprises active masks, wherein the RPU is configured to control which threads from the warp of threads are active during serialized execution of the instructions based on the active masks. ### Claim 12 (depends on 1) The system of claim 1, wherein the CPU and RPU are on the same chip. ### Claim 13 (depends on 1) The system of claim 1, wherein the CPU and RPU are on different chips. ### Claim 14 (depends on 13) The system of claim 13, wherein the CPU and RPU communicate via Peripheral Component Interconnect Express (PCIe). ### Claim 15 (depends on 1) The system of claim 1, wherein the CPU can split the batch and allow multi-path execution for requests having significantly longer millisecond scale latency. ### Claim 16 An integrated circuit, comprising: a central processing unit (CPU) core having a Simultaneous Multi-Threading (SMT) thread/execution model; and a request processing unit (RPU) core having an Out-of-Order Single Instruction Multiple Thread (SIMT) execution model, wherein the CPU is configured to: receive a plurality of requests; group a portion of the requests in a batch; cause the RPU to execute instructions corresponding to each request in the batch, and wherein the RPU is configured to: execute, with a plurality of threads, the instructions corresponding to the batch of requests in lockstep. ### Claim 17 A method, comprising receiving a plurality of requests via a network communication protocol; grouping, with a central processing unit (CPU), a portion of the requests in a batch based on at least one of: the requests invoking a same procedure, and the number of arguments in the requests; and executing, with a remote processing unit (RPU) the instructions corresponding to each request in a batch in lockstep, the RPU supporting the same instruction set architecture as the CPU. ### Claim 18 (depends on 17) The method of claim 17, wherein the network communications protocol is Hypertext Transfer Protocol (HTTP) or Remote Procedure Call (RPC). ### Claim 19 (depends on 17) The method of claim 17, further comprising: assigning a warp of threads to the requests of the batch, wherein each thread corresponds to a request. ### Claim 20 (depends on 17) The method of claim 17, further comprising: coalescing stack segments of the threads in a physical address space to minimize memory divergence. ### Claim 21 (depends on 17) The method of claim 17, further comprising optimizing, with the CPU, control flow reconvergence at Immediate Post-Dominator (IPDOM) points; and executing, with the RPU, the warp of threads according to the control flow.