# US 12,010,195 B2 · Efficient flow management utilizing control packets > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,010,195 B2 - Document ID: US12010195B2 - Status: Granted - Kind: B2 - Application number: 17/959,941 - Filed: 2022-10-04 - Published: 2023-01-26 - Granted: 2024-06-11 - Earliest claimed: 2021-04-29 - Assignees: Oracle International Corporation - Applicants: Oracle International Corporation - Inventors: Brijesh Singh - Web page: https://patentgrep.com/patent/US12010195B2 ## CPC - H04L 67/5683: Storage of data provided by user terminals, i.e. reverse caching - H04L 45/04: {Interdomain routing, e.g. hierarchical routing} - H04L 47/35: by embedding flow control information in regular packets, e.g. piggybacking - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - H04L 67/568: Storing data temporarily at an intermediate stage, e.g. caching - H04L 69/22: Parsing or analysis of headers - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Techniques are disclosed for utilizing control packets to manage flows by a smart network interface card (smartNIC). In one example, an accelerator determines that a cache entry is a candidate for removal, the cache entry being part of a cache that is managed by the accelerator, the cache entry storing flow state of a particular flow, the accelerator being responsible for forwarding packets associated with the particular flow based at least in part on the flow state, and the flow information formatted utilizing a particular header format. The accelerator generates an instruction to remove the cache entry of the particular flow from the cache based at least in part on receiving the determination that the cache entry is the candidate for removal. The accelerator removes the cache entry of the particular flow from the cache based at least in part on the instruction. ## Claims ### Claim 1 A method, comprising: determining, by an accelerator of a smart network interface card (smartNIC), that a cache entry is a candidate for removal, the cache entry being part of a cache that is managed by the accelerator, the cache entry storing flow state of a particular flow, the accelerator being responsible for forwarding packets associated with the particular flow based at least in part on the flow state, and the cache entry being indexed within the cache based at least in part on a hash that is generated based at least in part on flow information associated with the particular flow, and the flow information formatted utilizing a particular header format; generating, by the accelerator, an instruction to remove the cache entry of the particular flow from the cache based at least in part on receiving the determination that the cache entry is the candidate for removal; removing, by the accelerator, the cache entry of the particular flow from the cache based at least in part on the instruction; generating, by the accelerator, a control packet that includes the flow information associated with the particular flow being formatted utilizing the particular header format; and transmitting, by the accelerator, the control packet to a programming data plane utilizing a path that is used by the accelerator to forward non-control packets to the programming data plane, the path being utilized based at least in part on the control packet utilizing the particular header format. ### Claim 2 (depends on 1) The method of claim 1, wherein a second flow state of the particular flow is stored in a second cache entry of a second cache that is managed by the programming data plane, and wherein the programming data plane is configured to determine whether to remove the second cache entry from the second cache based at least in part on receiving the control packet. ### Claim 3 (depends on 1) The method of claim 1, wherein the particular header format comprises a five-tuple Internet Protocol (IP) format, and wherein data fields of the five-tuple IP format correspond to the flow information that is associated with the particular flow. ### Claim 4 (depends on 1) The method of claim 1, wherein the path that is used by the accelerator to forward non-control packets to the programming data plane corresponds to a network on a chip (NOC) that is configured to route packets that utilize the particular header format between the accelerator and the programming data plane. ### Claim 5 (depends on 1) The method of claim 1, wherein the method further comprises: receiving a data packet from the smart network interface card (smartNIC), wherein the data packet comprises the particular header format and the flow information associated with the particular flow, wherein the particular header format comprises a multi-tuple header; obtaining the multi-tuple header from the data packet, wherein the multi-tuple header comprises a source Internet Protocol (IP) address, a destination Internet Protocol (IP) address, a source port, a destination port, or a protocol type; generating, using a hash function, the hash of the multi-tuple header obtained from the data packet; and generating the cache entry for the cache, wherein the cache entry is indexed in the cache based at least in part on the hash, and wherein the cache entry comprises the flow information associated with the particular flow. ### Claim 6 (depends on 1) The method of claim 1, wherein the accelerator determines that the cache entry of the particular flow is the candidate for removal based at least in part on determining that a number of cache entries being stored in the cache matches a predefined storage capacity threshold, the number of cache entries respectively corresponding to flows being managed by the accelerator. ### Claim 7 (depends on 1) The method of claim 1, wherein the accelerator determines that the cache entry for the particular flow is the candidate for removal based at least in part on determining that a timer associated with the particular flow has expired, the timer indicating a period of time elapsed from a time that a packet associated with the particular flow was last received. ### Claim 8 (depends on 1) The method of claim 1, further comprising: receiving, by the accelerator, a data packet associated with the particular flow; and determining, by the accelerator, that the cache entry of the particular flow is the candidate for removal based at least in part on determining that the data packet indicates a termination or a reset of the particular flow. ### Claim 9 An accelerator of a smart network interface card (smartNIC), comprising: a memory configured to store computer-executable instructions; and a processor configured to access the memory and execute the computer-executable instructions to at least: determine, by the accelerator, that a cache entry is a candidate for removal, the cache entry being part of a cache that is managed by the accelerator, the cache entry storing flow state of a particular flow, the accelerator being responsible for forwarding packets associated with the particular flow based at least in part on the flow state, and the cache entry being indexed within the cache based at least in part on a hash that is generated based at least in part on flow information associated with the particular flow, and the flow information formatted utilizing a particular header format; generate, by the accelerator, an instruction to remove the cache entry of the particular flow from the cache based at least in part on receiving the determination that the cache entry is the candidate for removal; remove, by the accelerator, the cache entry of the particular flow from the cache based at least in part on the instruction; generate, by the accelerator, a control packet that includes the flow information associated with the particular flow being formatted utilizing the particular header format; and transmit, by the accelerator, the control packet to a programming data plane utilizing a path that is used by the accelerator to forward non-control packets to the programming data plane, the path being utilized based at least in part on the control packet utilizing the particular header format. ### Claim 10 (depends on 9) The accelerator of claim 9, wherein a path that is used by the accelerator to forward non-control packets to a programming data plane corresponds to a network on a chip (NOC) that is configured to route packets that utilize the particular header format between the accelerator and the programming data plane. ### Claim 11 (depends on 9) The accelerator of claim 9, wherein the processor is further configured to execute the computer-executable instructions to at least: receive a data packet associated with the particular flow; and determine that the cache entry of the particular flow is the candidate for removal based at least in part on determining that the data packet indicates a termination or a reset of the particular flow. ### Claim 12 (depends on 9) The accelerator of claim 9, wherein the processor is further configured to execute the computer-executable instructions to at least: receive a data packet of the smart network interface card (smartNIC), wherein the data packet comprises the particular header format and the flow information associated with the particular flow, wherein the particular header format comprises a multi-tuple header; obtain the multi-tuple header from the data packet, wherein the multi-tuple header comprises a source Internet Protocol (IP) address, a destination Internet Protocol (IP) address, a source port, a destination port, or a protocol type; generating, using a hash function, the hash of the multi-tuple header obtained from the data packet; and generating the cache entry for the cache, wherein the cache entry is indexed in the cache based at least in part on the hash, and wherein the cache entry comprises the flow information associated with the particular flow. ### Claim 13 (depends on 9) The accelerator of claim 9, wherein the instruction corresponds to a bit string of a plurality of bits, and wherein the instruction is prepended or appended to the header of a control packet. ### Claim 14 (depends on 9) The accelerator of claim 9, wherein the accelerator determines that the cache entry of the particular flow is the candidate for removal based at least in part on determining that a number of cache entries being stored in the cache matches a predefined storage capacity threshold, the number of cache entries respectively corresponding to flows being managed by the accelerator. ### Claim 15 (depends on 9) The accelerator of claim 9, wherein the accelerator determines that the cache entry of the particular flow is the candidate for removal based at least in part on determining that a timer associated with the particular flow has expired, the timer having been programmed based at least in part on instructions received from a programming data plane. ### Claim 16 One or more non-transitory computer-readable media comprising computer-executable instructions that, when executed by one or more processors of an accelerator of a smart network interface card (smartNIC), cause the one or more processors to at least: determine, by the accelerator, that a cache entry is a candidate for removal, the cache entry being part of a cache that is managed by the accelerator, the cache entry storing flow state of a particular flow, the accelerator being responsible for forwarding packets associated with the particular flow based at least in part on the flow state, and the cache entry being indexed within the cache based at least in part on a hash that is generated based at least in part on flow information associated with the particular flow, and the flow information formatted utilizing a particular header format; generate, by the accelerator, an instruction to remove the cache entry of the particular flow from the cache based at least in part on receiving the determination that the cache entry is the candidate for removal; remove, by the accelerator, the cache entry of the particular flow from the cache based at least in part on the instruction; generate, by the accelerator, first data associated with the particular flow being managed by the accelerator; include by the accelerator, the first data within a control packet; and transmit, by the accelerator, the first data within the control packet to a programming data plane utilizing a path that is used by the accelerator to forward non-control packets to the programming data plane. ### Claim 17 (depends on 16) The one or more non-transitory computer-readable media of claim 16, wherein the accelerator determines that the cache entry of the particular flow is the candidate for removal based at least in part on determining that a timer associated with the particular flow has expired, the timer having been programmed based at least in part on instructions received from a programming data plane. ### Claim 18 The one or more non-transitory computer-readable media of claimer 16, wherein the first data corresponds to at least one of: (I) flow expiry information associated with the particular flow, (II) flow statistics of the particular flow, or (III) a request for instructions associated with removing another cache entry of the cache managed by the accelerator.