# US 11,907,751 B2 · Resource fairness enforcement in shared IO interfaces > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,751 B2 - Document ID: US11907751B2 - Status: Granted - Kind: B2 - Application number: 18/083,705 - Filed: 2022-12-19 - Published: 2023-04-20 - Granted: 2024-02-20 - Earliest claimed: 2019-11-21 - Assignees: Pensando Systems Inc. - Applicants: Pensando Systems Inc. - Inventors: Michael Brian Galles - Web page: https://patentgrep.com/patent/US11907751B2 ## CPC - G06F 13/30: with priority control - G06F 9/45558: {Hypervisor-specific management and integration aspects} - H04L 41/0893: Assignment of logical groups to network elements - H04L 41/40: using virtualisation of network functions or resources, e.g. SDN or NFV entities - H04L 43/0876: Network utilisation, e.g. volume of load or congestion level - H04L 43/16: Threshold monitoring - H04L 43/20: the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV - H04L 47/50: Queue scheduling - H04L 47/70: Admission control; Resource allocation - H04L 49/254: {Centralised controller, i.e. arbitration or scheduling} - H04L 49/3009: {Header conversion, routing tables or routing tags} - G06F 2009/45579: {I/O management, e.g. providing access to device drivers or storage} - G06F 2009/45595: {Network integration; Enabling network access in virtual machine instances} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Described are platforms, systems, and methods for resource fairness enforcement. In one aspect, a programmable input output (IO) device comprises a memory unit, the memory unit having instructions stored thereon which, when executed by the programmable IO device, cause the programmable IO device to perform operations comprising: receiving an input from a logical interface (LIF); determining, by at least one meter, a metric regarding at least one resource used during a processing of the input through a programmable pipeline; and regulating additional input received from the LIF based on the metric and a threshold for the at least one resource. ## Claims ### Claim 1 An input output (IO) device comprising: a memory unit, the memory unit having instructions stored thereon which, when executed by the IO device, cause the IO device to perform operations comprising: receiving input from a first logical interface (LIF) and a second LIF; tracking a first metric regarding a usage per packet of at least one resource when processing input from the first LIF and tracking a second metric regarding a usage per packet of the at least one resource when processing input from the second LIF; and regulating additional input received from the first and second LIFs by comparing the first metric to a first threshold assigned to the first LIF for the at least one resource and by comparing the second metric to a second threshold assigned to the second LIF for the at least one resource, wherein the first and second thresholds are different. ### Claim 2 (depends on 1) The IO device of claim 1, wherein regulating the additional input received from the LIF comprises rejecting the additional input when the first and second metrics exceed the first and second thresholds. ### Claim 3 (depends on 2) The IO device of claim 2, wherein the operations comprise: redetermining the first metric regarding the at least one resource after a time threshold has been met, or a number of other inputs received from other LIFs have been processed, wherein regulating the additional input received from the LIF comprises accepting the additional input when the redetermined metric is below the first threshold. ### Claim 4 (depends on 1) The IO device of claim 1, wherein regulating the additional input received from the LIF comprises not dequeuing the additional input when the first metric exceeds the first threshold. ### Claim 5 (depends on 4) The IO device of claim 4, wherein the operations comprise: redetermining the first metric regarding the at least one resource after a time threshold has been met, or a number of other inputs received from other LIFs have been dequeued, wherein regulating the additional input received from the LIF comprises dequeuing the additional input when the redetermined metric is below the first threshold. ### Claim 6 (depends on 1) The IO device of claim 1, wherein the operations comprises processing the input through a programmable pipeline, the programmable pipeline comprises a pre-determined number of table engines and match processing unit (MPU) pipeline stages, wherein the programmable pipeline breaks down the processing of the input into a series of table lookups or matches and respective processing actions, and wherein each of the MPU pipeline stages comprise an MPU performing table-based actions. ### Claim 7 (depends on 6) The IO device of claim 6, wherein the MPU pipeline stages execute a targeted program, determined based on a domain specific instruction set, with results of the table lookups and headers from the input as parameters to determine table updates and header rewrite operations. ### Claim 8 (depends on 1) The IO device of claim 1, wherein the usage per packet of the at least one resource comprises a number of operations or events associated with the at least one resource, wherein the operations or events associated with the at least one resource comprise table access, executed instructions, a stall event, or a fence event. ### Claim 9 (depends on 1) The IO device of claim 1, wherein the first and second metrics are tracked by at least one meter, wherein the at least one meter is programmable and is further programmed to track a metric for a group of resources, wherein each of the resources in the group of resources are weighted using weighted values, and wherein the at least one meter is programmed according to the weighted values. ### Claim 10 (depends on 1) The IO device of claim 1, wherein the operations comprises processing the input through a programmable pipeline, the programmable pipeline comprises a packet receive path and a packet transmit path, wherein the input comprises a packet, and wherein the input is received by a packet buffer. ### Claim 11 (depends on 10) The IO device of claim 10, wherein the input is received by a scheduler, wherein the input originated from a server, and wherein the input is queued by the scheduler. ### Claim 12 (depends on 1) The IO device of claim 1, wherein the IO device comprises a smart network interface card (NIC). ### Claim 13 One or more non-transitory computer-readable storage media coupled to an input output (IO) device and having instructions stored thereon which, when executed by the IO device, cause the IO device to perform operations comprising: receiving input from a first logical interface (LIF) and a second LIF; tracking a first metric regarding a usage per packet of at least one resource when processing input from the first LIF and tracking a second metric regarding a usage per packet of the at least one resource when processing input from the second LIF; and regulating additional input received from the first and second LIFs by comparing the first metric to a first threshold assigned to the first LIF for the at least one resource and by comparing the second metric to a second threshold assigned to the second LIF for the at least one resource, wherein the first and second thresholds are different. ### Claim 14 An input output (IO) device comprising: a plurality of pipelined stages; and a plurality of hardware meters, each of which is assigned to a determine whether received packets are permitted to enter a respective stage of the plurality of pipelined stages, wherein each of the plurality of hardware meters is configured to: track a first metric regarding a usage of a hardware resource in the respective stage when processing packets received from a first logical interface (LIF), track a second metric regarding a usage of a hardware resource in the respective stage when processing packets received from a second LIF, block the packets received from the first LIF from being processed by the respective stage when the first metric satisfies a first threshold, and block the packets received from the second LIF from being processed by the respective stage when the second metric satisfies a second threshold, wherein the first and second thresholds are different. ### Claim 15 (depends on 14) The IO device of claim 14, wherein the 10 device is configured to redetemine the first metric regarding the usage of the hardware resource in a respective stage after a time threshold has been met, or a number of other packets received from other LIFs have been processed. ### Claim 16 (depends on 14) The IO device of claim 14, wherein the 10 device is configured to redetemine the first metric regarding the usage of the hardware resource in a respective stage after a time threshold has been met, or a number of other packets received from other LIFs have been dequeued. ### Claim 17 (depends on 14) The IO device of claim 14, wherein the plurality of pipeline stages comprises a pre-determined number of table engines and match processing unit (MPU) pipeline stages, wherein the plurality of pipeline stages break down processing of the received packets into a series of table lookups or matches and respective processing actions, wherein each of the MPU pipeline stages comprise an MPU performing table-based actions. ### Claim 18 (depends on 17) The IO device of claim 17, wherein the MPU pipeline stages execute a targeted program, determined based on a domain specific instruction set, with results of the table lookups and headers from the received packets as parameters to determine table updates and header rewrite operations. ### Claim 19 (depends on 14) The IO device of claim 14, wherein each of the plurality of hardware meters is programmable and is further programmed to track a metric for a group of resources, wherein each of the resources in the group of resources are weighted using weighted values, and wherein each of the plurality of hardware meters is programmed according to the weighted values. ### Claim 20 (depends on 14) The IO device of claim 14, wherein the plurality of pipelined stages and the plurality of hardware meters are in a smart network interface card (NIC).