# US 8,543,729 B2 · Virtualised receive side scaling > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,543,729 B2 - Document ID: US8543729B2 - Status: Granted - Kind: B2 - Application number: 12/745,535 - Filed: 2008-11-18 - Published: 2010-12-30 - Granted: 2013-09-24 - Earliest claimed: 2007-11-29 - Assignees: Solarflare Communications, Inc. - Applicants: Steven Pope; David Riddoch - Inventors: Steven Pope; David Riddoch - Web page: https://patentgrep.com/patent/US8543729B2 ## CPC - H04L 49/9063: {Intermediate storage in different physical parts of a node or terminal} - H04L 47/10: Flow control; Congestion control - H04L 49/90: Buffering arrangements - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A method for receiving packet data by means of a data processing system having a plurality of processing cores and supporting a network interface device and a set of at least two software domains, each software domain carrying a plurality of data flows and each supporting at least two delivery channels, the method comprising: receiving at the network interface device packet data that is part of a particular data flow; selecting in dependence on one or more characteristics of the packet data a delivery channel of a particular one of the software domains, said delivery channel being associated with a particular one of the processing cores of the system; and mapping the incoming packet data into said selected delivery channel such that receive processing of the packet is performed by the same processing core that performed receive processing for preceding packets of that data flow. ## Claims ### Claim 1 A method for receiving packet data by means of a data processing system having a plurality of processing cores and supporting a network interface device and a set of at least two untrusted software domains, each untrusted software domain: (a) carrying a plurality of data flows; (b) supporting at least two delivery channels between the network interface device and the respective untrusted software domain; and (c) being operable to perform receive processing of received packet data at a processing core of the data processing system, the method comprising: receiving at the network interface device packet data that is part of a particular data flow; selecting in dependence on one or more characteristics of the packet data a delivery channel of a particular one of the untrusted software domains, said delivery channel being associated with a particular one of the processing cores of the data processing system; and mapping the incoming packet data into the selected delivery channel such that receive processing of the packet data is performed in the untrusted software domain by the same processing core that performed receive processing for preceding packets of that data flow. ### Claim 2 (depends on 1) A method as claimed in claim 1, wherein at least some of the state of each delivery channel is supported at the network interface device. ### Claim 3 (depends on 1) A method as claimed in claim 1, wherein each delivery channel is arranged such that receive processing of all packet data accepted into a delivery channel is performed at the same processing core. ### Claim 4 (depends on 1) A method as claimed in claim 1, wherein the network interface performs at least some protocol processing of the packet data. ### Claim 5 (depends on 4) A method as claimed in claim 4, wherein the mapping step includes delivering a subset of the received packet data into the delivery channel. ### Claim 6 (depends on 5) A method as claimed in claim 5, wherein the subset comprises payload data. ### Claim 7 (depends on 6) A method as claimed in claim 6, wherein the payload data is TCP payload data. ### Claim 8 (depends on 1) A method as claimed in claim 1, wherein the selecting step is performed at the network interface device. ### Claim 9 (depends on 1) A method as claimed in claim 1, wherein the data flow corresponds to a network socket of the particular untrusted software domain. ### Claim 10 (depends on 1) A method as claimed in claim 1, wherein the one or more characteristics of the packet data comprise one or more fields of the packet header. ### Claim 11 (depends on 1) A method as claimed in claim 1, wherein the selecting step comprises: matching a first subset of the one or more characteristics of the packet data to a set of stored characteristics so as to identify the particular software domain; and choosing, in dependence on a second subset of the one or more characteristics of the packet data, the delivery channel within the particular untrusted software domain. ### Claim 12 (depends on 11) A method as claimed in claim 11, wherein the matching step is performed by means of a filter. ### Claim 13 (depends on 11) A method as claimed in claim 11, wherein the choosing step is performed by means of a hash function. ### Claim 14 (depends on 13) A method as claimed in claim 13, wherein the hash function is a Toeplitz function and the choosing step further includes using an indirection table. ### Claim 15 (depends on 1) A method as claimed in claim 1, wherein the mapping step comprises: writing the data packet to a delivery channel of the data processing system; and delivering a notification event into a notification channel associated with the selected delivery channel. ### Claim 16 (depends on 15) A method as claimed in claim 15, wherein the notification channel is configured to, on receiving the notification event, cause an interrupt to be delivered to the processing core associated with the selected delivery channel. ### Claim 17 (depends on 15) A method as claimed in claim 15, wherein the notification channel is configured to, on receiving the notification event, cause a wakeup notification event to be delivered to an interrupting notification channel that is arranged to cause an interrupt to be delivered to a processing core. ### Claim 18 (depends on 17) A method as claimed in claim 17, wherein the processing core is the processing core associated with the selected delivery channel. ### Claim 19 (depends on 1) A method as claimed in claim 1, wherein the network interface device performs stateless packet processing. ### Claim 20 (depends on 1) A method as claimed in claim 1, wherein the particular untrusted software domain is a virtualised operating system instance. ### Claim 21 (depends on 1) A method as claimed in claim 1, wherein the particular untrusted software domain is a user-level process. ### Claim 22 (depends on 1) A method as claimed in claim 1, wherein the untrusted software domains are virtualised operating systems having a lower privilege level than a system kernel or hypervisor supported by the data processing system. ### Claim 23 (depends on 1) A method as claimed in claim 1, wherein the untrusted software domains are applications having a lower privilege level than a system kernel or hypervisor supported by the data processing system. ### Claim 24 A data processing system arranged for receiving packet data, comprising a plurality of processing cores and supporting a network interface device and a set of at least two untrusted software domains, each untrusted software domain: (a) carrying a plurality of data flows; (b) supporting at least two delivery channels between the network interface device and the respective untrusted software domain; and (c) being operable to perform receive processing of received packet data at a processing core of the data processing system, the data processing system being configured to: receive at the network interface device packet data that is part of a particular data flow; select in dependence on one or more characteristics of the packet data a delivery channel of a particular one of the untrusted software domains, said delivery channel being associated with a particular one of the processing cores of the data processing system; and map the incoming packet data into the selected delivery channel such that receive processing of the packet data is performed in the untrusted software domain by the same processing core that performed receive processing for preceding packets of that data flow. ### Claim 25 A method for transmitting data packets onto a network by means of a data processing system having a plurality of processing cores and supporting a network interface device and a set of at least two untrusted software domains, each untrusted software domain: (a) carrying a plurality of data flows; (b) supporting at least two transmit channels between the network interface device and the respective untrusted software domain, and (c) being operable to process notification events associated with the transmission of data at a processing core of the data processing system, the method comprising: at a particular one of the untrusted software domains, selecting in dependence on the data flow to which a set of data for transmission belongs, one of at least two transmit channels, said transmit channel being associated with a particular one of the processing cores of the data processing system; and processing notification events associated with the transmission of data through the transmit channel on the processing core associated with said transmit channel. ### Claim 26 (depends on 25) A method as claimed in claim 25, wherein the transmit channel has an associated notification channel to which notification events associated with the transmit channel are delivered. ### Claim 27 (depends on 26) A method as claimed in claim 26, wherein the notification channel is associated with the particular processing core associated with the selected transmit channel. ### Claim 28 (depends on 26) A method as claimed in claim 26, wherein the notification channel is configured to, on receiving the notification event, cause an interrupt to be delivered to the particular processing core associated with the selected transmit channel. ### Claim 29 (depends on 26) A method as claimed in claim 26, wherein the notification channel is configured to, on receiving the notification event, cause a wakeup notification event to be delivered to an interrupting notification channel that is arranged to cause an interrupt to be delivered to a processing core. ### Claim 30 (depends on 29) A method as claimed in claim 29, wherein the processing core is the processing core associated with the selected transmit channel. ### Claim 31 (depends on 25) A method as claimed in claim 25, wherein the untrusted software domain is a virtualised operating system instance. ### Claim 32 (depends on 25) A method as claimed in claim 25, wherein the untrusted software domain is a user-level process. ### Claim 33 (depends on 25) A method as claimed in claim 25, wherein the untrusted software domains are virtualised operating systems having a lower privilege level than a system kernel or hypervisor supported by the data processing system. ### Claim 34 (depends on 25) A method as claimed in claim 25, wherein the untrusted software domains are applications having a lower privilege level than a system kernel or hypervisor supported by the data processing system. ### Claim 35 A data processing system arranged for transmitting data packets onto a network, comprising a plurality of processing cores and supporting a network interface device and a set of at least two untrusted software domains, each untrusted software domain: (a) carrying a plurality of data flows; (b) supporting at least two transmit channels between the network interface device and the respective untrusted software domain; and (c) being operable to process notification events associated with the transmission of data at a processing core of the data processing system, the data processing system being configured to: at a particular one of the untrusted software domains, select in dependence on the data flow to which a set of data for transmission belongs, one of at least two transmit channels, said transmit channel being associated with a particular one of the processing cores of the data processing system; and process notification events associated with the transmission of data through the transmit channel on the processing core associated with said transmit channel. ### Claim 36 A data processing system arranged to receive and transmit packet data by means of a data processing system having a plurality of processing cores and supporting a network interface device and a set of at least two untrusted software domains, each untrusted software domain: (a) carrying a plurality of data flows; (b) supporting at least two delivery channels between the network interface device and the respective untrusted software domain; (c) being operable to perform receive processing of received packet data at a processing core of the data processing system; and (d) being operable to process notification events associated with the transmission of data at a processing core of the data processing system, the data processing system arranged to perform data receiving steps of: receiving at the network interface device packet data that is part of a first data flow; selecting in dependence on one or more characteristics of the packet data a delivery channel of a particular one of the untrusted software domains, said delivery channel being associated with a first one of the processing cores of the data processing system; and mapping the incoming packet data into the selected delivery channel such that receive processing of the packet data is performed in the untrusted software domain by the same processing core that performed receive processing for preceding packets of that first data flow, the data processing system further arranged to perform data transmitting steps of: at a particular one of the untrusted software domains, selecting in dependence on a second data flow to which a set of data for transmission belongs, one of at least two transmit channels, said selected transmit channel being associated with a second one of the processing cores of the data processing system; and processing notification events associated with the transmission of data through the selected transmit channel on the second processing core, the data processing system further arranged to determine whether the first and second data flows are the same data flow, and if so, then to select the same processing core for both the data receiving steps of the first flow and the data transmitting steps of the second flow. ### Claim 37 A method for managing interaction between a data processing system and a network interface device, the data processing system having a plurality of processing cores and supporting a set of at least two untrusted software domains, each untrusted software domain: (a) carrying a plurality of data flows; (b) supporting a set of at least two notification channels; and (c) being operable to process notification events at a processing core of the data processing system, the method comprising: at the network interface device, in response to processing data of a data flow of one of the untrusted software domains, selecting in dependence on one or more characteristics of the data flow one of the set of notification channels of the software domain, each notification channel being associated with a particular one of the processing cores of the data processing system; delivering a notification event to the selected notification channel; and responsive to delivering the notification event to the selected notification channel, causing an interrupt to be delivered to the processing core associated with the selected notification channel such that processing of the notification event is performed at that processing core by a processing entity of the software domain. ### Claim 38 (depends on 37) A method as claimed in claim 37, wherein the notification event indicates that one or more data packets have been received at the network interface device. ### Claim 39 (depends on 37) A method as claimed in claim 37, wherein the notification event indicates that one or more data packets have been transmitted by the network interface device. ### Claim 40 (depends on 37) A method as claimed in claim 37, wherein at least some protocol processing of the data packets is performed at the network interface device. ### Claim 41 (depends on 25) A method as claimed in claim 25, wherein the untrusted software domains are applications having a lower privilege level than a system kernel or hypervisor supported by the data processing system.