# US 10,873,534 B1 · Data plane with flow learning circuit > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,873,534 B1 - Document ID: US10873534B1 - Status: Granted - Kind: B1 - Application number: 15/927,723 - Filed: 2018-03-21 - Granted: 2020-12-22 - Earliest claimed: 2017-03-21 - Assignees: Barefoot Networks, Inc. - Applicants: Barefoot Networks, Inc. - Inventors: Michael Gregory Ferrara; Jay Evan Scott Peterson; Steven Licking; Jeongkeun Lee; Patrick Bosshart; Anurag Agrawal - Web page: https://patentgrep.com/patent/US10873534B1 ## CPC - H04L 45/38: {Flow based routing} - H04L 45/08: {Learning-based routing, e.g. using neural networks or artificial intelligence} - H04L 45/742: {Route cache; Operation thereof} - H04L 47/11: Identifying congestion - H04L 47/2483: involving identification of individual flows - H04L 67/568: Storing data temporarily at an intermediate stage, e.g. caching - H04L 41/22: comprising specially adapted graphical user interfaces [GUI] - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Some embodiments provide a data-plane forwarding circuit that can be configured to learn about a new message flow and to maintain metadata about the new message flow without first having a control plane first configure the data plane to maintain metadata about the flow. To perform its forwarding operations, the data plane includes several data message processing stages that are configured to process the data tuples associated with the data messages received by the data plane. In some embodiments, parts of the data plane message-processing stages are also configured to operate as a flow-tracking circuit that includes (1) a flow-identifying circuit to identify message flows received by the data plane, and (2) a first set of storages to store metadata about the identified flows. ## Claims ### Claim 1 A network forwarding element comprising a data-plane forwarding circuit comprising: a plurality of programmable message processing stages, wherein at least one of the programmable message processing stages is to implement a flow-learning circuit and at least one of the programmable message processing stages is programmed to perform data message forwarding operations in order to forward data messages and wherein to implement the flow-learning circuit, the at least one of the programmable message processing stages is to collect metadata regarding a received flow before a control-plane has requested collection of metadata for the received flow and prior to passing attributes about the received flow to the control-plane in order to cause the control-plane to configure the flow-learning circuit to store or not store metadata about the flow. ### Claim 2 (depends on 1) The network forwarding element of claim 1, wherein the control-plane is to operate on a remote server. ### Claim 3 (depends on 1) The network forwarding element of claim 1, comprising the control-plane coupled to the data-plane forwarding circuit. ### Claim 4 (depends on 1) The network forwarding element of claim 1, wherein the flow-learning circuit is to learn about message flows without previously having been configured with identifiers of the message flows. ### Claim 5 (depends on 1) The network forwarding element of claim 1, wherein the metadata comprises one or more of: message header values, media access control (MAC), five tuple identifiers, message payload values, counters, flags, average delay experienced by the messages, average queue depth experienced by the messages, byte count of the message payloads, or average byte count of the message payloads. ### Claim 6 (depends on 1) The network forwarding element of claim 1, wherein the metadata includes values extracted from data message header values, the header values including at least one of layer 2 header values, layer 3 header values, and layer 4 header values. ### Claim 7 (depends on 1) The data-plane forwarding circuit of claim 1, wherein the metadata includes statistics regarding the message flows processed by the data-plane forwarding circuit. ### Claim 8 (depends on 1) The network forwarding element of claim 1, wherein the data-plane forwarding circuit is to receive a configuration from the control-plane to not store metadata about the flow that comprises an indication to filter the flow from metadata storage. ### Claim 9 (depends on 1) The network forwarding element of claim 1, wherein the flow comprises one or more of: a flow associated with a tenant in a multi-tenant datacenter, a flow with a virtual network identifier in its headers, a flow from a source machine, a flows to a destination machine, a flow that uses a particular layer port, or a flow using particular protocols. ### Claim 10 (depends on 1) The network forwarding element of claim 1, wherein the network forwarding element comprises a network interface card, switch, or a router. ### Claim 11 (depends on 1) The network forwarding element of claim 1, comprising a storage to store a flow identifier for a flow, wherein: when the storage stores the flow identifier of a received message, the storage is to output a map-storage identifier to identify a location in a mapping storage that stores a metadata storage identifier for the message's flow and the metadata storage identifier is retrieved from the mapping storage in order to store metadata; and when the storage does not store the flow identifier of the received message, the flow identifier is stored in the storage, a map-storage identifier is output from the storage, a metadata-storage identifier for the message's flow is generated, the metadata storage identifier is stored in the mapping storage at a location specified by the map-storage identifier. ### Claim 12 (depends on 11) The network forwarding element of claim 11, wherein the data-plane forwarding circuit comprises a plurality of stateful processing units, the stateful processing units comprise a plurality of stateful storages, and the mapping storage and the storage are implemented by a plurality of stateful processing units and their associated stateful storages. ### Claim 13 A computer-implemented method comprising: using a set of programmable message processing stages to implement flow-learning circuit to learn about received messages in a flow by: collecting metadata regarding a flow and storing metadata regarding the flow prior to configuration by a control-plane to store metadata associated with the flow and not storing metadata regarding the flow based on the control-plane indicating to not store metadata for the flow; and using a set of programmable message processing stages to perform data message forwarding operations to forward received messages. ### Claim 14 (depends on 13) The method of claim 13, wherein the flow-learning circuit learns about message flows without previously having been configured with identifiers of the message flows. ### Claim 15 (depends on 13) The method of claim 13, wherein the metadata comprises one or more of: message header values, media access control (MAC), five tuple identifiers, message payload values, counters, flags, average delay experienced by the messages, average queue depth experienced by the messages, byte count of the message payloads, or average byte count of the message payloads. ### Claim 16 (depends on 13) The method of claim 13, wherein the metadata comprises one or more of: includes extracted from data message header values, the header values including at least one of layer 2 header values, layer 3 header values, and layer 4 header values. ### Claim 17 (depends on 13) The method of claim 13, wherein the metadata includes statistics regarding the message flows processed by a data-plane forwarding circuit. ### Claim 18 (depends on 13) The method of claim 13, comprising receiving a configuration from the control-plane to not store metadata about the flow that comprises an indication to filter the flow from metadata storage. ### Claim 19 (depends on 13) The method of claim 13, wherein the flow comprises one or more of: a flow associated with a tenant in a multi-tenant datacenter, a flow with a virtual network identifier in its headers, a flow from a source machine, a flows to a destination machine, a flow that uses a particular layer port, or a flow using particular protocols. ### Claim 20 (depends on 13) The method of claim 13, comprising: based on storage of a flow identifier of the received message, outputting a map-storage identifier that identifies a location in the mapping storage that stores a metadata storage identifier for the message's flow and using the metadata storage identifier in order to store metadata and based on non-storage of the flow identifier of the received message, storing the flow identifier, outputting a map-storage identifier, generating a metadata-storage identifier for the message's flow, and storing the metadata storage identifier at a location specified by the map-storage identifier. ### Claim 21 A non-transitory computer-readable medium comprising instructions stored thereon, that if executed by one or more processors, cause the one or more processors to: use a set of message processing stages perform a flow-learning that is to: collect metadata regarding a received flow before a control-plane has requested collection of metadata for the received flow and prior to passing attributes about the received flow to the control-plane in order to cause the control-plane to control whether to store or not store metadata about the flow. ### Claim 22 (depends on 21) The computer-readable medium of claim 21, comprising instructions stored thereon, that if executed by one or more processors, cause the one or more processors to: learn about message flows without previously having been configured with identifiers of the message flows. ### Claim 23 (depends on 21) The computer-readable medium of claim 21, wherein the metadata comprises one or more of: message header values, media access control (MAC), five tuple identifiers, message payload values, counters, flags, average delay experienced by the messages, average queue depth experienced by the messages, byte count of the message payloads, or average byte count of the message payloads. ### Claim 24 (depends on 21) The computer-readable medium of claim 21, wherein the metadata includes values extracted from data message header values, the header values including at least one of layer 2 header values, layer 3 header values, and layer 4 header values. ### Claim 25 (depends on 21) The computer-readable medium of claim 21, wherein the metadata includes statistics regarding the message flows processed by a data-plane forwarding circuit. ### Claim 26 (depends on 21) The computer-readable medium of claim 21, comprising instructions stored thereon, that if executed by one or more processors, cause the one or more processors to: receive a configuration from the control-plane to not store metadata about the flow that comprises an indication to filter the flow from metadata storage. ### Claim 27 (depends on 21) The computer-readable medium of claim 21, wherein the flow comprises one or more of: a flow associated with a tenant in a multi-tenant datacenter, a flow with a virtual network identifier in its headers, a flow from a source machine, a flows to a destination machine, a flow that uses a particular layer port, or a flow using particular protocols.