# US 12,009,987 B2 · Methods to support dynamic transit paths through hub clustering across branches in SD-WAN > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,009,987 B2 - Document ID: US12009987B2 - Status: Granted - Kind: B2 - Application number: 17/735,029 - Filed: 2022-05-02 - Published: 2022-11-03 - Granted: 2024-06-11 - Earliest claimed: 2021-05-03 - Assignees: VMware, Inc. - Applicants: VMware LLC - Inventors: Navaneeth Krishnan Ramaswamy - Web page: https://patentgrep.com/patent/US12009987B2 ## CPC - H04L 41/122: of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV] - H04L 12/4633: {Interconnection of networks using encapsulation techniques, e.g. tunneling} - H04L 12/4641: {Virtual LANs, VLANs, e.g. virtual private networks [VPN] (LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272)} - H04L 41/0893: Assignment of logical groups to network elements - H04L 41/0895: Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements - H04L 63/0272: {Virtual private networks} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Some embodiments provide a method of transmitting data in a logical network that includes multiple hubs in a hub cluster and multiple branches. Each branch connects to a hub of the cluster through a virtual private network (VPN) tunnel. The method is performed by a network controller. The method assigns one of the hubs as a master hub. The method then sends a command to each of the other hubs in the hub cluster to establish a VPN tunnel between the other hub and the master hub. The method then advertises, to the other hubs, routes between the other hubs through the master hub. Each branch, in some embodiments is connected to only one hub in the hub cluster. ## Claims ### Claim 1 A method of securely passing data messages through a software defined wide area network (SD-WAN), the method comprising: assigning a particular hub router of a hub cluster comprising a plurality of hub routers located in a set of one or more datacenters as a master hub router to serve as the hub router for all of the other hub routers in the hub cluster, the plurality of hub routers implementing the SD-WAN with a plurality of edge routers each located at one of a plurality of sites, the SD-WAN serving as a network that connects the machines operating at the plurality of sites to each other and to the datacenter set; sending a command to each of a plurality of other hub routers in the hub cluster to establish a VPN (virtual private network) tunnel between the other hub router and the master hub router; and advertising, to the other hub routers, routes between the other hub routers through the master hub router, said other hub routers comprising first and second hub routers, said other hub routers using said routes when the first hub router (i) connects to a first site through a first edge router of the first site but not to a second site that is connected to the second hub router that connects to the second site through a second edge router of the second site and (ii) needs to securely forward data messages received from the first site through the first edge router to the second site through the master hub router, the second hub router, and the second edge router. ### Claim 2 (depends on 1) The method of claim 1, wherein the data messages are forwarded through the first hub router. ### Claim 3 (depends on 1) The method of claim 1, wherein the master hub router acts as a transit node for end-to-end VPN tunnels from the first site to the second site. ### Claim 4 (depends on 3) The method of claim 3, wherein data sent between the first site and the second site does not leave a VPN tunnel to pass through an underlying network outside of the VPN tunnels. ### Claim 5 (depends on 1) The method of claim 1, wherein each site of the plurality of sites is connected to only one hub router in the hub cluster. ### Claim 6 (depends on 1) The method of claim 1, wherein at least two sites of the plurality of sites are directly connected to the master hub router, wherein each site is connected to the master hub router through a VPN tunnel. ### Claim 7 (depends on 1) The method of claim 1, wherein no sites are directly connected to the master hub router. ### Claim 8 (depends on 1) The method of claim 1, wherein the master hub router is a first master hub router of the hub cluster, the method further comprising assigning a second hub router a second master hub router of the hub cluster. ### Claim 9 (depends on 8) The method of claim 8 further comprising: sending a command to each of a plurality of other hub routers in the hub cluster to establish a VPN tunnel between the other hub router and the second master hub router; for a first plurality of pairs of sites, each of the pair assigned to a different hub router than the other of the pair, assigning routes between the pairs of sites that pass through the first master hub router; and for a second plurality of pairs of sites, each of the pair assigned to a different hub router than the other of the pair, assigning routes between the pairs of sites that pass through the second master hub router. ### Claim 10 (depends on 9) The method of claim 9, wherein the assignment of routes is based on loads on at least one of the first master hub router and the second master hub router. ### Claim 11 (depends on 9) The method of claim 9, wherein a first route from a particular site of a third hub router to a site of a fourth hub router is assigned to pass through the first master hub router and a second route from the particular site of the third hub router to a site of a fifth hub router is assigned to pass through the second master hub router. ### Claim 12 (depends on 1) The method of claim 1, wherein using the master hub router that has VPN tunnels to the other hub routers of the hub cluster reduces a number of VPN tunnels between hub routers because using the master hub router does not require each hub router of the hub cluster to set up a VPN tunnel to each other hub router of the hub cluster. ### Claim 13 A non-transitory machine readable medium storing a program which when implemented by at least one processing unit securely passes data messages through a software defined wide area network (SD-WAN) that is implemented with a plurality of hubs and a plurality of edge forwarding elements at a plurality of sites connected through the SD-WAN, each hub serving as a focal point for receiving and transmitting data messages between two edge forwarding elements at two sites, the program comprising sets of instructions for: assigning a first hub of the plurality of hubs as a master hub to serve as the hub for all of the other hubs in a hub cluster; sending a command to each of a plurality of other hubs in the hub cluster to establish a VPN (virtual private network) tunnel between the other hub and the master hub; and advertising, to the other hubs, routes between the other hubs through the master hub, said other hubs using said routes when a first hub in the other hubs (i) connects to a first site but not a second site that is connected to second hub in the other hubs and (ii) need to forward data messages securely from the first site to the second site through the master hub and the second hub of the other hubs. ### Claim 14 (depends on 13) The non-transitory machine readable medium of claim 13, wherein the data messages are forward through the first hub of the other hubs. ### Claim 15 (depends on 13) The non-transitory machine readable medium of claim 13, wherein the master hub acts as a transit node for end-to-end VPN tunnels from the first site to the second site. ### Claim 16 (depends on 15) The non-transitory machine readable medium of claim 15, wherein data sent between the first site and the second site does not leave a VPN tunnel to pass through an underlying network outside of the VPN tunnels. ### Claim 17 (depends on 13) The non-transitory machine readable medium of claim 13, wherein each site of the plurality of sites is connected to only one hub in the hub cluster. ### Claim 18 (depends on 13) The non-transitory machine readable medium of claim 13, wherein at least two sites of the plurality of sites are directly connected to the master hub, wherein each site is connected to the master hub through a VPN tunnel. ### Claim 19 (depends on 13) The non-transitory machine readable medium of claim 13, wherein no sites are directly connected to the master hub. ### Claim 20 (depends on 13) The non-transitory machine readable medium of claim 13, wherein the master hub is a first master hub of the hub cluster, the program further comprising sets of instructions for assigning a second hub as a second master hub of the hub cluster.