Anycast inside the Data Centers

Why

Anycast services networks are advertised from multiple hosts to the data center. This enables multiple next hops for the given service, which is the advertised prefix. We may achieve load balancing, better failover, and scalability.

Solution

In order for all hosts to receive traffic, we must use ECMP (Equal Cost Multipath) for the anycast prefix. This depends on the type of fabric and the protocols used to run the data center.

EVPN & VXLAN Based:

  • Add/Receive Multiple Paths for EVPN address family: Anycast services require not only the best path but also advertising and receiving multiple paths for the same NLRI (Network Layer Reachability Information).
  • Maximum Path configuration not only for L2 EVPN address family but also on the edge switches unicast address family where hosts are connected.
  • As-Path ignore/relax for best path calculation depending on your configurations.
  • External routes are carried with EVPN route-type 5. Routes originated from the same Leaf switch require some additional tuning as they will look the same when encapsulated into route-type 5. Setting the gateway IP to the next-hop, then the host advertising the prefix may resolve the issue. But this time, you cannot use RFC5549 between the switch and the host as the gateway IP must be IPv4. We cannot use RFC5549 for anycast services. Additional BGP tunings may be required depending on the EBGP or IBGP-based configurations. BGP best path selection algorithms should be tuned for the solution.

Routed/IP fabric: Mostly, we are talking about BGP-based setups. Depending on whether EBGP or IBGP is used, problems change and may even not be possible.

      • Receiving multiple paths from the same neighborship: Spine and Leaf form EBGP neighborship. When a leaf forwards multiple paths for the same destination, a leaf that is connected to more than one anycast-advertising host, the Spine will only install a single path among them.
LEAF BGP Table – It has 4 multi paths which are locally connected.
Leaf is advertising all them to the Spine
Leaf BGP configuration
Spine receives all them but only one them is selected for multipath
Spine BGP and routing table
Spine BGP configuration

As a result all leafs will have at least a single route for the remote next hops for any cast prefix but will loose the weight of them as only one route will be selected by the spine for each Leaf’s route.

You may try IBGP but this time you will hit EBGP over IBGP selection for the edge networks. Even you may try BGP PIC commands but all them may result in very complex setups and you may require to use vendor specific configuration to overcome the problem.

We should use different address family rather than ipv4 unicast like vpnv4 or evpn for advertising the edge networks for supporting any cast services. Pure IPv4 fabric will not help us for any cast services inside the fabric. Only if you have dedicated switches for any cast services, for N-S traffic for example, like firewall, external load balancers, and etc which are equally connected to multiple switches ( they will connected to same switches with same number of links) which will result in ECMP equal to the number of switches, where each switch will do its own ECMP towards the hosts.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.