If you are patient you may miss the below pre while configuring ESI multihoming on Nexus switches;

If eBGP is used with VXLAN EVPN multi-homing, the administrative distance for local learned endpoints must be lower than the value of eBGP. The administrative distance can be changed by entering the fabric forwarding admin-distance distance command.

My switches are connected to SRV with SVI configured on both of them;

Switch side configurations which are same on both them;

interface port-channel9
switchport mode trunk
switchport trunk allowed vlan 100,200
ethernet-segment 9
system-mac 0000.0000.2011
spanning-tree port type edge

interface Vlan100
no shutdown
vrf member FW_ZONE_X
ip address 10.1.100.0/31
fabric forwarding mode anycast-gateway

 

When you look at the ip route table;

S1-BL2# sh ip route 10.1.100.1 vrf FW_ZONE_X
IP Route Table for VRF “FW_ZONE_X”
‘*’ denotes best ucast next-hop
‘**’ denotes best mcast next-hop
‘[x/y]’ denotes [preference/metric]
‘%<string>’ in via output denotes VRF <string>

10.1.100.1/32, ubest/mbest: 1/0
*via 192.168.1.98%default, [20/0], 1d11h, bgp-61099, external, tag 61201, segid: 100100 tunnelid: 0xc0a80162 encap: VXLAN

via 10.1.100.1, Vlan100, [190/0], 00:08:55, hmm

This causes the traffic for 10.1.100.1 loop between them as each will have route pointing to each other. Thus after lowering the admin distance of the local routes;

fabric forwarding admin-distance 19

S1-BL2# sh ip route 10.1.100.1 vrf FW_ZONE_X
IP Route Table for VRF “FW_ZONE_X”
‘*’ denotes best ucast next-hop
‘**’ denotes best mcast next-hop
‘[x/y]’ denotes [preference/metric]
‘%<string>’ in via output denotes VRF <string>

10.1.100.1/32, ubest/mbest: 1/0, attached
*via 10.1.100.1, Vlan100, [19/0], 00:00:26, hmm
via 192.168.1.98%default, [20/0], 1d11h, bgp-61099, external, tag 61201, segid: 100100 tunnelid: 0xc0a80162 encap: VXLAN

Why

Anycast services networks are advertised from multiple hosts to the data center. This enables multiple next hops for the given service, which is the advertised prefix. We may achieve load balancing, better failover, and scalability.

Solution

In order for all hosts to receive traffic, we must use ECMP (Equal Cost Multipath) for the anycast prefix. This depends on the type of fabric and the protocols used to run the data center.

EVPN & VXLAN Based:

• Add/Receive Multiple Paths for EVPN address family: Anycast services require not only the best path but also advertising and receiving multiple paths for the same NLRI (Network Layer Reachability Information).
• Maximum Path configuration not only for L2 EVPN address family but also on the edge switches unicast address family where hosts are connected.
• As-Path ignore/relax for best path calculation depending on your configurations.
• External routes are carried with EVPN route-type 5. Routes originated from the same Leaf switch require some additional tuning as they will look the same when encapsulated into route-type 5. Setting the gateway IP to the next-hop, then the host advertising the prefix may resolve the issue. But this time, you cannot use RFC5549 between the switch and the host as the gateway IP must be IPv4. We cannot use RFC5549 for anycast services. Additional BGP tunings may be required depending on the EBGP or IBGP-based configurations. BGP best path selection algorithms should be tuned for the solution.

Routed/IP fabric: Mostly, we are talking about BGP-based setups. Depending on whether EBGP or IBGP is used, problems change and may even not be possible.

• • • Receiving multiple paths from the same neighborship: Spine and Leaf form EBGP neighborship. When a leaf forwards multiple paths for the same destination, a leaf that is connected to more than one anycast-advertising host, the Spine will only install a single path among them.