The Routing Over Large Clouds Mailing List Archive by date

[Curtis Villamizar <curtis@ans.net>]

I think I had an action item from the last ROLC meeting to document
the route loop problem associated with NHRP and multihomed sites.

There are a number of scenarios.

The simplest occurs if there is no redirect.  There are scaling
questions associated with sending redirects from routers that are very
active on the NBMA fabric and well connected off the NBMA fabric.
active on the NBMA fabric and well connected off the NBMA fabric.

Consider a router A that advertises primary.  Router B advertises
Consider a router A that advertises primary.  Router B advertises
secondary, router C advertises tertiary.  If the route to A goes down,
secondary, router C advertises tertiary.  If the route to A goes down,
then A points at B.  If the route to B goes down, then B points to C.
then A points at B.  If the route to B goes down, then B points to C.
Redirects would solve this.

There are also cases that redirects would not solve, even if the
redicts could be made scalable.  In this case, sending the prefix and
prefix length of the route being redirected to would solve this.  But
an additional route change can also make create a loop.

Consider router A advertising x.y/10, and routers B and C are
Consider router A advertising x.y/10, and routers B and C are
advertising x/8, router B is primary for x/8, router C is secondary
for x/8, and router B is also secondary for x.y/10.  If the connection
between x.y/10 and router A goes, router C redirects to router B.
between x.y/10 and router A goes, router C redirects to router B.
Router B then loses the route to x.y/10, but still has a route to x/8.
It sends the traffic toward x/8.  Somewhere along the line behind the
NBMA the advertisement of x.y/10 still present at router C directs the
NBMA the advertisement of x.y/10 still present at router C directs the
traffic toward router C since the route is more specific.  Router C
sends to router B.  Router B will not redirect since it has a route to
x/8 that is perfectly usable for these packets.  Router C will not
withdraw it's route from off the NBMA since router A redirected C
withdraw it's route from off the NBMA since router A redirected C
toward B.

The above scenario is then OK if A sends a prefix/length tuple of the
The above scenario is then OK if A sends a prefix/length tuple of the
route being redirected to at B.  If C advertises x/8, then B will
not route x.y/10 traffic toward C as long as B has a better route.
Somewhere where x/8 was aggregated the x.y/10 traffic will get dropped
as it should.  The problem then exists when x/8 goes down.  If B sees
a secondary route toward C, B will strat sending off NMBA toward C.  C
a secondary route toward C, B will strat sending off NMBA toward C.  C
will send through the NBMA toward B.
will send through the NBMA toward B.

This last route loop cannot be solved by including BGP AS path
This last route loop cannot be solved by including BGP AS path
attributes in NHRP unless redirects are sent each time an AS path
attributes in NHRP unless redirects are sent each time an AS path
changes to anyone that might be using the route.  This would require
knowing which VC knows the old route (explosion of state) or sending
the redirect on all VCs (explosion of redirects for each path change
in each route).

The bottom line is that if you are a router with connections on the
NBMA and connections off the NBMA, you need to run a routing protocol
NBMA and connections off the NBMA, you need to run a routing protocol
and never advertise anything learned by NHRP off the NBMA.
and never advertise anything learned by NHRP off the NBMA.

Need a picture?  If it is going into the NHRP draft or other document,
I'll do it in latex eepic or troff pic.  Otherwise, I'll do a quick
ascii if anyone is having trouble following the above descriptions.

I don't think there are any errors in the scenarios above, but it's
certainly not impossible.

Curtis