DMVPN Phase 3 with dual hub
Replace a tunnel destination with tunnel mode gre multipoint and one interface reaches every site, but now the router must find each packet's far end. NHRP, an ARP-like protocol for NBMA networks, solves it: the hubs are next hop servers and the spokes register with them. Each hub advertises one summary, so the first packet goes via a hub, which sends an NHRP redirect.
By Arif, founder of Daveth · Updated
The full 9 min walkthrough is in Daveth Pro (Network). See what the plan opens
What this page answers
What is NHRP in DMVPN?
An ARP-like protocol that maps tunnel addresses to real public addresses on a non-broadcast network. The hubs are next hop servers and the spokes are next hop clients that register with them. Read it in “The hub is a database, and the spokes are the ones who fill it in”
How does a DMVPN Phase 3 spoke reach another spoke?
Its first packet follows the hub's summary through the hub. The hub, seeing traffic go back into the same cloud, sends the spoke an NHRP redirect, and the spoke resolves the other spoke's address to build a direct tunnel. Read it in “The hub forwards the packet and tells the sender to stop”
What routes does a DMVPN Phase 3 spoke learn?
Each hub advertises one summary, and each spoke advertises only its own LAN, so no spoke learns a specific route to another spoke: everything not its own is behind a hub. Read it in “The first packet, and the second lookup that no other interface needs”
Delete one line, and the tunnel has to work out where to send every packet
A point-to-point GRE tunnel is configured with a tunnel destination. That address has to be routable, the path to it must not run back through the tunnel, and the whole interface depends on it. Take the line away and put tunnel mode gre multipoint in its place, and one interface can now reach every other site instead of one.
That is the scaling argument, and it creates the problem this page is about. With many possible far ends and no configured destination, the router has to decide per packet which real address to send to. By the end you will be able to read a spoke's cache and its routing table together and say, for any destination, whether the traffic is about to take a shortcut or go the long way round, and when it goes the long way, which of five routers is the reason.
State 1 What actually exists
What is physically joined to what, before any tunnel is configured?

Five routers, each with one link to the internet and one public address on it. There is no circuit between any two sites. Those public addresses are the only way any of these routers can reach any other, and that layer has a name worth knowing: NBMA, non-broadcast multi-access. The name is doing real work: a network where you cannot broadcast is a network where ARP cannot help you, which is exactly why a different resolution protocol has to exist.
SPOKE1# show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
H - NHRP, G - NHRP registered, g - NHRP registration summary
o - ODR, P - periodic downloaded static route, l - LISP
+ - replicated route, % - next hop override, p - overrides from PfR
Gateway of last resort is not set
C 10.0.0.0/24 is directly connected, Tunnel0
C 192.168.1.0/24 is directly connected, GigabitEthernet2
C 203.0.113.0/24 is directly connected, GigabitEthernet1
! one connected network, and nothing else: no site can see any other site's LANThe hub is a database, and the spokes are the ones who fill it in
The protocol that solves the lookup is NHRP, and the published description says what to carry across from something you already know: it is an Address Resolution Protocol (ARP)-like protocol that dynamically maps a Non-Broadcast Multi-Access (NBMA) network. ARP maps an IP address to a MAC address on a segment you can broadcast on. NHRP maps a tunnel address to a real public address on one you cannot.
It has two roles. The hub is the next hop server, the spokes are next hop clients, and the hub maintains an NHRP database of the public interface addresses of each spoke. Read what the hub is in that sentence. Not a concentrator, not a forwarder: a place where every spoke's real address is known, because every spoke said so.
The hubs first. They need no mapping configured at all, because they are the ones being registered with.
HUB1> enable HUB1# configure terminal HUB1(config)# interface Tunnel0 HUB1(config-if)# ip address 10.0.0.11 255.255.255.0 HUB1(config-if)# ipv6 address 2001:DB8:FFFF::11/64 HUB1(config-if)# tunnel source GigabitEthernet1 HUB1(config-if)# tunnel mode gre multipoint HUB1(config-if)# ip nhrp network-id 100 HUB1(config-if)# ip nhrp holdtime 360 HUB1(config-if)# ip nhrp authentication DMVPN-KEY HUB1(config-if)# ip nhrp map multicast dynamic HUB1(config-if)# ip nhrp redirect HUB1(config-if)# end
The tunnel comes up on its source address alone: an mGRE interface does not need a far end to be up, which is the first thing that separates it from the point-to-point kind. ip nhrp holdtime 360 is the number to hold on to, because the registration interval is derived from it rather than configured: registrations go out every one third of the holdtime, so 360 seconds means one every 120, and three of them fit inside every holdtime. That is the ordinary lose two and still be fine margin.
Then the spokes, which need one thing the hubs do not.
SPOKE1# configure terminal SPOKE1(config)# interface Tunnel0 SPOKE1(config-if)# ip nhrp nhs 10.0.0.11 nbma 203.0.113.11 multicast SPOKE1(config-if)# ip nhrp nhs 10.0.0.12 nbma 203.0.113.12 multicast SPOKE1(config-if)# ip nhrp shortcut SPOKE1(config-if)# end
Those two lines are the bootstrap, and they are the only mappings in this network that a router is given rather than learns. Without them a spoke has a multipoint tunnel and no idea where to send anything, including the message that would tell it. Note that each spoke is given both hubs: a spoke registers with every server it has, continuously, and there is no primary and no standby in NHRP's sense.
State 2 SPOKE1 registers, twice
What does a hub know afterwards that it did not know before, and what does the spoke know?

The spoke sends a registration to each server over the tunnel, inside a GRE packet addressed to the public address the static mapping gave it, and each hub writes down the pair.
HUB1# show ip nhrp 10.0.0.21/32 via 10.0.0.21 Tunnel0 created 00:00:12, expire 00:05:48 Type: dynamic NBMA address: 203.0.113.21 ! the real address SPOKE1 registered from: this line is the database 10.0.0.22/32 via 10.0.0.22 Tunnel0 created 00:00:12, expire 00:05:48 Type: dynamic NBMA address: 203.0.113.22 10.0.0.23/32 via 10.0.0.23 Tunnel0 created 00:00:12, expire 00:05:48 Type: dynamic NBMA address: 203.0.113.23
Now read the same command on the spoke, and read it for what is absent.
SPOKE1# show ip nhrp 10.0.0.11/32 via 10.0.0.11 Tunnel0 created 00:00:12 Type: static ! static: this was configured, not learned NBMA address: 203.0.113.11 10.0.0.12/32 via 10.0.0.12 Tunnel0 created 00:00:12 Type: static NBMA address: 203.0.113.12
Two entries, both static, both about hubs. A spoke knows nothing about any other spoke. Every hub holds the complete directory; nobody who actually needs it has any of it.
What these outputs show, and what they leave out. Every line above is computed from the modelled network. Two things this course could not source are therefore left out rather than invented: a real device prints a Flags line on each cache entry, and priority and cluster columns on the server list, and no page reached here defines that vocabulary. The administrative distance shown later on an NHRP route is in the same position. The written lesson records the same gap.
Proves: each hub has a mapping from every spoke's tunnel address to its public address, learned from a registration, and each spoke has reached both of its servers.
Does not prove: that any spoke can reach any other spoke directly: no spoke holds a mapping for another spoke, and none has been asked for one.
The first packet, and the second lookup that no other interface needs
Routing over the tunnel is deliberately dull. Each hub advertises one summary to the spokes, each spoke advertises only its own LAN, and no spoke learns a specific route to any other spoke. So every spoke believes the same correct thing: everything that is not mine is behind a hub.
SPOKE1# show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
H - NHRP, G - NHRP registered, g - NHRP registration summary
o - ODR, P - periodic downloaded static route, l - LISP
+ - replicated route, % - next hop override, p - overrides from PfR
Gateway of last resort is not set
C 10.0.0.0/24 is directly connected, Tunnel0
D 192.168.0.0/16 [90/26880256] via 10.0.0.11, Tunnel0
! one summary, next hop a hub: this is the whole of SPOKE1's knowledge of the other sites
C 192.168.1.0/24 is directly connected, GigabitEthernet2
C 203.0.113.0/24 is directly connected, GigabitEthernet1PC1 sends a packet to PC3. SPOKE1 does the lookup any router would do and gets 192.168.0.0/16 via 10.0.0.11, out Tunnel0. Then it has to do a second one, and this is where DMVPN stops resembling ordinary routing: Tunnel0 has no destination, so a next hop of 10.0.0.11 is not yet something you can send to. It has to become a real address first.
State 3 The long way
The packet reaches PC3. How many times did it cross the internet, and why?

Twice. Spoke to hub, then hub to spoke, because the only mapping SPOKE1 held was the one it was configured with. To every router in between this is an ordinary packet from 203.0.113.21 to 203.0.113.11; the tunnel, the private addresses and the encryption are payload. That is why the NBMA layer can be anything at all (broadband, an MPLS service, four different providers) as long as those two public addresses can reach each other.
The hub takes the packet out of its GRE wrapper, looks the destination up, finds SPOKE3's LAN learned over the tunnel, and wraps it again for SPOKE3's public address. Nothing is broken and nothing is slow enough to complain about. It is simply going a long way round.
HUB1# show ip route 192.168.3.10
Routing entry for 192.168.3.0/24
Known via "eigrp 1", distance 90, metric 26880256
Routing Descriptor Blocks:
* 10.0.0.23, from 10.0.0.23, via Tunnel0
! the hub does know where SPOKE3 is: it learned the LAN over the tunnel
Route metric is 26880256, traffic share count is 1Proves: the path works end to end, and the hub is forwarding between two spokes.
Does not prove: that this is how it will stay. The next thing that happens is caused by exactly this: a hub that finds itself forwarding between two spokes.
The hub forwards the packet and tells the sender to stop
Having forwarded it, HUB1 notices something about itself: it is not the exit point, and the traffic it just handled is going back out into the same cloud it came from. So it sends an NHRP redirect to SPOKE1. There is something quietly excellent about a device whose main contribution here is to say not me.
SPOKE1 reacts in two steps, and the first one is easy to skim past. It inserts an incomplete cache entry for 192.168.3.0/24. Read the word: the spoke now knows it needs a mapping and does not have one, so the entry exists as a place to put an answer. It adds no route to its routing table, and until the answer arrives the traffic keeps going through the hub.
SPOKE1# show ip nhrp 10.0.0.11/32 via 10.0.0.11 Tunnel0 created 00:00:12 Type: static NBMA address: 203.0.113.11 10.0.0.12/32 via 10.0.0.12 Tunnel0 created 00:00:12 Type: static NBMA address: 203.0.113.12 192.168.3.0/24 via 0.0.0.0 Tunnel0 created 00:00:12, expire 00:00:03 Type: incomplete ! incomplete: the question has been asked and not yet answered
Then it asks. Notice the path the question takes: the resolution request travels over the existing tunnel to the hub, because that is still the only path SPOKE1 has to anywhere in this cloud. The one thing it is trying to escape is the one thing carrying the escape request.
SPOKE3 receives the request, and the request carried SPOKE1's public address. So SPOKE3 has what it needs to build the tunnel and it builds it, then replies across it. The end that received the question initiates, and predicting that wrong sends you to the wrong device's logs.
State 4 The short way
Same frame, same devices. What is different, and what did it cost?

One arc, and one line in each table. The incomplete entry became a real mapping with a real address on it, and, this is the part worth slowing down for, a route was added, not just a cache entry.
SPOKE1# show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
H - NHRP, G - NHRP registered, g - NHRP registration summary
o - ODR, P - periodic downloaded static route, l - LISP
+ - replicated route, % - next hop override, p - overrides from PfR
Gateway of last resort is not set
C 10.0.0.0/24 is directly connected, Tunnel0
D 192.168.0.0/16 [90/26880256] via 10.0.0.11, Tunnel0
C 192.168.1.0/24 is directly connected, GigabitEthernet2
H 192.168.3.0/24 [250/1] via 10.0.0.23, Tunnel0
! the NHRP route: more specific than the summary, so longest match takes it
C 203.0.113.0/24 is directly connected, GigabitEthernet1Both entries are in the table at once. The summary still points at the hub and is still perfectly valid; it just stopped being the longest match. Nothing was overridden or withdrawn: a more specific route simply arrived, which is the ordinary rule you already trust, doing the whole job.
Continue the lesson
The whole lesson, with its drawings and its questions. The rest of this section and 5 more after it.





