eBGP neighbors: sessions, states and a relationship that won't come up

BGP is the path vector exterior gateway protocol that carries the Internet's routes, built for scale and stability: it sends incremental updates and never refreshes its whole table on a timer. A BGP session runs over TCP port 179, and each peer moves through six states, Idle, Connect, Active, OpenSent, OpenConfirm and Established: first TCP comes up, then the OPENs are accepted.

In Daveth Pro (Network)9 min video · lab: eBGP

By , founder of Daveth · Updated

Opening frame of the video walkthrough of eBGP neighbors: sessions, states and a relationship that won't come up

Watch this lesson

9 min · Video walkthrough

In Daveth Pro (Network)

The full 9 min walkthrough is in Daveth Pro (Network). See what the plan opens

Upcoming practical lab

Lab E7: eBGP

What the lab covers

What this page answers

What is BGP?

The Border Gateway Protocol: a standardised path vector exterior gateway protocol, today's BGP-4 in RFC 4271, built to connect organisations on the Internet and on private networks, with scalability, flexibility and stability in mind. Read it in “4.1 What BGP is for”

What are the BGP neighbour states?

Six: Idle, Connect, Active, OpenSent, OpenConfirm and Established. Read them in two halves: getting TCP up, then getting the OPEN messages accepted, so a stuck state says which half the fault is in. Read it in the lesson

How does a BGP session form?

Over TCP port 179 between two configured peers. Directly connected peers find each other with ARP; a multi-hop session needs a route to the peer, and a default route is not enough. Read it in “4.1 What BGP is for”

4.1 What BGP is for

BGP is an exterior gateway protocol: a standardised path vector protocol, defined in RFC 1654 (today's BGP-4 is RFC 4271), built for scalability, flexibility and stability. It was designed to connect organisations, on the Internet and on private networks, and it's the protocol that carries the Internet's routes, more than 780,000 IPv4 of them when the guide was written. 02-1 placed it in the path vector family; this lesson opens it up. It's basically like the difference between a building's internal extensions and the international dialling codes: extensions get shuffled every week, country codes almost never change, and nobody wants them to.

That's why BGP prizes stability: with tables that size, one link flapping could mean recomputing thousands of routes. It sends incremental updates as things change and never refreshes the whole table on a timer. If you've ever rearranged a whole wedding seating plan because one guest swapped tables, you know why nobody wants a flap to ripple through 780,000 routes.

BGP would rather be right tomorrow than fast today, which also describes most change-control boards.

To BGP, an autonomous system (AS) is the set of routers one organisation controls, running one or more IGPs with common metrics inside, and looking consistent in routing policy to everyone outside. It doesn't even need an IGP: BGP can be the only routing protocol in an AS.

An organisation that connects to the Internet needs an AS number (ASN). They were 2 bytes, 65,535 possible; running out, RFC 4893 widened them to 4 bytes, 4,294,967,295. Two blocks are private, usable on the condition that they're never exchanged on the Internet: 64,512 to 65,534 in the 16-bit range, and 4,200,000,000 to 4,294,967,294 in the 32-bit range. Consider it your car's number plate: issued by an authority, one per car, and a private ASN is a prop plate for a film shoot, fine on a closed lot and never on a public road.

The guide's private range runs to 65,535, but IANA has reserved that one (RFC 7300). It's the last space in the car park, and it's painted as a fire lane.

IANA assigns public ASNs so they're globally unique, and asks for proof of a publicly allocated address range, proof of more than one Internet connection, and the need for a routing policy of your own, rather like a bank asking for proof of address before it opens a business account. If you can't show that, use the ASN your provider gives you. Use only an ASN from IANA, from your provider, or a private one: using another organisation's ASN without permission can cause traffic loss and "havoc on the Internet". (The guide's quiz answer gives the private 16-bit range as 64,512 to 65,535. On a quiz that offers only the guide's range, choose it, knowing its last number is the reserved one.)

BGP attaches path attributes (PAs) to every path, and they're what gives it such fine control over routing policy. RFC 4271 sorts them four ways:

ClassRecognised by every BGP implementation?Carried how
Well-known mandatoryyesin every prefix advertisement
Well-known discretionaryyesmay or may not be included
Optional transitivenostays with the route from AS to AS
Optional non-transitivenoisn't passed from AS to AS

Think about the last time you filled in a form at a new clinic. Some boxes every clinic reads and you must fill (your name); some every clinic reads but you can leave blank; and some only a specialist reads, which either travel with your file to the next clinic or stay behind. The guide calls an update's NLRI the prefix, its length and that route's attributes; RFC 4271 keeps the attributes separate from the NLRI proper.

One well-known mandatory attribute does BGP's loop prevention. AS_Path lists every AS a prefix has passed through from its source, and a router that finds its own AS in the list discards the prefix. That's roughly equivalent to a passport full of entry stamps: every AS stamps it on the way out, and an AS that finds its own stamp already there refuses entry. Figure 11-1 shows it: AS 100 advertises 172.16.1.0/24 to AS 200, which passes it to AS 400, then to AS 300, which sends it back to AS 100 with the AS_Path 300 400 200 100; AS 100 sees itself and drops it. BGP never has the full topology; like distance vector protocols, it only needs to be sure the path is loop-free.

AS 100 finds its own number in the AS_Path and bins the route: the routing equivalent of receiving your own chain letter.

BGP started out IPv4-only. RFC 2858 added Multiprotocol BGP (MP-BGP, or MBGP): an address family identifier (AFI), IPv4 or IPv6, and a subsequent AFI (SAFI), unicast or multicast, carried in the MP_REACH_NLRI and MP_UNREACH_NLRI attributes of UPDATEs (IPv6 over MP-BGP: 02-10). Each AFI/SAFI keeps its own database and configuration, so policy can differ per family over the same session, and every route advertisement carries its AFI and SAFI. It's a bit like one streaming subscription with separate profiles: one login, and a different watch list for everyone.

BGP has no hellos and can't discover neighbours: you define each one by IP address, because an inter-AS protocol's neighbours should change rarely and by agreement. It runs over TCP port 179, which handles fragmentation, sequencing and reliability; most implementations set the DF bit and rely on path MTU discovery. It acts kind of like a wedding guest list: nobody gets in by wandering past, and every guest is named in advance.

BGP doesn't do hellos. You introduce it, by IP address, to every single neighbour, like a very formal dinner party.

A BGP session is the established adjacency between two BGP routers. IGP hellos can't cross a router, so IGPs follow the physical topology; TCP can, so BGP sessions can be multi-hop. The router then needs a route to the peer in its RIB, static or from any protocol, and iBGP neighbours commonly get it from a static route or an IGP. In Figure 11-2, R1 and R2 peer across their shared link; R2 and R4 peer through R3, using static routes to each other's networks, and R3 has no idea a BGP session is passing through it. Picture yourself sending a parcel through a courier's hub: the hub handles the box and never reads the letter inside. Directly connected peers find each other with ARP; a multi-hop session needs the routing table, and a default route isn't enough.

So BGP is as much an application as a routing protocol: its routers don't have to sit in the data path to swap prefixes, but every router in the data path must know the routes traffic will use. R3 is the router that proves it in section 4.5.

There are two kinds of session:

  • iBGP: with a router in the same AS (or the same confederation). Its routes get AD 200.
  • eBGP: with a router in a different AS. Its routes get AD 20.

An eBGP route's AD of 20 beats OSPF's 110 for the same prefix, so the outsider's word beats your own IGP's. Office politics, in numbers.

iBGP matters when an AS needs several routing policies or carries transit between other ASes. In Figure 11-3, AS 65200 connects AS 65100 (at R2) and AS 65300 (at R4). An iBGP session straight from R2 to R4 isn't enough: R3, in the middle, would get the transit traffic with no route for it (Figure 11-4). Redistributing BGP into the IGP isn't the answer: IGPs can't hold 780,000 prefixes, they choose by metric alone where BGP uses several steps driven by its attributes (02-11), and they can't carry those attributes. Have you ever noticed how a plan only two people in a group chat know leaves everyone else guessing? R3 is the one guessing. The guide's fix is a full mesh of iBGP sessions between R2, R3 and R4 (Figure 11-5), which also gets past what it calls "the loop-prevention behavior of iBGP learned routes"; and a note that providers offer transit, while an enterprise shouldn't carry traffic between other ASes across the Internet. The iBGP rules themselves are beyond 3.2.c.

eBGP is the Internet's core, and four behaviours differ from iBGP:

eBGPiBGP
TTL of BGP packets1 by default, so a multi-hop attempt drops in transit255, so multi-hop works
Next hop in advertisementschanged to the advertising router's own address(beyond 3.2.c)
AS_Paththe advertiser prepends its ASN(beyond 3.2.c)
On receiptthe AS_Path loop check runs-

The configuration is the same except for one thing: the remote-as differs from the local AS. TTL 1 works like a ticket valid for one stop only, which is why an eBGP peer two hops away can't be reached with the defaults (D1 shows it). Whether IOS XE 17.9 actually sends TTL 1 on a directly connected eBGP session is unconfirmed: the guide's own Example 11-4 shows "Outgoing TTL 255" on one.

Before the rules, a prediction to hold on to: R1's hold time is 180 seconds and R2's is 30. Does the session come up, and how often do keepalives go? D1's last case answers it.

BGP uses four messages:

TypeMessageDoes
1OPENsets up the adjacency
2UPDATEadvertises, updates or withdraws routes
3NOTIFICATIONreports an error, and closes the connection
4KEEPALIVEproves the neighbour is still alive

The OPEN carries the BGP version, the sender's ASN, its hold time, its BGP identifier and optional parameters: the capabilities the two sides negotiate before peering. The hold time is the heartbeat: every UPDATE or KEEPALIVE resets it, and at zero the session is torn down, the neighbour's routes are removed and withdrawals go to the other neighbours. The two routers use the smaller of their two values; it must be at least 3 seconds, or 0 to switch keepalives off, and Cisco's default is 180 s. It works just like when you're on lifeguard duty: a swimmer ducking under is fine, but past the agreed time without seeing them, the alarm goes up.

The BGP identifier, the router ID, is a 32-bit number that identifies the router and helps prevent loops inside the AS; it's set by hand or automatically, and it must be nonzero for a neighbourship to form. KEEPALIVEs go every third of the agreed hold time, so 60 s by default, and none at all if the hold time is 0: BGP doesn't trust the TCP connection's state alone. If you scaled that down to a parking meter, it's topping up every 20 minutes on a one-hour meter: one late top-up never costs you the space.

Set the hold time to 0 and keepalives stop entirely, so the session never checks on its neighbour, which also means it never notices when the neighbour's gone.

An UPDATE advertises feasible routes, withdraws old ones, or both; an advertisement carries the NLRI with its attributes, a withdrawal just the prefix, and an UPDATE counts as a keepalive too. A NOTIFICATION goes out on an error (hold timer expired, capabilities changed, a reset requested) and closes the connection.

BGP tracks each peer in a finite-state machine with six states: Idle, Connect, Active, OpenSent, OpenConfirm, Established (Figure 11-6). Read them as two halves: first get TCP up, then get the OPENs accepted.

The first half, TCP. In Idle, BGP sees a start event, starts a TCP connection to the peer and listens for the peer's own. Every time it falls back to Idle after the first, the ConnectRetryTimer is 60 s, and it doubles with each further failure. In Connect, the three-way handshake runs; the router that opens the connection uses a random source port, and the destination is always 179. show tcp brief shows the connection (Example 11-1). Active is another attempt: on success, the router sends its OPEN, sets a 4-minute hold timer and moves on. While TCP keeps failing, the guide has the neighbour cycling between Connect and Active, and any other input in Connect sends it back to Idle. (RFC 4271's machine can also drop straight to Idle on a TCP failure, so a neighbour that never finished TCP isn't always shown in Active.)

The second half, the OPENs. In OpenSent, a router has sent its OPEN and waits for the peer's, then checks it: the BGP version matches; the OPEN's source address is the one configured for that neighbour; the ASN is the one configured; the router IDs are unique; security settings (password, TTL) are right. If everything passes, the hold time is negotiated (the lower value), a KEEPALIVE goes out unless the hold time is 0, and the state is OpenConfirm. If a check fails, a NOTIFICATION goes out and it's back to Idle; a TCP disconnect sends it to Active; anything else, to Idle. In OpenConfirm it waits for the peer's KEEPALIVE (then Established) or a NOTIFICATION or hold-timer expiry (then Idle). Established is where UPDATEs flow; every UPDATE or KEEPALIVE resets the hold timer, and a hold-timer expiry or an error sends it back to Idle.