Spine and Leaf Architecture

In a data centre much of the traffic flows between servers in different racks. Spine and leaf, also called a Clos network, is a fixed shape for that: each leaf connects to every spine, each spine to every leaf, leaves never connect to each other, nor spines to each other, and endpoints connect only to leaves. Any two leaves are one spine apart.

In Daveth Pro (Network)8 min video

By , founder of Daveth · Updated

Opening frame of the video walkthrough of Spine and Leaf Architecture

Watch this lesson

8 min · Video walkthrough

In Daveth Pro (Network)

The full 8 min walkthrough is in Daveth Pro (Network). See what the plan opens

What this page answers

What is spine and leaf architecture?

A data centre design, also called a Clos network: every leaf switch connects to every spine switch, leaves do not connect to each other, spines do not connect to each other, and servers and other endpoints connect only to leaves. Read it in “Spine and leaf”

Racks, and the traffic between them

Thursday, month nine. The fourth floor is to get a proper server room (planned): three racks, so the servers now crammed around HQD (PAY1, DHCP1 and BK1 among them) can move in, with the rest of VLAN 98 to follow. That would also settle 09.2's worry about PAY1 hanging off HQD alone. A server is a place to run applications that people reach over the network. Data centre servers are built in a rack-sized form factor, their shape and size, so many fit in one room without wasting space; each supplies processors, memory, storage and network cards, and most need no keyboard, video screen or mouse (KVM), because everyone who uses or manages it does so over the network. What Ravi has to decide is how to cable the racks' switches together.

The traditional way copies the campus. Each server is often cabled to two switches at the top of its rack, top-of-rack (ToR) switches, so it has two paths into the LAN. Each ToR switch acts as an access switch, and each is cabled to an end-of-row (EoR) switch, one per row of racks, which acts as a distribution switch and connects to the rest of the network.

But a server room's traffic isn't a campus's. On a campus, most traffic goes from users out to services. Here, the servers talk to each other: every night BK1, planned for rack 3, will copy PAY1's data from rack 1, before uploading it off-site as it has since month six. Traffic between racks wants a short path that is the same length whichever racks are involved.

Spine and leaf

Spine and leaf, also called a Clos network after one of its designers, is a fixed shape that suits it. Other parts of a network need all sorts of shapes, but a data centre can be made standard and consistent. There are spine switches and leaf switches, and five rules (for one site):

  1. Each leaf connects to every spine.
  2. Each spine connects to every leaf.
  3. Leaves don't connect to each other.
  4. Spines don't connect to each other.
  5. Endpoints connect only to leaves. Most are servers; the routers to the world outside are endpoints too.

A link can be a single cable or several in parallel. The spines and leaves together are called the fabric. Ravi's plan for three racks has a leaf in each rack and two spines, and R3's trunk (or HQD) would plug into a leaf like any other endpoint, which is how the Staff LAN would reach PAY1:

Network drawing: Spines (planned), Leaves: one per rack (planned), SP1, SP2, LF1, LF2, LF3, PAY1, DHCP1, BK1

Ravi's plan: SP1 and SP2 on top, a leaf per rack below, each leaf linked to both spines, and a server under each leaf.

Three switches, every time. BK1's leaf, LF3, has a link to each spine and nothing else in the fabric; so does PAY1's leaf, LF1. The only way between them is up to a spine and down again: leaf, spine, leaf. Any two racks are two links apart, and it doesn't matter which two.

If you've ever changed trains at a hub station, the shape will feel familiar: from any local line to any other, one change at the hub. (Where it stops: a railway usually has one hub; a fabric has several spines, any of which will do.)

The rules forbid it, and not out of fussiness. A cable from LF3 to LF1 would make that one pair of racks special, with a road of its own, and the fabric would stop being the same shape everywhere. Capacity is added with spines, or more cables on the leaf-to-spine links. Here is how four proposed cables fare:

CableAllowed?Rule
LF1 to LF3noleaves don't connect to each other
SP1 to SP2nospines don't connect to each other
BK1 to SP1noendpoints connect only to leaves
LF2 to SP3yesleaf to spine: every leaf connects to every spine

The racks don't need to phone each other directly; they all use the same exchanges.