This is the first of a series of blog posts about an effort we’re calling Linode: NextGen. In the coming days, we’ll tell you about other improvements and changes, but today we want to let you know about network upgrades.
We’re spending $1 million making our network faster. Way faster. Cisco Nexus 7000 routers. Cisco Nexus 5000 switches with Nexus 2000 Fabric Extenders. Linode outbound network cap increased 5x. Outbound monthly transfer quota increased 10x.
We’re upgrading our entire network, in all six datacenters. Everything is being replaced, from top-level routers right down to the switches the host machines connect to. We’ve been able to do this with zero downtime, and expect to complete the upgrades within the next few weeks.
Why do all of this? Quite simply, we’ve outgrown our old network architecture and needed something that could:
- deal with our scale;
- greatly improve throughput;
- decrease latency; and
- add redundancy at access/host layer
We chose the Cisco Nexus 7000 series routers because these are the only devices Cisco offers that have large enough MAC address and routing tables to handle our requirements. For our new access layer switches, we decided on Cisco Nexus 5000 series switches with Nexus 2000 series Fabric Extenders for top-of-rack connectivity.
The entirety of our network stack in each datacenter has redundancy built in, from top to bottom. Each host is connected to two separate fabric extenders with bonded gigabit links. This not only provides redundancy in the case of a NIC failure, but traffic to the host is load-balanced across its NICs.
Each fabric extender is connected to a pair of Cisco Nexus 5000 switches using 10 Gbps links, load-balancing its traffic to each switch. The load-balancing here means that the effective throughput between the pair of fabric extenders and pair of switches is 40 Gbps.
Each pair of 5k switches is connected in a similar fashion to our two routers in the datacenter: four 10 Gbps links between them, load-balanced, again with a total of 40 Gbps of throughput. Finally, each router has multiple 10 Gbps links to our upstream provider, also is capable of failover and load-balancing.
We’re using the best switching equipment Cisco sells. The larger network capacity greatly increases our resiliency to DoS attacks, and the load-balanced and redundant configuration means hardware failures won’t affect connectivity.
We’ve also upgraded the default outbound Linode network limit from 50 Mbits to 250 Mbits/sec. All you need to do is to reboot to receive this five-fold increase. Remember that this is a limit on outbound traffic only. Inbound is not limited.
To top things off we’ve increased the amount of outbound transfer included with all plans by 1,000%. That’s right, 10 times the included transfer!
- Linode 512 upgraded from 200GB to 2000GB (2TB)
- Linode 1G upgraded from 400GB to 4000GB (4TB)
- Linode 2G upgraded from 800GB to 8000GB (8TB)
- Linode 4G upgraded from 1600GB to 16000GB (16TB)
- Linode 8G upgraded from 2000GB to 20000GB (20TB)
As before, all inbound transfer is free.
In the next few days we’ll tell you about parts 2 and 3 of Linode: NextGen. We think you’ll find them just as exciting. Enjoy!