Linode NextGen: The Network
This is the first of a series of blog posts about an effort we’re calling Linode: NextGen. In the coming days, we’ll tell you about other improvements and changes, but today we want to let you know about network upgrades.
We’re spending $1 million making our network faster. Way faster. Cisco Nexus 7000 routers. Cisco Nexus 5000 switches with Nexus 2000 Fabric Extenders. Linode outbound network cap increased 5x. Outbound monthly transfer quota increased 10x.
We’re upgrading our entire network, in all six datacenters. Everything is being replaced, from top-level routers right down to the switches the host machines connect to. We’ve been able to do this with zero downtime, and expect to complete the upgrades within the next few weeks.
Why do all of this? Quite simply, we’ve outgrown our old network architecture and needed something that could:
- deal with our scale;
- greatly improve throughput;
- decrease latency; and
- add redundancy at access/host layer
We chose the Cisco Nexus 7000 series routers because these are the only devices Cisco offers that have large enough MAC address and routing tables to handle our requirements. For our new access layer switches, we decided on Cisco Nexus 5000 series switches with Nexus 2000 series Fabric Extenders for top-of-rack connectivity.
The entirety of our network stack in each datacenter has redundancy built in, from top to bottom. Each host is connected to two separate fabric extenders with bonded gigabit links. This not only provides redundancy in the case of a NIC failure, but traffic to the host is load-balanced across its NICs.
Each fabric extender is connected to a pair of Cisco Nexus 5000 switches using 10 Gbps links, load-balancing its traffic to each switch. The load-balancing here means that the effective throughput between the pair of fabric extenders and pair of switches is 40 Gbps.
Each pair of 5k switches is connected in a similar fashion to our two routers in the datacenter: four 10 Gbps links between them, load-balanced, again with a total of 40 Gbps of throughput. Finally, each router has multiple 10 Gbps links to our upstream provider, also is capable of failover and load-balancing.
We’re using the best switching equipment Cisco sells. The larger network capacity greatly increases our resiliency to DoS attacks, and the load-balanced and redundant configuration means hardware failures won’t affect connectivity.
We’ve also upgraded the default outbound Linode network limit from 50 Mbits to 250 Mbits/sec. All you need to do is to reboot to receive this five-fold increase. Remember that this is a limit on outbound traffic only. Inbound is not limited.
To top things off we’ve increased the amount of outbound transfer included with all plans by 1,000%. That’s right, 10 times the included transfer!
- Linode 512 upgraded from 200GB to 2000GB (2TB)
- Linode 1G upgraded from 400GB to 4000GB (4TB)
- Linode 2G upgraded from 800GB to 8000GB (8TB)
- Linode 4G upgraded from 1600GB to 16000GB (16TB)
- Linode 8G upgraded from 2000GB to 20000GB (20TB)
As before, all inbound transfer is free.
In the next few days we’ll tell you about parts 2 and 3 of Linode: NextGen. We think you’ll find them just as exciting. Enjoy!
Well, THAT’s a nice day-early birthday present. Thanks!
Pedantic mode: isn’t that a 900% increase, for a new total of 1000% of the original? Either way, pretty awesome!
Bob, can’t you do FreeBSD using pvgrub?
Wow, thanks!
WP engine customer saying thanks and thinking it’s been a benefit me to. I have a friend that wants to upgrade shared hosting to something faster I’ve given him some decent alternatives and I’m wondering if anyone has any great ideas for under 20 bucks that somebody used to shared hosting could run?
Wow.
I love you, Linode.
Wow! Very nice of you to do this for us. This reminds me of why I love Linode so much.
@Shane (3/7)
Yea… Mine looks nearly as bad.
http://puu.sh/2eXtV
I logged in to my manager and my jaw hit the floor when I seen the transfer. Awesome!!
Linode! Can you please tell me why you are so awesome? Eagerly waiting for the next piece of the cake 😀
Thanks a lot!
Thanks for the upgrade!!
Hi,
Excellent upgrade! I’m looking forward to the next surprises!
Blake
This is really great! Thanks for the extraordinary Support Team who are always on top of the Support Tickets.
Linode always has something to offer every other quarter. Part 1 of NextGen is awesome!
What will be even more awesome in NextGen Part 2 & 3?
– Bigger Plans with More Cores/CPUs (today its at 4)?
– SSD Drives for better IO?
– More RAM?
– Built in Firewall?
– Elastic IP / Global Load Balancer across the DCs?
– and may be pre-configured MySQL like DBs for HA n Replication
Would be really great to see Digital Ocean like hardware specs/configurations – specifically CPUs & SSD drives.
Cool! I wish they also increased the amount of RAM included in each plan
Awesome! Love Linode.
This is great – well done linode!
I’m running the exact same configuration in my data centers and it’s been very reliable. Just make sure you pick a good code base and have all the EPLD updates done before you go into production.
I’d be interested in learning what you’re using for a border router.
Thanks!
woah! Great! Linode is really always on top of the line. keep it up!
Gonna be very hard filling up this bar
http://i.imgur.com/1bazgLG.png
Solid improvements Linode, very happy with service so far… keep up the good work and you’ve got a customer for life!
Thanks,
Paul
I guess the next part maybe upgrading the hard drive to hybrid type (HDD+SSD).
Nice move! Great thanks folks!
When part 2 & 3 would come??
You guys are the BEST! Thanks!!
i bought 512mb ram plan for one year. using it almost for one month. everything was good until yesterday. starting to saw sudden crashes and server totally stops responding until initiate reboot.
i was just thinking to move mediatemple vps because they are offering 2x more ram with same price but unfortunately their servers are located only in us.
also saw the commercials for digitalocean ssd for storage and again looks cheaper but their promises looks very unrealistic to me. i think i should wait and see how serious they are.
in a nutshell, i would love to see some RAM upgrade and probably bandwith was not the main concern for majority of users.
Linode rocks.
I’ll have to second the opinions of many commenters here and say that RAM is probably more of a concern for many of us (including myself!).
I’m having to spend ‘extras’ each month just so that I can configure a good server, I hope to see RAM increased in the near future to match the competitors.
In my eyes this is the only think that’s niggling me at the moment with Linode…
Thanks,
Paul
You are the best! Thank you!
Out of curiosity, do Linode staff actually read these comments? Paul
Paul: Yup.
Crossing my fingers, hoping the next free upgrade involves storage 😀
Go Linode!
I never understood the RAM and Storage ‘extras’ from a customer view. It is cheaper to upgrade to a larger Linode. It requires a reboot/resize either way.
Great work linode! You guys are great!
Great news ! thanks a lot
Linode, this is insanely sick!!!! You just equipped superman with the Iron Man Suit. UNBEATABLE!!!
I rarely have to log in to linode as the service is soo rock solid I rarely need to.
However I was actually checking as I am going to change how I use my VPS and feared my Linode will not be able to handle it… I am blown away by your free upgrade (which will indeed handle what I want to do) and all I can say is WOW! Thank you, I get to stay!
Amazing!!, that´s why I moved all from GoDaddy to Linode…you rock guys!!!
Linode is the only serious player in the market, you certainly beat everybody else. Thanks!
Linode Team – Just sending you guys some love – you’re so awesome.
Thanks for such a great service – you’re an example to others of how to get it right….
Wow, Made my day. Love it Thank you guys for being awesome
@Dash, I looked at the offerings on DO, Ramnode, SSDVPS, lowendbox..
These folks are running RAID10 on Intel 520 SSDs (consumer grade SSD or equivalent), I don’t think that’s a very good idea, I don’t even run RAID0 on my HDD setup (I run RAID1).
The prices there are rock bottom but I guess I’ll find out why when an entire host goes down from multiple SSD failures (If I hosted with them I’d be sharing a VPS with write-happy people), and I have to tell my client has been down for 3 days because I can’t get ahold of anybody.
They also seem like they’re almost fly-by-night, but at least DO seems to be totally domestic and has photos of their peeps.
SSD for caching I like, such as with ZFS l2arc; but as the main drive, and shared with other people? NOOOOOOOO, just mho