Dream host replaces VMWare SDN with Open Source For Big Savings

In a convincing example of the viability of open source networking, cloud provider Dream Host saved 70% in capital and 40% in operational costs by replacing VMware’s NSX SDN with open source alternatives.

In a presentation at the Open Networking Summit here, suppliers Cumulus Networks and Akanda – a Dream Host spin-out NFV business — said the cloud provider replaced NSX due to scaling and Layer 3 support issues. Dream Host did not speak and was not present during the presentation, but posted a blog entry on the project here last Friday.
The project involved Dream Host’s Dream Compute public cloud compute service, which is based on OpenStack and Ceph object store and file system. The core networking requirements for Dream Compute are Layer 2 tenant isolation, IPv6 and 10G+ “everywhere.”
The first generation of the DreamCompute networking infrastructure included Nicira’s NVP network virtualization software for Layer 2 isolation, and Cumulus Linux as the network operating system running on white box switches. Layer 3 requirements were not met by Nicira NVP nor by software routing vendors who did not understand cloud, said Mark McClain, Akanda CTO
he second generation of the DreamCompute network include Layer 3 capabilities in VMware NSX, which acquired Nicira, renamed the NVP product and enhanced it. But in a bake-off with the Astara open source network orchestration service for OpenStack – which was developed by DreamHost — Astara comes out on top and, with some enhancements, allows DreamCompute to scale to over 1,000 customers and thousands of VMs.
“Honestly, we expected Astara to lose this challenge,” states Jonathan LaCour, DreamHost vice president of cloud and development, in his blog. “However, Astara absolutely came out victorious, offering a significantly better experience and more reliability.”

In the third generation of the DreamCompute infrastructure, NSX was found to have scale limitations of 1,250 tenants. Open vSwitch was slow and unstable, and the software was difficult to debug and operate, the presenters said. As a result, NSX was replaced for Layer 2 isolation by hardware accelerated VXLAN in the switch and hypervisor, and by Astara for Layer 3-7 service orchestration.
Cumulus Linux remained as the physical underlay for the DreamCompute network.

Astara virtual network appliances allowed for easy scale, while VXLAN tunnels scaled “massively,” presenters said. Astara also simplified OpenStack Neutron networking deployments by requiring fewer Layer 2, DHCP and advanced services agents, and is generally easier to operate because it, VXLAN and the Linux networking stack on DreamCompute switches are “open” and familiar, presenters said.

“As far as performance and scale, DreamCompute is breaking through those limits we met with VMWare NSX,” LaCour states in his blog. “This is largely due to reductions in complexity, thanks to management and automation through OpenStack and Astara.”

VMware wouldn’t comment specifically on the DreamHost project but through a spokesperson said it is “very happy with the success” NSX has had in some of the largest OpenStack environments in the world, “as well as our track record in open networking through things like the Open vSwitch project.”

DreamHost’s project mirrors that of other cloud and Webscale providers, like Google and Facebook, that have opted to develop their own networking solutions to overcome the limitations of commercial offerings, and reduce capex and opex. That open source provides such a significant capex improvement over commercial products should perhaps come as no surprise.

But the opex reduction might be the proof point that familiar open source code, customized for specific operator requirements, is just as capable – if not more so – than commercially available, vendor-integrated products.