Loaded with the new hardware that Microsoft has rolled out across its long range of data centers, the networking between virtual 
machines in Microsoft Azure is going to get faster in a significant manner. Last week company made an announcement of deploying the millions of FPGAs (Field Programmable Gate Arrays) throughout 15 countries in five different continents. As a part of the first party Microsoft services, these chips are accelerating networking on the company’s Azzure cloud platform.
Along with boosting networking speed, the FPGAs also speeds up the machine learning tasks and many other key cloud functionalities. Microsoft hasn’t said exactly what the contents of the boards include, other than revealing that they hold an FPGA, static RAM chips and hardened digital signal processors.
The deployment of programmable hardware was an essential step to be taken by Microsoft in order to overcome the consistent slowing down the CPU speeds. FPGAs help in increasing the speed of processing power for the particular task that they’ve been configured to work on and reducing the time for the tasks like managing the network flow traffic or text transfer.
Networking Acceleration is the latest feature that FPGA deployment enables and can provide the users speeds as high as 25Gbps and latency of about 100 microseconds, for no extra charge. Azure CTO Mark Russinovich said using the FPGAs was key to helping Azure take advantage of the networking hardware that it put into its data centers. While the hardware could support 40Gbps speeds, actually moving all that network traffic with the different software-defined networking rules that are attached to it took a massive amount of CPU power.
“That’s just not economically viable,” he said in an interview. “Why take those CPUs away from what we can sell to customers in virtual machines, when we could potentially have that off-loaded into FPGA? They could serve that purpose as well as future purposes, and get us familiar with FPGAs in our data center. It became a pretty clear win for us.” The project is the brainchild of Doug Burger, a distinguished engineer in Microsoft Research’s New Experiences and Technologies (NExT) group.
Microsoft is not the solo practitioner engaged in turning custom silicon in such a functionality. Earlier this year Google unveiled a Tenser processing Unit that’s supposed to accelerate the similar machine learning task in its cloud. This is a sort of an ASIC (Application Specific Integrated Circuit), a purpose built chip. Now the question arises that why Miicrosoft choose FPGAs while Google preferred ASICs over FPGAs due it its speed and efficiency superioties?
The industry moves far too quickly for him to be confident a particular ASIC will do what needs to be done over time, Burger said. While using only the reprogrammable hardware in an FPGA wouldn’t be great for performance, the hardened SRAM and DSP chips on the FPGA board can speed up certain applications, shrinking the performance gap. “I’m not comfortable locking the control path down for three years and saying ‘I know what to do now,'” Burger said.
Right now, Accelerated Networking is only available for DS15v2 instances in Azure’s West Central U.S. and Western Europe regions. It’s only compatible with Windows Server 2012 R2 and Windows Server 2016 Technical Preview 5, though Microsoft plans to make it work with Linux instances soon.
As per the predictions, the future Accelerated Networking service will expand to Azure’s other virtual machine types and OSs. It will be like switching from an opt-in enhancement to being a free, opt-out benefit that will improve default network speed. For future, Microsoft is planning to put FPGAs to put in use of machine learning applications.
“This is going to be a journey for how we expose this capability to customers,” Russinovich said. “I think the first thing that we’re talking about doing is [deep learning] where we train the models and then let customers run them on CPUs or GPUs in our data center. In the future, they’ll be able to run the scoring on FPGAs, and potentially also train models themselves if they want to on the FPGAs. But we’re a ways off.”
For Burger, one of the biggest questions will be what the right mix of FPGAs and CPUs is inside an Azure datacenter. Even though Microsoft has hundreds of thousands of FPGAs already deployed, they aren’t enough to meet the company’s needs as more teams start using them.
“The CPUs are important and will continue to be important for all the software and all these products and services we have,” he said. “But I think for applications, the big breakthrough at scale is going to come from non-CPU technologies.”
Article By Bharti Amlani
