***Updated July 5th***
This is the question I asked myself today and of course I couldn’t find this documented anywhere. I’m assuming there is no guarantee and it probably depends on current utilization, etc. If I’m wrong, someone please point me to the documentation that states the available speed. I primarily looked here and here.
So I set up two Windows 2016 D4s v3 instances, one in Central US and one in East US 2, which are paired regions.
If you don’t know what peering is, it essentially lets you to easily connect two different Azure virtual networks. Peering is very easy to setup, just make sure you configure it from both Virtual Networks, I made that mistake at first. Once it is configured properly it will look something like this.
I then downloaded iPerf3 on each of the servers and began my testing. At first I had some pretty disappointing results.
But then upon doing some research, I found that running multiple threads and increasing the window size reports a more accurate measurement of the available bandwidth. I tried a few different setting and seemed to max at at just about 1.9 Gbps on average, much better than 45 Mbps!
The client parameters I used to produce the best results are as follows:
iperf3.exe -c 10.0.3.4 -w32M -P 4 -t 30
A sample of that output looks something like this.
- - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 2.00-3.00 sec 34.1 MBytes 286 Mbits/sec [ 6] 2.00-3.00 sec 39.2 MBytes 329 Mbits/sec [ 8] 2.00-3.00 sec 56.1 MBytes 471 Mbits/sec [ 10] 2.00-3.00 sec 73.2 MBytes 615 Mbits/sec [SUM] 2.00-3.00 sec 203 MBytes 1.70 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 3.00-4.00 sec 37.5 MBytes 315 Mbits/sec [ 6] 3.00-4.00 sec 19.9 MBytes 167 Mbits/sec [ 8] 3.00-4.00 sec 97.0 MBytes 814 Mbits/sec [ 10] 3.00-4.00 sec 96.8 MBytes 812 Mbits/sec [SUM] 3.00-4.00 sec 251 MBytes 2.11 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 4.00-5.00 sec 34.6 MBytes 290 Mbits/sec [ 6] 4.00-5.00 sec 24.6 MBytes 207 Mbits/sec [ 8] 4.00-5.00 sec 70.1 MBytes 588 Mbits/sec [ 10] 4.00-5.00 sec 97.8 MBytes 820 Mbits/sec [SUM] 4.00-5.00 sec 227 MBytes 1.91 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 5.00-6.00 sec 34.5 MBytes 289 Mbits/sec [ 6] 5.00-6.00 sec 31.9 MBytes 267 Mbits/sec [ 8] 5.00-6.00 sec 73.9 MBytes 620 Mbits/sec [ 10] 5.00-6.00 sec 86.4 MBytes 724 Mbits/sec [SUM] 5.00-6.00 sec 227 MBytes 1.90 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 6.00-7.00 sec 35.4 MBytes 297 Mbits/sec [ 6] 6.00-7.00 sec 32.1 MBytes 269 Mbits/sec [ 8] 6.00-7.00 sec 80.9 MBytes 678 Mbits/sec [ 10] 6.00-7.00 sec 78.5 MBytes 658 Mbits/sec [SUM] 6.00-7.00 sec 227 MBytes 1.90 Gbits/sec
I saw spikes as high as 2.5 Gbps and lows as low as 1.3 Gbps.
So I received some feedback from @jvallery that I must try out.
First thing I did was bump up my existing instances to D64sv3 and used -P 64. I saw a significant increase
iperf3.exe -c 10.0.3.4 -w32M -P 64 -t 30 [SUM] 0.00-1.00 sec 2.55 GBytes 21.8 Gbits/sec
I then spun up some F72v2 instances as suggested and I saw even better results.
iperf3.exe -c 10.0.2.5 -w32M -P 72 -t 30 [SUM] 0.00-1.00 sec 2.86 GBytes 24.5 Gbits/sec
I’m not well versed enough in Linux to warrant me spending a bunch of extra money fumbling my way through configuring this on Linux, but suffice it to say there seems to be a reasonable amount of bandwidth available between Azure regions when using peered networks.
If someone wanted to repeat this test using Linux as @jvallery suggested I’ll be glad to post your results here!
Using these two peered networks I am helping a client address SQL Server disaster recovery using SIOS DataKeeper to asynchronously replicate SQL data between regions for disaster recovery.
In this particular scenario we were measuring a RPO measured in milliseconds. As you’ll see in the video below, during a DISKSPD test meant to simulate a typical SQL Server workload the RPO was <1 second.
I’d love to hear from you regarding your experience regarding any network speed you measure in Azure and how you are using peered networks in Azure.
4 thoughts on “What is the network speed between Azure regions connected with Virtual Network Peering?”
Have you ever run this test without peering, via public IPs and whitelisting (if necessary)? I would assume the peering doesn’t effect the throughput, just the accessibility/security between the processes in the different regions.
No, I haven’t. If you give it a try be sure to respond back to let us know how it goes.
Hi, I ran your tests over the public internet and got roughly the same results in terms of bitrate: 25Gbits/sec. However, running them over Azure’s global Vnet peering backbone, I could only reach around 8Gbits/sec. Any thoughts on why that may be? I used the exact same settings on the iperf3 client. I wonder if there can be some bandwidth limits on peering that I am not aware of.
Not sure, I haven’t thought about this in a long time. I have never run into bandwidth issues between regions, so I’ve been less concerned over time.