How to Measure VPN Throughput Using iPerf
Overview
This guide explains how to measure upstream and downstream throughput on a VPN network using iPerf.
Build a topology as shown in the diagram below
- Ensure both iPerf server and client are connected to VLANs which are participating in the VPN.
Consider the below
Layer 1 and Layer 2
- Ensure that the cables are fine, not showing any CRC/duplex error and showing connected with the expected speed. If possible change the cables.
WAN Speed.
- Confirm the ISP speed on both MXs. In the example below, the MX67 has an ISP link with 500Mbps bandwidth available on the WAN, and the MX75 has an ISP link with 400Mbps bandwidth on the WAN. This makes the maximum throughput of between the site at currently 400Mbps (the lowest of the two).
- Do the iperf test on the public iperf server when the client is directly connected to the MX.
During testing, disable all other LAN ports from the dashboard on both MXs to ensure no other devices are utilizing network resources, disable all MX features like IDS/IPS, AMP, Content filtering etc.
- Do the iperf test on the public iperf server when the client is connected to the upstream device of the MX. (bypassing MX).
MX Sizing Guide
- Check both MXs on the MX sizing Guide and Principles to see the maximum throughput for AutoVPN. The lowest AutoVPN throughput will be the maximum theoritical Throughput between the sites.
Step-by-step instructions
Step 1. Set up the iPerf server on one end of the tunnel
- Install iPerf on a client connected directly to the MX. This client acts as the iPerf server.
Step 2. Set up the iPerf client on other end of the tunnel.
- Install iPerf on a client connected directly to the MX. This client acts as the iPerf server.
Step 3. Start iperf on the server
Run the following command on the wired client acting as the server:
$ iperf -s
-s = host acting as server
Note: If using iperf v3, use the command iperf3 -s.
Step 4. Measure upstream throughput
- Start the iperf data transfer on the client to measure the throughput.
- Upstream throughput is the traffic flowing from the client to the server.
- Run the following TCP commands on the clients:
PC-A# iperf3 -c 192.168.13.3 -w 64kb Connecting to host 192.168.13.3, port 5201 [ 5] local 192.168.20.3 port 33170 connected to 192.168.13.3 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 4.02 MBytes 33.7 Mbits/sec 0 132 KBytes [ 5] 1.00-2.00 sec 3.95 MBytes 33.2 Mbits/sec 0 132 KBytes --truncated-- [ 5] 9.00-10.00 sec 4.13 MBytes 34.7 Mbits/sec 0 132 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 41.3 MBytes 34.7 Mbits/sec 0 sender [ 5] 0.00-10.03 sec 41.3 MBytes 34.5 Mbits/sec receiver
The test above shows PC-A only getting 35Mbps.
Test with higher TCP Window Sizes and incorporating 2 Parallel Streams
PC-A# iperf3 -c 192.168.13.3 -w 128kb -P 2 Connecting to host 192.168.13.3, port 5201 [ 5] local 192.168.20.3 port 33890 connected to 192.168.13.3 port 5201 [ 7] local 192.168.20.3 port 33892 connected to 192.168.13.3 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 7.88 MBytes 66.1 Mbits/sec 0 273 KBytes [ 7] 0.00-1.00 sec 8.02 MBytes 67.3 Mbits/sec 0 266 KBytes [SUM] 0.00-1.00 sec 15.9 MBytes 133 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 1.00-2.00 sec 8.40 MBytes 70.4 Mbits/sec 0 273 KBytes [ 7] 1.00-2.00 sec 8.32 MBytes 69.8 Mbits/sec 0 266 KBytes [SUM] 1.00-2.00 sec 16.7 MBytes 140 Mbits/sec 0 --truncated-- [ 7] 9.00-10.00 sec 8.15 MBytes 68.3 Mbits/sec 0 266 KBytes [SUM] 9.00-10.00 sec 16.2 MBytes 135 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 82.0 MBytes 68.8 Mbits/sec 0 sender [ 5] 0.00-10.02 sec 82.0 MBytes 68.6 Mbits/sec receiver [ 7] 0.00-10.00 sec 82.2 MBytes 68.9 Mbits/sec 0 sender [ 7] 0.00-10.02 sec 82.1 MBytes 68.7 Mbits/sec receiver [SUM] 0.00-10.00 sec 164 MBytes 138 Mbits/sec 0 sender [SUM] 0.00-10.02 sec 164 MBytes 137 Mbits/sec receiver
The test above shows PC-A getting ~138Mbps
Test with higher TCP Window Sizes and incorporating 3 Parallel Streams
PC-A# iperf3 -c 192.168.13.3 -w 256kb -P 3 Connecting to host 192.168.13.3, port 5201 [ 5] local 192.168.20.3 port 53944 connected to 192.168.13.3 port 5201 [ 7] local 192.168.20.3 port 53960 connected to 192.168.13.3 port 5201 [ 9] local 192.168.20.3 port 53970 connected to 192.168.13.3 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 12.0 MBytes 100 Mbits/sec 0 417 KBytes [ 7] 0.00-1.00 sec 12.0 MBytes 101 Mbits/sec 0 417 KBytes [ 9] 0.00-1.00 sec 12.0 MBytes 100 Mbits/sec 0 434 KBytes [SUM] 0.00-1.00 sec 35.9 MBytes 301 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 1.00-2.00 sec 12.0 MBytes 100 Mbits/sec 0 417 KBytes [ 7] 1.00-2.00 sec 12.1 MBytes 101 Mbits/sec 0 417 KBytes [ 9] 1.00-2.00 sec 12.0 MBytes 101 Mbits/sec 0 434 KBytes [SUM] 1.00-2.00 sec 36.0 MBytes 302 Mbits/sec 0 --truncated-- [ 5] 9.00-10.00 sec 12.5 MBytes 105 Mbits/sec 0 417 KBytes [ 7] 9.00-10.00 sec 12.6 MBytes 106 Mbits/sec 0 417 KBytes [ 9] 9.00-10.00 sec 12.5 MBytes 105 Mbits/sec 0 434 KBytes [SUM] 9.00-10.00 sec 37.7 MBytes 316 Mbits/sec 0 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 123 MBytes 103 Mbits/sec 0 sender [ 5] 0.00-10.01 sec 123 MBytes 103 Mbits/sec receiver [ 7] 0.00-10.00 sec 123 MBytes 103 Mbits/sec 0 sender [ 7] 0.00-10.01 sec 123 MBytes 103 Mbits/sec receiver [ 9] 0.00-10.00 sec 123 MBytes 103 Mbits/sec 0 sender [ 9] 0.00-10.01 sec 123 MBytes 103 Mbits/sec receiver [SUM] 0.00-10.00 sec 370 MBytes 310 Mbits/sec 0 sender [SUM] 0.00-10.01 sec 369 MBytes 309 Mbits/sec receiver$
The test above shows 310Mbps which confirms there is no TCP throughput issue between the sites.
- Run the following UDP commands on the clients:
PC-A# iperf3 -c 192.168.13.3 -u Connecting to host 192.168.13.3, port 5201 [ 5] local 192.168.20.3 port 49647 connected to 192.168.13.3 port 5201 [ ID] Interval Transfer Bitrate Total Datagrams [ 5] 0.00-1.00 sec 129 KBytes 1.05 Mbits/sec 91 [ 5] 1.00-2.00 sec 127 KBytes 1.04 Mbits/sec 90 --truncated-- [ 5] 9.00-10.00 sec 129 KBytes 1.05 Mbits/sec 91 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 5] 0.00-10.00 sec 1.25 MBytes 1.05 Mbits/sec 0.000 ms 0/906 (0%) sender [ 5] 0.00-10.06 sec 1.25 MBytes 1.04 Mbits/sec 0.046 ms 0/906 (0%) receiver
PC-A# iperf3 -c 192.168.13.3 -u -b 100M Connecting to host 192.168.13.3, port 5201 [ 5] local 192.168.20.3 port 58797 connected to 192.168.13.3 port 5201 [ ID] Interval Transfer Bitrate Total Datagrams [ 5] 0.00-1.00 sec 11.9 MBytes 99.9 Mbits/sec 8625 [ 5] 1.00-2.00 sec 11.9 MBytes 100 Mbits/sec 8633 --truncated-- [ 5] 9.00-10.00 sec 11.9 MBytes 100 Mbits/sec 8632 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 5] 0.00-10.00 sec 119 MBytes 100 Mbits/sec 0.000 ms 0/86320 (0%) sender [ 5] 0.00-10.05 sec 119 MBytes 99.4 Mbits/sec 0.046 ms 34/86320 (0.039%) receiver
PC-A# iperf3 -c 192.168.13.3 -u -b 400M Connecting to host 192.168.13.3, port 5201 [ 5] local 192.168.20.3 port 44152 connected to 192.168.13.3 port 5201 [ ID] Interval Transfer Bitrate Total Datagrams [ 5] 0.00-1.00 sec 47.6 MBytes 400 Mbits/sec 34504 [ 5] 1.00-2.00 sec 47.7 MBytes 400 Mbits/sec 34531 --truncated-- [ 5] 9.00-10.00 sec 47.7 MBytes 400 Mbits/sec 34528 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 5] 0.00-10.00 sec 477 MBytes 400 Mbits/sec 0.000 ms 0/345277 (0%) sender [ 5] 0.00-10.06 sec 345 MBytes 288 Mbits/sec 0.061 ms 95316/345171 (28%) receiver
The receiver is getting a speed for 288Mbps. The throughput is within the expected range as dictated by the sizing guide, and there is no UDP throughput issue
- To learn more about the flags and their usage, see iPerf user documentation.
Measure downstream throughput
Use the -R flag to reverse the traffic direction so that the server sends traffic to the client. By default, iPerf sends traffic from the client to the server
Downstream throughput is the traffic flowing from the Server to the client
- Run the following command on the client
iperf3 -c 192.168.13.3 -w 64kb -R
iperf3 -c 192.168.13.3 -w 128kb -P 2 -R
iperf3 -c 192.168.13.3 -w 256kb -P 3 -R
$ iperf -c 192.168.13.3 -u -i 1 -t 10 -b 900M -R
Downstream output example server:
| Server | Client |
|
----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 192.168.20.3, port 59825 [ 6] local 192.168.13.3 port 5201 connected to 192.168.20.3 port 59825 [ ID] Interval Transfer Bitrate Total Datagrams [ 6] 0.00-1.00 sec 60.8 MBytes 510 Mbits/sec 44013 [ 6] 1.00-2.00 sec 76.6 MBytes 643 Mbits/sec 55469 [ 6] 2.00-3.00 sec 76.6 MBytes 643 Mbits/sec 55466 [ 6] 3.00-4.00 sec 76.5 MBytes 642 Mbits/sec 55418 [ 6] 4.00-5.00 sec 76.7 MBytes 643 Mbits/sec 55526 [ 6] 5.00-6.00 sec 76.8 MBytes 644 Mbits/sec 55608 [ 6] 6.00-7.00 sec 76.5 MBytes 641 Mbits/sec 55364 [ 6] 7.00-8.00 sec 76.6 MBytes 643 Mbits/sec 55483 [ 6] 8.00-9.00 sec 76.4 MBytes 641 Mbits/sec 55304 [ 6] 9.00-10.00 sec 76.1 MBytes 638 Mbits/sec 55088 [ 6] 10.00-10.03 sec 1.91 MBytes 640 Mbits/sec 1386 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 6] 0.00-10.03 sec 751 MBytes 629 Mbits/sec 0.000 ms 0/544125 (0%) sender |
Connecting to host 192.168.13.3, port 5201 Reverse mode, remote host 192.168.13.3 is sending [ 13] local 192.168.20.3 port 59825 connected to 192.168.13.3 port 5201 [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 13] 0.00-1.00 sec 9.34 MBytes 78.4 Mbits/sec 0.290 ms 28490/35256 (81%) [ 13] 1.00-2.00 sec 9.45 MBytes 79.3 Mbits/sec 0.271 ms 48924/55770 (88%) [ 13] 2.00-3.00 sec 8.88 MBytes 74.5 Mbits/sec 0.425 ms 48069/54502 (88%) [ 13] 3.00-4.00 sec 8.87 MBytes 74.4 Mbits/sec 0.229 ms 49659/56085 (89%) [ 13] 4.00-5.00 sec 8.04 MBytes 67.3 Mbits/sec 0.249 ms 46848/52670 (89%) [ 13] 5.00-6.00 sec 8.83 MBytes 74.2 Mbits/sec 0.274 ms 52210/58605 (89%) [ 13] 6.00-7.00 sec 9.03 MBytes 75.7 Mbits/sec 0.194 ms 48272/54810 (88%) [ 13] 7.00-8.00 sec 9.26 MBytes 77.7 Mbits/sec 0.299 ms 49219/55925 (88%) [ 13] 8.00-9.00 sec 9.20 MBytes 77.2 Mbits/sec 0.274 ms 48075/54739 (88%) [ 13] 9.00-10.00 sec 8.27 MBytes 69.3 Mbits/sec 0.304 ms 44997/50983 (88%) - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 13] 0.00-10.03 sec 751 MBytes 629 Mbits/sec 0.000 ms 0/544125 (0%) sender [SUM] 0.0-10.0 sec 2 datagrams received out-of-order [ 13] 0.00-10.00 sec 89.2 MBytes 74.8 Mbits/sec 0.304 ms 464763/529345 (88%) receiver |
In the downstream example, the server bitrate is approximately 640 Mbits/sec, which represents the total traffic the server attempted to send to the client during each test interval. The client output shows the actual traffic the client received.