In this world of Microservices and the distributed systems, a single request (generally) hops through multiple servers before being served. More often than not, these hops are also across the Network cards making the Network Performance the source of slowness in the application.
These parameters makes the need to measure Network performance between servers/systems more critical for benchmarking or debugging.
Iperf3 is one of the open source tools which can be used for network throughput measurement. Below are some of its features.
- Iperf3 can be used for testing maximum TCP and UDP throughput between two servers.
- Iperf3 tests can also be run in a controlled to way to not test the maximum limits but ingest and constant lower network traffic for testing.
- Iperf3 has options for parallel mode(-P) where multiple clients can be used, setting CPU affinity(-A), pausing certain intervals between two requests(-i), setting the length of buffer to read or write(-l), setting target bandwidth (-b) etc.
- More important than anything is the fact that iperf3 runs as an independent tool outside your application code. The results from this tool removes any ambiguities/doubts on the application code which might be causing the network problems.
Installation of iperf3 tool:
sudo apt-get install iperf3
iperf3 tool has be installed on both servers between which you want to measure the network performance. One of the machines is treated as client and other as server.
Command to run on the server:
Below command when run on one of the two servers under test, signifies that the machine is acting as a server for the iperf test.
iperf3 -s -f K- -s — runs in server mode
- -f K — signifies the format as KBytes.
Note : If you do not want to use the default port (which is 5201) for the test, then specify the port with the option -p in the above command and use the same on client as well.
Command to run on the client:
Below command when run on the other server under test, pushes network bandwidth to server and reports the network capacity based on options used.
iperf3 -c 192.XX.XX.XX -f K
==== output ====
Connecting to host 192.XX.XX.XX, port 5201
[ 4] local 192.XX.XX.XX port 50880 connected to 192.XX.XX.XX port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 678 MBytes 693729 KBytes/sec 74 1.06 MBytes
[ 4] 1.00-2.00 sec 750 MBytes 767998 KBytes/sec 0 1.48 MBytes
[ 4] 2.00-3.00 sec 606 MBytes 620723 KBytes/sec 143 1.22 MBytes
[ 4] 3.00-4.00 sec 661 MBytes 677201 KBytes/sec 0 1.57 MBytes
[ 4] 4.00-5.00 sec 620 MBytes 634523 KBytes/sec 0 1.83 MBytes
[ 4] 5.00-6.00 sec 609 MBytes 623718 KBytes/sec 1095 1.44 MBytes
[ 4] 6.00-7.00 sec 730 MBytes 747525 KBytes/sec 0 1.76 MBytes
[ 4] 7.00-8.00 sec 716 MBytes 733224 KBytes/sec 0 2.04 MBytes
[ 4] 8.00-9.00 sec 772 MBytes 791192 KBytes/sec 0 2.29 MBytes
[ 4] 9.00-10.00 sec 944 MBytes 966472 KBytes/sec 212 1.63 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 6.92 GBytes 725627 KBytes/sec 1524 sender
[ 4] 0.00-10.00 sec 6.92 GBytes 725350 KBytes/sec receiver- -c — run in client mode
- The ip mentioned is the ip of the server.
- From the output (last two lines), it can be seen that the total bandwidth available between the two servers is 708MBytes/sec.
There are also various other options available for iperf3 tool. Like the below command specifies the test to be run for 60secs, which is by default 10secs (-t 60), specifies a target bandwidth of 10MB (-b 10M), number of parallel client streams set to 10 (-P 10).
iperf3 -c 192.XX.XX.XX -t 60 -b 10M -P 10 --get-server-output- –get-server-output : this get the command line output from server and prints it on client terminal
- If you want to use udp instead of tcp, same can be achieved using the option –u
- More details available here on man page – link
Below are some of the cases where I have used iperf3 for debugging purpose:
- Throughput of the application doesn’t scale but there is no obvious resource contention in cpu, memory or disk. On running “sar -n DEV 2 20” I could see network usage doesn’t peak above 30MB/sec. On using Iperf3 for benchmarking we could see 30Mb/sec was the max network capacity between the servers.
- When we wanted to find the impact of vpn on the network throughput we used iperf tool for comparative analysis.
Hope this gave you a sneak peak into iperf3 tool’s capability and usages.
Happy tuning!
Engineering Blog 