Bandwidth

為什麼頻寬不是~10G?

  • August 12, 2016

我最初發佈在“網路工程”上,但我被告知在這裡問。

我有 2 台伺服器直接通過 10G 電纜連接。我需要知道連接是否良好,並且確實具有 10G 吞吐量。如果不是,我該如何檢查問題所在?我使用 IPERF 檢查頻寬。

更新 我配置了巨型數據包、極端中斷率、接收/發送緩衝區的最大值。對於 IPERF,我只對伺服器使用“iperf3.exe -s”,對客戶端使用“iperf3.exe -c IP”。

根據測試,頻寬為 1.5-1.7 Gbits/sec。這是否意味著連接是10G?可能是什麼問題呢?

伺服器 A 是 Windows 8.1 Pro 64bit

伺服器 B 是 Windows 7 Enterprise 64bit

這是 ServerA (192.168.10.100) 是伺服器且 ServerB (192.168.10.200) 是客戶端時的輸出:

-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.10.200, port 63772
[  5] local 192.168.10.100 port 5201 connected to 192.168.10.200 port 63773
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   170 MBytes  1.42 Gbits/sec
[  5]   1.00-2.00   sec   217 MBytes  1.82 Gbits/sec
[  5]   2.00-3.00   sec   217 MBytes  1.82 Gbits/sec
[  5]   3.00-4.00   sec   214 MBytes  1.80 Gbits/sec
[  5]   4.00-5.00   sec   217 MBytes  1.82 Gbits/sec
[  5]   5.00-6.00   sec   207 MBytes  1.74 Gbits/sec
[  5]   6.00-7.00   sec   199 MBytes  1.67 Gbits/sec
[  5]   7.00-8.00   sec   206 MBytes  1.73 Gbits/sec
[  5]   8.00-9.00   sec   192 MBytes  1.61 Gbits/sec
[  5]   9.00-10.00  sec   209 MBytes  1.75 Gbits/sec
[  5]  10.00-10.16  sec  30.3 MBytes  1.63 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.16  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-10.16  sec  2.03 GBytes  1.72 Gbits/sec                  receiver

這是當 B (192.168.10.200) 是伺服器並且 A (192.168.10.100) 是客戶端時的輸出:

-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.10.100, port 55565
[  5] local 192.168.10.200 port 5201 connected to 192.168.10.100 port 55566
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   170 MBytes  1.43 Gbits/sec
[  5]   1.00-2.00   sec   175 MBytes  1.47 Gbits/sec
[  5]   2.00-3.00   sec   175 MBytes  1.47 Gbits/sec
[  5]   3.00-4.00   sec   179 MBytes  1.50 Gbits/sec
[  5]   4.00-5.00   sec   181 MBytes  1.52 Gbits/sec
[  5]   5.00-6.00   sec   200 MBytes  1.68 Gbits/sec
[  5]   6.00-7.00   sec   176 MBytes  1.47 Gbits/sec
[  5]   7.00-8.00   sec   177 MBytes  1.48 Gbits/sec
[  5]   8.00-9.00   sec   186 MBytes  1.56 Gbits/sec
[  5]   9.00-10.00  sec   179 MBytes  1.50 Gbits/sec
[  5]  10.00-10.04  sec  6.42 MBytes  1.46 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.04  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-10.04  sec  1.76 GBytes  1.51 Gbits/sec                  receiver

更新2-UDP

當 192.168.10.100 為伺服器時

iperf3.exe -c 192.168.10.100 -u -V
iperf 3.1.3
CYGWIN_NT-6.1 ServerB 2.5.1(0.297/5/3) 2016-04-21 22:14 x86_64
Time: Tue, 09 Aug 2016 19:58:03 GMT
Connecting to host 192.168.10.100, port 5201
     Cookie: ServerB.1470772683.561800.7c902a3e559
[  4] local 192.168.10.200 port 63675 connected to 192.168.10.100 port 5201
Starting Test: protocol: UDP, 1 streams, 8192 byte blocks, omitting 0 seconds, 1
0 second test
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  16
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datag
rams
[  4]   0.00-10.00  sec  1.25 MBytes  1.05 Mbits/sec  0.437 ms  0/160 (0%)
[  4] Sent 160 datagrams
CPU Utilization: local/sender 1.7% (0.9%u/0.8%s), remote/receiver 0.1% (0.0%u/0.
1%s)

當 192.168.10.200 為伺服器時

iperf3.exe -c 192.168.10.200 -u -V
iperf 3.1.3
CYGWIN_NT-6.3 ServerA 2.5.1(0.297/5/3) 2016-04-21 22:14 x86_64
Time: Tue, 09 Aug 2016 20:37:11 GMT
Connecting to host 192.168.10.200, port 5201
     Cookie: ServerA.1470775031.986727.6f8e913650af9b
[  4] local 192.168.10.100 port 51834 connected to 192.168.10.200 port 5201
Starting Test: protocol: UDP, 1 streams, 8192 byte blocks, omitting 0 seconds, 1
0 second test
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   152 KBytes  1.24 Mbits/sec  19
[  4]   1.00-2.00   sec   144 KBytes  1.18 Mbits/sec  18
[  4]   2.00-3.00   sec   160 KBytes  1.31 Mbits/sec  20
[  4]   3.00-4.00   sec   160 KBytes  1.31 Mbits/sec  20
[  4]   4.00-5.00   sec   152 KBytes  1.25 Mbits/sec  19
[  4]   5.00-6.00   sec   152 KBytes  1.24 Mbits/sec  19
[  4]   6.00-7.00   sec   152 KBytes  1.25 Mbits/sec  19
[  4]   7.00-8.00   sec   152 KBytes  1.25 Mbits/sec  19
[  4]   8.00-9.00   sec   152 KBytes  1.24 Mbits/sec  19
[  4]   9.00-10.00  sec   144 KBytes  1.18 Mbits/sec  18
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datag
rams
[  4]   0.00-10.00  sec  1.48 MBytes  1.25 Mbits/sec  0.202 ms  0/189 (0%)
[  4] Sent 189 datagrams
CPU Utilization: local/sender 0.1% (0.1%u/0.0%s), remote/receiver 0.0% (0.0%u/0.
0%s)

UPDATE3 - NTTTCP W/O -NDL

服務

ntttcp.exe -s -m 8,*,192.168.10.200 -l 128k -a 2 -t 15
Copyright Version 5.31
Network activity progressing...


Thread  Time(s) Throughput(KB/s) Avg B / Compl
======  ======= ================ =============
    0   15.000        65544.533    131072.000
    1   15.000        65518.933    131072.000
    2   15.000        93585.067    131072.000
    3   15.002        63906.146    131072.000
    4   15.000        65536.000    131072.000
    5   15.000        62310.400    131072.000
    6   15.001        81837.744    131072.000
    7   15.001        73262.316    131072.000


#####  Totals:  #####


  Bytes(MEG)    realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
    8371.875000      15.001       8854.867          558.088


Throughput(Buffers/s) Cycles/Byte       Buffers
===================== =========== =============
            4464.702      17.198     66975.000


DPCs(count/s) Pkts(num/DPC)   Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
   12120.992         0.823       30450.170          0.328


Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
     991381           149616           9      0     36.026

伺服器B

ntttcp.exe -r -m 8,*,192.168.10.200 -rb 2M -a 16 -t 15
Copyright Version 5.31
Network activity progressing...


Thread  Time(s) Throughput(KB/s) Avg B / Compl
======  ======= ================ =============
    0   15.000        65546.450     60268.990
    1   14.999        65523.318     60229.282
    2   14.999        93582.889     60659.943
    3   14.999        63900.843     61621.870
    4   15.000        65536.817     60278.174
    5   14.999        62316.071     61141.628
    6   14.999        81840.156     60394.032
    7   14.999        73264.934     60858.538


#####  Totals:  #####


  Bytes(MEG)    realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
    8371.320061      14.999       8942.835          558.125


Throughput(Buffers/s) Cycles/Byte       Buffers
===================== =========== =============
            8930.003       5.314    133941.121


DPCs(count/s) Pkts(num/DPC)   Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
    7965.264         8.216       14260.151          4.589


Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
     146841           981564           1      0      9.699

UPDATE4 - NTTTCP 與 -NDL

服務

ntttcp.exe -s -m 8,*,192.168.10.200 -l 128k -a 2 -t 15 -ndl
Copyright Version 5.31
Network activity progressing...


Thread  Time(s) Throughput(KB/s) Avg B / Compl
======  ======= ================ =============
    0   14.999        78059.604    131072.000
    1   15.000        76902.400    131072.000
    2   14.999        59882.392    131072.000
    3   14.999        62485.232    131072.000
    4   14.999        72734.449    131072.000
    5   15.001        77520.165    131072.000
    6   15.000        62702.933    131072.000
    7   15.000        62771.200    131072.000


#####  Totals:  #####


  Bytes(MEG)    realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
    8101.250000      15.000       8912.094          540.083


Throughput(Buffers/s) Cycles/Byte       Buffers
===================== =========== =============
            4320.667      17.355     64810.000


DPCs(count/s) Pkts(num/DPC)   Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
    9966.333         0.974       29515.333          0.329


Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
     953174           145579           1      0     35.182

伺服器B

ntttcp.exe -r -m 8,*,192.168.10.200 -rb 2M -a 16 -t 15 -ndl
Copyright Version 5.31
Network activity progressing...


Thread  Time(s) Throughput(KB/s) Avg B / Compl
======  ======= ================ =============
    0   15.000        78054.000     60255.789
    1   15.001        76903.606     60839.375
    2   15.000        59880.183     61786.888
    3   14.999        62483.832     60812.856
    4   15.000        72730.333     61516.405
    5   14.999        77520.335     60398.365
    6   14.998        62701.660     60628.917
    7   14.998        62771.436     60585.766


#####  Totals:  #####


  Bytes(MEG)    realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
    8100.944093      14.999       8943.556          540.099


Throughput(Buffers/s) Cycles/Byte       Buffers
===================== =========== =============
            8641.583       4.888    129615.105


DPCs(count/s) Pkts(num/DPC)   Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
    7912.594         8.003       14043.670          4.509


Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
     143914           949785           0      0      8.633

事實證明,問題與 Windows 7/8 客戶端如何管理 TCP 視窗縮放有關。

預設情況下,Windows 客戶端版本更喜歡相對較小的 TCP 視窗,優先考慮延遲與吞吐量。當使用非常快的連結時,例如在這種情況下,將自動調整設置為正常可以帶來顯著的頻寬增益。

要將自動調整設置為更明智的值,您可以使用命令netsh interface tcp set heuristics disabled。無論如何,請在這裡查看更多資訊。

引用自:https://serverfault.com/questions/795860