HAProxy 和後端伺服器之間失去一半的數據

我在 Ubuntu 上有這樣的設置

瀏覽器 –> HAProxy –> 後端伺服器

後端伺服器是一個 ASP.NET Core Web 應用程序。

它的工作時間為 99.9%，除非正在上傳二進製文件（帶有多部分錶單數據的簡單 POST），在這種情況下我會收到錯誤消息：

System.IO.IOException: Unexpected end of Stream, the content may have already been read by another component. 

  at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)

  at Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)

  at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)

  at Microsoft.AspNetCore.Http.Features.FormFeature.InnerReadFormAsync(CancellationToken cancellationToken)

  at Microsoft.AspNetCore.Antiforgery.DefaultAntiforgeryTokenStore.GetRequestTokensAsync(HttpContext httpContext)

  at Microsoft.AspNetCore.Antiforgery.DefaultAntiforgery.ValidateRequestAsync(HttpContext httpContext)

  at Microsoft.AspNetCore.Mvc.ViewFeatures.Filters.ValidateAntiforgeryTokenAuthorizationFilter.OnAuthorizationAsync(AuthorizationFilterContext context)

  at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)

  at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)

  at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)

  at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

所以，我一直在嘗試調查原因。

我直接嘗試了 ASP.NET Core 後端，它可以工作。

所以我將傳入的數據記錄到後端，發現如果我直接這樣做，我會得到預期的請求標頭，然後是正文中的上傳文件。

然後我通過 HAProxy 將傳入的數據記錄到後端，並且一半的數據失去了。

IE。後端看到的第一個數據似乎是

04 E2 22 FC 60 FF 2B E1  BF 85 D2 75 F9 44 94 86

但這些字節大約是我文件的一半，即 25,126 字節。我根本看不到任何標題資訊。

我準備接受我的日誌記錄不完善且不一定準確。似乎以某種方式將請求分成兩部分，或者緩衝區已被填充然後被覆蓋。

問題可能是什麼？

global

   chroot /var/lib/haproxy
   log /dev/log    local0 info

   stats socket /run/haproxy/admin.sock mode 660 level admin
   stats timeout 30s
   user haproxy
   group users
   daemon
       
   lua-load /home/user/haproxy-mapping.lua

   ssl-default-bind-ciphers ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS:!AESCCM
   ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
   tune.ssl.default-dh-param 2048


defaults
   log     global
   mode    http

   timeout connect 30000
   timeout client  50000
   timeout server  50000
   option forwardfor
   option http-server-close #have also try http-keep-alive


frontend httpfront
   mode http
   bind *:80
   redirect scheme https code 301 if !{ ssl_fc }


frontend web_front_end

   bind *:443 ssl crt /home/.....file.pem
   mode http

    log /var/lib/haproxy/dev/log local0 info

       # Rate limiting
       stick-table  type ip  size 100k  expire 600s  store http_req_rate(60s) #store up to 100k requests for 60s, see if over 60s there are more than 600
       http-request track-sc0 src
       http-request deny deny_status 429 if { sc_http_req_rate(0) gt 600 }


   # Ensure we have a clean state to start with
   http-request del-header X-SERVER-SNI

   # Set the concatenated value of the SNI value to a temporary header
   http-request set-header X-SERVER-SNI haproxy.%[ssl_fc_sni] if { ssl_fc_sni -m found }

   # Set the value of the header to a transaction-level variable
   http-request set-var(txn.fc_sni) ssl_fc_sni #hdr(X-SERVER-SNI) if { hdr(X-SERVER-SNI) -m found }

   #use Lua code to determine which backend to send to
   use_backend %[lua.legacy_new_backend]





backend backendnodes_1_https
   balance roundrobin
   option forwardfor
   server node1 127.0.0.1:446 ssl verify none sni var(txn.fc_sni)

現在使用 HAProxy 配置似乎可以正常工作

option http-keep-alive

代替

option http-server-close

我會發誓我以前嘗試過，但它確實有幫助。

引用自：https://serverfault.com/questions/1085353