Linux

Debian 613 執行緒限制

  • November 22, 2011

在我的專用 debian 伺服器上執行這個程序thread-limit.c時,輸出顯示我的系統不能創建超過 600 個執行緒。我需要創建更多執行緒,並修復我的系統配置錯誤。

以下是有關我的專用伺服器的一些資訊:

de801:/# uname -a
Linux de801.ispfr.net 2.6.18-028stab085.5 #1 SMP Thu Apr 14 15:06:33 MSD 2011 x86_64 GNU/Linux
de801:/# java -version
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
de801:/# ldd $(which java)
       linux-vdso.so.1 =>  (0x00007fffbc3fd000)
       libpthread.so.0 => /lib/libpthread.so.0 (0x00002af013225000)
       libjli.so => /usr/lib/jvm/java-6-sun-1.6.0.26/jre/bin/../lib/amd64/jli/libjli.so (0x00002af013441000)
       libdl.so.2 => /lib/libdl.so.2 (0x00002af01354b000)
       libc.so.6 => /lib/libc.so.6 (0x00002af013750000)
       /lib64/ld-linux-x86-64.so.2 (0x00002af013008000)
de801:/# cat /proc/sys/kernel/threads-max
1589248
de801:/# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 794624
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 10240
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 128
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

這是C程序的輸出

de801:/test# ./thread-limit
Creating threads ...
Address of c = 1061520 KB
Address of c = 1081300 KB
Address of c = 1080904 KB
Address of c = 1081168 KB
Address of c = 1080508 KB
Address of c = 1080640 KB
Address of c = 1081432 KB
Address of c = 1081036 KB
Address of c = 1080772 KB
100 threads so far ...
200 threads so far ...
300 threads so far ...
400 threads so far ...
500 threads so far ...
600 threads so far ...
Failed with return code 12 creating thread 637.

有什麼想法可以解決這個問題嗎?

您一次不需要創建超過 600 個執行緒。你只需要執行緒來處理你可以同時做的有用的事情。您的系統一次不能有效地做超過 600 件事情。您不需要更多執行緒來完成更多工作。

無論如何,您設置了錯誤的堆棧大小。您正在設置初始執行緒堆棧的限制大小,該堆棧不是由您的程式碼創建的。您應該設置程式碼創建的執行緒堆棧的大小。

改變:

 printf("Creating threads ...\n");
 for (i = 0; i < MAX_THREADS && rc == 0; i++) {
   rc = pthread_create(&(thread[i]), NULL, (void *) &run, NULL);

到(大致):

 printf("Creating threads ...\n");
 pthread_attr_t pa;
 pthread_attr_init(&pa);
 pthread_attr_setstacksize(&pa, 2*1024*1024);
 for (i = 0; i < MAX_THREADS && rc == 0; i++) {
   rc = pthread_create(&(thread[i]), &pa, (void *) &run, NULL);
 phtread_attr_destroy(&pa);

您也可以呼叫pthread_attr_getstacksize來查找我們您平台的預設執行緒堆棧大小。請注意,如果執行緒超出其分配的堆棧,您的程序將死亡。

即使使用 2MB 的堆棧(這可能非常危險),您仍然會看到 1.2GB 的堆棧只有 600 個執行緒。無論您的工作是什麼,執行緒都不是適合它的工具。

這與執行緒限制無關。錯誤程式碼 12 不適用於執行緒限制。

$ perror 12
OS error code  12:  Cannot allocate memory

我想如果你在堆棧上執行的任何東西,如果你將它設置為 8MB 的 redhat 預設值,你可能會能夠執行更多的執行緒。

這與顯式堆棧大小設置為 8MB 的連結相同。看看當你編譯這個時會發生什麼。

/* compile with:   gcc -lpthread -o thread-limit thread-limit.c */
/* originally from: http://www.volano.com/linuxnotes.html */

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>
#include <sys/time.h>
#include <sys/resource.h>

#define MAX_THREADS 10000
int i;

void run(void) {
 char c;
 if (i < 10)
   printf("Address of c = %u KB\n", (unsigned int) &c / 1024);
 sleep(60 * 60);
}

int main(int argc, char *argv[]) {
 struct rlimit limits;
 limits.rlim_cur = 1024*1024*8;
 limits.rlim_max = 1024*1024*8;
 if (setrlimit(RLIMIT_STACK, &limits) == -1) {
   perror("Failed to set stack limit");
   exit(1);
 }
 int rc = 0;
 pthread_t thread[MAX_THREADS];
 printf("Creating threads ...\n");
 for (i = 0; i < MAX_THREADS && rc == 0; i++) {
   rc = pthread_create(&(thread[i]), NULL, (void *) &run, NULL);
   if (rc == 0) {
     pthread_detach(thread[i]);
     if ((i + 1) % 100 == 0)
   printf("%i threads so far ...\n", i + 1);
   }
   else
     printf("Failed with return code %i creating thread %i.\n",
        rc, i + 1);
 }
 exit(0);
}

引用自:https://serverfault.com/questions/333407