-
- Downloads
"b_asic/git@gitlab.liu.se:lukja239/B-ASIC.git" did not exist on "6cf50b3429a8d6bbd63e5cbac0f58340d5f1c9fe"
Logserver: Keep more log files open.
By default, rsyslog keeps at most 10 files open for each log action when using dynamic file templates like we do. This can make rsyslogd close and open files constantly, which may increase filesystem load and decrease performance. Some sampling on NSC's current log server (using strace -C) shows that it is opening and closing log files some 10-15 times per second, and is on average writing less than 15 log messages to each file before it is closed. This amount is not a big concern, but it feels enough that we should concider mitigating it. We here increase that cache of open files to 256 for the "main" log action. As NSC currently has about 200 active syslog clients, that should make rsyslogd basically never have to close and reopen a log file. We have also tested smaller cache size. At 100 open files, rsyslogd still does about 1.0-1.5 close and open calls per second. Since a few hundred open files is not a big deal today, we can easily afford having the cache size set to 256 and almost never have to close a log file. If we start taking logs from worker nodes in our compute clusters, we will probably need to revisit this, and possibly change how we store logs, as we would then have some 3000-4000 clients or more, each with its own log file.
Please register or sign in to comment