Rate Limiting Log Messages
The SMI uses the systemd-journald service—a Linux system service for collecting and storing log data—for storing Kubernetes system and pods level log messages to files on the disk. You can configure Rate Limiting to reduce the number of messages logged. Also, Rate Limiting discards some log messages while limiting others. You can apply Rate Limiting to all the messages in the system based on the service so that logs from the services do not interfere with each others limit.
You can configure Rate Limit by defining the RateLimitIntervalSec and RateLimitBurst parameter in /etc/systemd/journald.conf file. If the messages exceed the specified value defined in the RateLimitBurst parameter within the specified interval defined in the RateLimitIntervalSec parameter, the log messages are dropped until the interval period is over.
RateLimitIntervalSec=1s
RateLimitBurst=10000
The disk usage reserved for journal log affects the RateLimitBurst parameter. The value defined in the RateLimitBurst parameter is multiplied by a factor based on the disk usage reserved for the journal logs. More messages are dropped within interval when less disk space is available.
You can run the following command to find out if the log messages are dropped:
sudo systemctl status systemd-journald
Note | Using this command, you can verify the number of suppressed messages as well. |
The following example shows the number of suppressed messages from the docker.service:
Sep 02 21:09:58 tb15-ultram-cnat-cnat-core-protocol-data1 systemd-journald[3791]:
Suppressed 12229 messages from docker.service