Log Forwarding

Log Forwarding allows you to forward the log entries (including the host and container level log entries) stored in JournalD to the external collectors. SMI/CNDP supports target hosts such as Fluent-x, Splunk, Loki and Grafana Cloud for log forwarding. To stream data, Fluent-x uses the Forward protocol and Splunk uses HTTPS. Fluent Bit sends logs to Grafana Cloud by providing the appropriate URL and ensuring that TLS is enabled.

Note

SMI/CNDP enables only one target host of Grafana Cloud type for logs forwarding. However, Splunk, Fluent-bit, and Loki can be enabled in parallel.

The CEE utilizes FluentD for buffering and persistent connection support. FluentD is an open-source data collection and consumption software. Using FluentD, you can collect logging events from various sources and unify it for better usage and understanding. For more information of FluentD, see https://docs.fluentd.org/.

By default, FluentD is configured with the following parameters to support buffering and keepalive:

total_limit_size 1GB 
chunk_limit_size 8MB 
compress text 
flush_mode interval 
flush_interval 5s 
overflow_action drop_oldest_chunk 
retry_timeout 1h 

The following figure depicts the high-level Log Forwarding architecture:

Log Forwarding Architecture