Log Forwarding - Missing or No Log

Problem

This section describes how to resolve the "missing logs" issue when Log Forwarding is enabled.

Log forwarding is enabled, but logs are missing or not seen on the external collector.

Resolution

  1. Ping the Kubernetes node from the collection host to verify the network connectivity between each Kubernetes nodes and the collection server (where Fluentd or Fluentbit endpoint is hosted).

    ping [k8s-node-ip]
  2. Verify whether the client machine's system clock is in synchronization with the Kubernetes node's clock. Any mismatch in the time will result in incorrect query in the front-end visualization tool (Kibana or Grafana), which uses the host clock as query parameter.

  3. When Logs Forwarding is enabled, the Logs Forwarder dumps the entire JournalD entries - from the beginning - to the external collector. Also, the number of nodes on the deployment increases the amount of data available for processing. It may take a while for pushing out and processing the log entries before the frontend tools visualize it.