Logging#

Elasticsearch, Fluentd, and Kibana, or EFK stack, is an industry standard for collecting, storing, and visualizing Kubernetes logs. Fluentd collects the Kubernetes containers logs and transmits them to Elasticsearch. Elasticsearch stores the logs and exposes them through a RESTful API. Kibana provides a graphical representation of the collected logs along with filtering and search capabilities. All components of EFK are deployed as Kubernetes Pods in the rackspace-system namespace.

The default retention period for Elasticsearch logs and data is seven days. If your organization requires you store such information for an extended a period of time, contact your Rackspace representative or submit a support ticket.

Using Kibana#

You can access Kibana by using your Managed Services credentials and the URL listed in Access the Rackspace KaaS dashboards. To view and visualize the logs You need to configure index patterns. By default, Kibana suggests using the logstash-* index pattern. You can use the default setting or configure additional patterns as needed.

To configure Kibana, complete the following steps:

  1. Log in to the Kibana UI using the URL and credentials provided in Access the Rackspace KaaS dashboards.

  2. Go to DevTools > Console.

  3. List available indices:

    GET _cat/indices
    

    Alternatively, you can use curl -XGET "http://<path-to-Kibana-UI>_cat/indices".

    Example of system response:

    green open .monitoring-es-6-2018.06.21     woJ90oNBTY2IdrN8edLh2w 1 1 182422 648 344.5mb 172.2mb
    green open .monitoring-es-6-2018.06.22     Br6Ioc8PRKu1ySQAt-d-cA 1 1 121440 432   229mb 114.8mb
    green open .watcher-history-3-2018.06.20   sopRUohGTIWtPRGKf-VGEA 1 1   5760   0   9.6mb   4.8mb
    green open .monitoring-kibana-6-2018.06.22 FE6508QfTyWDgc-WoQr_bg 1 1   5737   0   4.5mb   2.2mb
    green open .triggered_watches              eca1efrIQN2E1XLiujeXEw 1 1      0   0   4.9mb   3.3mb
    green open .kibana                         eKGb8L2XRvCoOcUWBsDA1A 1 1      3   0    90kb    45kb
    green open .monitoring-es-6-2018.06.20     wH20lwaHT2ikRlwTCS1J_Q 1 1 182573 594 339.9mb   170mb
    green open logstash-2018.06.20             iuaiT6abSsmYSFTN7laLoA 5 1 188593   0 264.8mb 132.4mb
    green open .watches                        V1r07Q0wQ1WpHAsPsyNm3Q 1 1      4   0  30.7kb  19.3kb
    green open logstash-2018.06.22             O9me9g3BSE6H-0jtKU5oYA 5 1 121590   0 162.1mb    81mb
    green open .monitoring-kibana-6-2018.06.20 qZrbShReQka1Eufxb6iQ9A 1 1   8630   0   6.9mb   3.4mb
    green open logstash-2018.06.21             jsk2AKCPRfSwkFgg3_TGkQ 5 1 181909   0 251.4mb 125.8mb
    green open .monitoring-kibana-6-2018.06.21 Se2Qfs8WQtGTXlAzMd5Q-A 1 1   8632   0   6.7mb   3.3mb
    green open .watcher-history-3-2018.06.22   ENhLUYkmRlOpxOx2HavA6w 1 1   3828   0   6.3mb   3.1mb
    green open .watcher-history-3-2018.06.21   GQZ2-GvaSIq_dA7VTidHdA 1 1   5760   0   9.5mb   4.7mb
    
  4. Click Management > Index Pattern > Create Index Pattern.

  5. Configure the default logstash pattern or a custom pattern as needed.

  6. Click Create.

  7. In the left navigation panel, click Discover to view the logs.

    Example:

    ../../../_images/kibana-example.png

The EFK stack provides rich capabilities to configure logging. For more information, see Kibana documentation, Fluentd documentation, and Elasticsearch documentation.