Managing Loki (Log Aggregation Tool)

Loki is a log aggregation tool integrated with Grafana to visualize cluster logs. It eases the debugging process.

Warning

By default, Loki is disabled after installation or upgrade due to the CPU/LOAD high resource usage that might cause performance or stability issues. If you activate Loki, make sure it is for a limited amount of time. Closely monitor the Loki partition and when the partition is close to full, disable Loki.

Prerequisites

These elements must be strictly under the following thresholds:

• Memory (%): < 90%

• System Load: < 100

Note

These values can be monitored in the Dashboard Monitoring.

Enable Loki

1. Update bootstrap configuration

   [artesca-os@bootstrap ~]$
   

   sudo salt-call --local state.single file.serialize \
     /etc/metalk8s/bootstrap.yaml \
     dataset="{ \
       'addons': { \
         'fluent-bit': { \
           'enabled': True \
         }, \
         'loki': { \
           'enabled': True \
         } \
       } \
     }" \
     serializer=yaml merge_if_exists=True show_changes=True
   

   Copy

2. Apply the changes

   [artesca-os@bootstrap ~]$
   

   sudo kubectl exec --kubeconfig /etc/kubernetes/admin.conf \
       $(sudo kubectl --kubeconfig /etc/kubernetes/admin.conf get pod -n kube-system -l app=salt-master -o name) \
       -n kube-system -c salt-master -- \
       salt-run state.sls metalk8s.addons.logging.deployed \
       saltenv=metalk8s-128.0.3
   

   Copy

3. Closely monitor the Loki partition in the ARTESCA UI.

Disable Loki

1. Update bootstrap configuration

   [artesca-os@bootstrap ~]$
   

   sudo salt-call --local state.single file.serialize \
     /etc/metalk8s/bootstrap.yaml \
     dataset="{ \
       'addons': { \
         'fluent-bit': { \
           'enabled': False \
         }, \
         'loki': { \
           'enabled': False \
         } \
       } \
     }" \
     serializer=yaml merge_if_exists=True show_changes=True
   

   Copy

2. Apply the changes

   [artesca-os@bootstrap ~]$
   

   sudo kubectl exec --kubeconfig /etc/kubernetes/admin.conf \
       $(sudo kubectl --kubeconfig /etc/kubernetes/admin.conf get pod -n kube-system -l app=salt-master -o name) \
       -n kube-system -c salt-master -- \
       salt-run state.sls metalk8s.addons.logging.deployed \
       saltenv=metalk8s-128.0.3
   

   Copy