ARTESCA Storage Service is Unavailable After a Reboot

Problem Description

When rebooting ARTESCA, an alert reporting that the cluster is unavailable displays in the Alerts tile of the Overview page.

For more information about the alert, open the Alerts menu.

Root Cause

When rebooting ARTESCA, a data drive is missing, but the corresponding Volume is not deleted from the ARTESCA Storage Services configuration.

Investigation Steps

1. Check the Data Service status.

   [artesca-os@bootstrap ~]$
   

   sudo kubectl --kubeconfig /etc/kubernetes/admin.conf -n xcore get dataservice -o wide
   

   
   NAME                             CONFIG READY    SERVICE READY   SERVICE MONITOR READY   STORAGE READY   STATEFULLSET READY   READY   VERSION READY
   artesca-storage-service-1234abcd True            True            True                    True            False                False   True
   

   The False value in STATEFULLSET READY and READY columns indicates that the Data Service is unavailable because it fails to mount at least one drive or the index drive.

2. Describe the Data Service pod to identify the missing data drive.

   [artesca-os@bootstrap ~]$
   

   sudo kubectl --kubeconfig /etc/kubernetes/admin.conf -n xcore describe pod artesca-storage-service-1234abcd-ds-0
   

   
   Type     Reason       Age                      From     Message
   ----     ------       ----                     ----     -------
   **Warning  FailedMount  14m (x114 over 4h35m)    kubelet  MountVolume.NewMounter initialization failed for volume "storage-data-11-cfu2fe-artesca" : path "/dev/disk/by-uuid/b325cbf6-4d8b-47b5-8d4d-3c95f0943def" does not exist**
   Warning  FailedMount  9m55s (x161 over 4h17m)  kubelet  (combined from similar events): Unable to attach or mount volumes: **unmounted volumes=[data-10]**, unattached volumes=[data-5 config-volume data-6 data-11 kube-api-access-s7jpd data-7 data-8 data-3 index data-1 data-9 data-4 data-2]: timed out waiting for the condition
   

   In this example, [data-10] is missing, as specified in unmounted volumes list.

Problem Resolution

Contact Scality Support for assistance.