Monitoring Metrics External Integration

This documentation provides guidance on integrating monitoring metrics and alerts from the ARTESCA platform into external systems. By leveraging APIs, it is possible to seamlessly connect ARTESCA monitoring metrics with third-party tools.

On some metrics, alerts can be set up along with rules, trigger reasons, and actions to be taken when triggered. For more information on alert handling, refer to Alert Management.

Prerequisites

  • Ensure that the platforms are communicating over the network using the 8443 port.

  • The user querying the endpoints must belong to one of the authorized groups (PlatformReader, PlatformAdmin, or StorageManager). For more information on role assignment, refer to Map Roles to Users and Groups.

    Note

    Scality recommends to create a dedicated identity with only the Platform Reader role to consume this API in read-only.

Authentication

The monitoring and alerting endpoints are protected by an OIDC proxy. All API requests must include a valid token in the Authorization header.

To obtain a token, follow the procedure described in Retrieve Access Tokens.

Once you have obtained the token, include the following header in all your requests to the monitoring endpoints:

Authorization: Bearer <TOKEN>

Note

Tokens have a limited lifetime. If your requests start returning 401 Unauthorized responses, obtain a new token.

Integrate Active Alerts

This section describes how to retrieve active alerts from the ARTESCA platform.

All active alerts

Description: Active notifications or warnings indicating issues within the system. The result is a collection of individual alert entries, categorized by severity, and returned for the current time.

URL:

https://<MngtIp>:8443/api/prometheus/api/v1/query?query=ALERTS

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "UUID": "d86bea2f-1485-4beb-a49f-3485504c9766",
          "__name__": "ALERTS",
          "alertname": "DataServiceDegraded",
          "alertstate": "firing",
          "container": "hd",
          "endpoint": "http",
          "instance": "10.160.122.207",
          "job": "artesca-storage-service-ds-a4ac5e27",
          "namespace": "xcore",
          "pod": "artesca-storage-service-a4ac5e27-ds-0",
          "prometheus": "metalk8s-monitoring/prometheus-operator-prometheus",
          "service": "artesca-storage-service-ds-a4ac5e27",
          "severity": "warning",
          "solution": "xcore",
          "xcore_scality_com_node_name": "ip-10-160-122-207.eu-north-1.compute.internal",
          "xcore_scality_com_resource_type": "dataserver"
        },
        "value": [1750238681.094, "1"
        ]
      },
    ],
    "analysis": {

    }
  }
}

Integrate Monitoring Metrics

This section describes how to retrieve metrics from the ARTESCA platform.

Cluster Net Capacity Usage

Description: Net data used on the cluster. The result is in bytes and returned for current time.

URL:

https://<MngtIp>:8443/api/prometheus/api/v1/query?query=sum(hyperdrive_disk_capacity-hyperdrive_disk_free)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
    "status": "success",
    "data": {
        "resultType": "vector",
        "result": [
            {
                "metric": {},
                "value": [
                    1736945861.278,
                    "58624823296"
                ]
            }
        ],
        "analysis": {}
    }
}
Cluster Net Capacity Usage (Percent)

Description: Net data used on the cluster. The result is a percentage and returned for current time.

URL:

https://<MngtIp>:8443/api/prometheus/api/v1/query?query=sum(hyperdrive_disk_capacity-hyperdrive_disk_free)/sum(hyperdrive_disk_capacity)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
    "status": "success",
    "data": {
        "resultType": "vector",
        "result": [
            {
                "metric": {},
                "value": [
                    1736945861.278,
                    "0.15"
                ]
            }
        ],
        "analysis": {}
    }
}

Alert: Refer to Alert Management for more information.

Cluster Protected Capacity Usage

Description: Protected data used on the cluster. The result is in bytes and returned for current time.

URL:

https://<MngtIp>:8443/api/prometheus/api/v1/query?query=sum(hyperdrive_http_bytes_net)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
    "status": "success",
    "data": {
        "resultType": "vector",
        "result": [
            {
                "metric": {},
                "value": [
                    1736945861.28,
                    "3523327"
                ]
            }
        ],
        "analysis": {}
    }
}
Instance CPU

Description: CPU usage of one node within a cluster. The result is a percentage and returned for current time. Replace <instanceIP> by the node IP.

URL:

https://<MngtIp>:8443/api/prometheus/api/v1/query?query=100-(avg%20by(instance)(irate(node_cpu_seconds_total{instance="<instanceIP>"}[5m]))*100)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "instance": "10.160.122.207:9100"
        },
        "value": [1758012262.161, "87.04114583333343"
        ]
      }
    ],
    "analysis": {

    }
  }
}
Instance Disk Read IOPS

Description: The rate of read operations completed on a disk for an instance. The result is input/output operations per second (IOPS) and is calculated over the last 5 minutes. Replace <instanceIP> by the node IP.

URL:

https://<MngtIP>:8443/api/prometheus/api/v1/query?query=sum(irate(node_disk_reads_completed_total{instance="<instanceIP>"}[5m]))by(instance)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "instance": "10.160.122.207:9100"
        },
        "value": [1758012578.877, "0"
        ]
      }
    ],
    "analysis": {

    }
  }
}
Instance Disk Write IOPS

Description: The rate of write operations completed on a disk for one instance. The result is input/output operations per second (IOPS) and is calculated over the last 5 minutes. Replace <instanceIP> by the node IP.

URL:

https://<MngtIP>:8443/api/prometheus/api/v1/query?query=sum(irate(node_disk_writes_completed_total{instance="<instanceIP>"}[5m]))by(instance)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "instance": "10.160.122.207:9100"
        },
        "value": [1758012489.234, "198.83333333333334"
        ]
      }
    ],
    "analysis": {

    }
  }
}
Instance RAM

Description: The amount of Random Access Memory (RAM) currently in use of one instance. The result is a percentage of total RAM and returned for current time. Replace <instanceIP> by the node IP.

URL:

https://<MngtIp>:8443/api/prometheus/api/v1/query?query=sum(100-((node_memory_MemAvailable_bytes{instance="<instanceIP>"}*100)/node_memory_MemTotal_bytes))by(instance)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "instance": "10.160.122.207:9100"
        },
        "value": [1758011945.762, "75.33367497647332"
        ]
      }
    ],
    "analysis": {

    }
  }
}
Net Storage Total

Description: The total logical storage capacity available for user data. The result is in bytes and returned for current time.

URL:

https://<MngtIP>:8443/api/prometheus/api/v1/query?query=sum(last_over_time(hyperdrive_disk_capacity[1m]))

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {

        },
        "value": [1752647692.834, "125568270336"
        ]
      }
    ],
    "analysis": {

    }
  }
}
Net Storage Usage

Description: The total logical size of all data stored on the system before data protection (e.g., erasure coding or replication) is applied. The result is in bytes and returned for current time.

URL:

https://<MngtIP>:8443/api/prometheus/api/v1/query?query=sum(hyperdrive_disk_capacity-hyperdrive_disk_free)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {

        },
        "value": [1752648167.106, "21286436864"
        ]
      }
    ],
    "analysis": {

    }
  }
}
Node CPU

Description: CPU usage of every node within a cluster. The result is a percentage and returned for current time.

URL:

https://<MngtIp>:8443/api/prometheus/api/v1/query?query=100-(avg%20by(instance)(irate(node_cpu_seconds_total{mode="idle"}[5m]))*100)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "instance": "1.1.1.1:9100"
        },
        "value": [1749806941.727, "12.740277777777848"
        ]
      }
    ],
    "analysis": {

    }
  }
}
Node Disk Read IOPS

Description: The rate of read operations completed on a disk for every node. The result is input/output operations per second (IOPS) and is calculated over the last 5 minutes.

URL:

https://<MngtIP>:8443/api/prometheus/api/v1/query?query=sum(irate(node_disk_reads_completed_total[5m]))by(instance)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "instance": "10.160.122.207:9100"
        },
        "value": [1750343347.173, "25.433333333333334"
        ]
      }
    ],
    "analysis": {

    }
  }
}
Node Disk Write IOPS

Description: The rate of write operations completed on a disk for every node. The result is input/output operations per second (IOPS) and is calculated over the last 5 minutes.

URL:

https://<MngtIP>:8443/api/prometheus/api/v1/query?query=sum(irate(node_disk_writes_completed_total[5m]))by(instance)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "instance": "10.160.122.207:9100"
        },
        "value": [1750343328.642, "296.93333333333334"
        ]
      }
    ],
    "analysis": {

    }
  }
}
Node RAM

Description: The amount of Random Access Memory (RAM) currently in use on every node within the cluster. The result is a percentage of total RAM and returned for current time.

URL:

https://<MngtIp>:8443/api/prometheus/api/v1/query?query=sum(100-((node_memory_MemAvailable_bytes*100)/node_memory_MemTotal_bytes))by(instance)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "instance": "10.160.122.207:9100"
        },
        "value": [1750237774.248, "85.07605030207182"
        ]
      }
    ],
    "analysis": {

    }
  }
}
Objects

Description: The total number of individual data objects stored in the system.

URL:

https://<MngtIP>:8443/api/prometheus/api/v1/query?query=sum(hyperdrive_http_object_net)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {

        },
        "value": [1752647575.521, "592"
        ]
      }
    ],
    "analysis": {

    }
  }
}
Protected data

Description: The amount of user data that is safeguarded through replication, erasure coding, or other protection mechanisms. The result is in bytes and returned for current time.

URL:

https://<MngtIP>:8443/api/prometheus/api/v1/query?query=sum(hyperdrive_http_bytes_net)

Required header:

Authorization: Bearer <TOKEN>

Output example:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {

        },
        "value": [1752648242.88, "2565402507"
        ]
      }
    ],
    "analysis": {

    }
  }
}
WP Ingress status

Description: Indicates whether the endpoint is accessible or not. If more than two endpoints are not accessible, it means that ARTESCA services are down. To be executed on all nodes, using the WP IP of each node.

URL:

https://<WP Node IP>/healthz

Output example:

200 OK
5XX or TimeOut (10 sec) NOK

Alert: Refer to Alert Management for more information.