Advanced ARTESCA Installer Configuration

ARTESCA installer requires an input configuration file. This file has all the installation information, and each step has its own dedicated section.

Platform Section

  1. /var/lib/containerd and /var/lib/etcd must be on dedicated partitions. The installer can create LVM LogicalVolume for those two mount points automatically on a specified LVM VolumeGroup if you want to use LVM (otherwise you can create and mount those partitions manually, and mount them at the right place). For each LVM LogicalVolume you must provide a size (e.g.: 100G) and a vgName.

  2. Define appropriate values for the following Bootstrap configuration points:

    • networks are used to specify the IP address ranges used in the cluster, defined in CIDR notation:

      • controlPlane and workloadPlane entries are mandatory. These IP ranges will be used at the host level for each cluster member. All servers in the cluster must be connected to both of these networks. The same range may be used for both controlPlane and workloadPlane.

      • For controlPlane entry, an ingress can also be provided. This section allow to set the IP that will be used to connect to all the control plane components, like control plane UI and the whole monitoring stack. We suggest using a Virtual IP that will sit on a working control plane Node. The default value for this Ingress IP is the control plane IP of the Bootstrap node (which means that if you lose the Bootstrap node, you no longer have access to any control plane component).

      • A value for MTU can also be defined for the workloadPlane network. This value must be the lowest MTU accross all the links in this network, and defaults to 1460 (1500 - 40, where 40 is the size of an IP-in-IP encapsulation header).

      • pods and services are virtual networks, only used internally to the cluster. They are optional configurations, and can be used to match the constraints of existing networking infrastructure (for instance, if all or part of the default subnets are already routed). The default values are:

        networks:
  pods: 10.233.0.0/16
  services: 10.96.0.0/12

    • proxies key is used to configure proxies between the deployed cluster and the public Internet (used for installing system packages from the core OS repositories and pulling container images). For convenience, it follows Linux conventions for the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables.

  3. Define the list of extra nodes to deploy.

    Note

    You do not need to put the node on which you will run the installer. This one is automatically used as bootstrap node, except if you want to override its name.

    For each node, you need to define the following options:

    • node_N_name the name of the node.

    • host a hostname or an IP to reach this node using SSH from the bootstrap (local) node.

    • port the port to connect to the node via SSH (defaults to 22).

    • key the SSH private key to use to connect to this node with SSH. This SSH key must be accessible from MetalK8s salt-master container. Scality recommends putting the SSH private key in the /etc/metalk8s/pki directory.

    • user the user to connect to the node via SSH (defaults to artesca-os).

    • sudo whether or not you need to use sudo to escalate privileges on the node with the defined user (defaults to true).

    All these options can also be passed through the CLI, when running the config generate command, using the following syntax:

    [artesca-os@bootstrap ~]$

    ./artesca-installer config generate \
    --architecture 3-nodes \
    --node '{name: node-02, host: 10.10.1.2, user: <ssh_user>, key: /etc/metalk8s/pki/id_rsa, sudo: true, port: 22}' \
    --node '{name: node-03, host: 10.10.1.3, user: <ssh_user>, key: /etc/metalk8s/pki/id_rsa, sudo: true, port: 22}'

    To customize the local node’s (bootstrap) name, use the --local-node-name option.

  4. The platform section from Installer should look like this, when replacing the placeholders with the values picked earlier.

    platform:
 etcd_lv:
   vgName: vgartesca
   size: 5G

 bootstrap:
   networks:
     controlPlane:
       cidr: <CIDR-notation>
     workloadPlane:
       cidr: <CIDR-notation>
       mtu: <network-MTU>
     pods: <CIDR-notation>
     services: <CIDR-notation>
   proxies:
     http: <http://proxy-ip:proxy-port>
     https: <https://proxy-ip:proxy-port>
     no_proxy:
       - <host>
       - <ip/cidr>

 # This section is optional and only needed when you want to deploy
 # multiple nodes, it mostly lists the "extra nodes" SSH information.
 # The local node (bootstrap) does not need to be part of this list, except
 # if you want to override its name which defaults to its hostname.
 nodes:
   <local_node_name>: {}
   <node_2_name>:
     # SSH information to reach this node from Bootstrap node
     ssh:
       host: <node-ip-or-hostname>
       port: 22
       key: <ssh-key-path>
       user: root
       sudo: false
   <node_3_name>:
     ssh:
       host: <node-ip-or-hostname>
       port: 22
       key: <ssh-key-path>
       user: root
       sudo: false

 # This section is optional and only needed if you want to scale the infra
 # services, only needed when you run in multi node
 services:
   prometheus:
     spec:
       deployment:
         replicas: 3
   alertmanager:
     spec:
       deployment:
         replicas: 3
   loki:
     spec:
       deployment:
         replicas: 3

    Tip

    A simplified example, when running on single-node, where no proxy is required and both control and workload planes use the same network (10.200.0.0/16), with etcd and containerd partitions created and mounted manually, looks like this:

    platform:
  bootstrap:
    networks:
      controlPlane:
        cidr: 10.200.0.0/16
      workloadPlane:
        cidr: 10.200.0.0/16

Volumes Section

ARTESCA installer can create all the necessary Kubernetes volumes. To do so, describe the volumes in the input file, as follows:

  • A volume-name which is the name of the volume.

  • A storage-class which is the name of the Kubernetes storage class to use to format the volume.

  • Volume type with required information:

    • use rawBlockDevice to create a MetalK8s volume from a device on the host. It needs a device-path which is the path to this block device (for example: /dev/sdb).

      volumes:
  <volume-name>:
    storageClassName: <storage-class>
    rawBlockDevice:
      devicePath: <device-path>

    • Use lvmLogicalVolume to create a LVM Logical Volume from an existing LVM Volume Group that will be used by the resulting MetalK8s volume. It needs a vg-name which is the name of the LVM VG and a size (for example: 10G)

      volumes:
  <volume-name>:
    storageClassName: <storage-class>
    lvmLogicalVolume:
      vgName: <vg-name>
      size: <size>

  • Any other spec that can be passed to the MetalK8s volume.

Here is an example of all needed volumes for a single-node deployment (HPE DL325):

volumes:
  # MetalK8s infra volumes
  prometheus:
    storageClassName: metalk8s
    lvmLogicalVolume:
      vgName: vgartesca
      size: 20Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: 'prometheus-operator-prometheus'

  alertmanager:
    storageClassName: metalk8s
    lvmLogicalVolume:
      vgName: vgartesca
      size: 2Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: 'prometheus-operator-alertmanager'

  loki:
    storageClassName: metalk8s
    lvmLogicalVolume:
      vgName: vgartesca
      size: 20Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: 'loki'

  # Keycloak volume
  keycloak-postgresql:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 10Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: keycloak-postgresql

  # XCore volumes
  storage-index:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 10Gi
    template:
      metadata:
        labels:
          xcore.scality.com/volume-type: index

  storage-data-1: &storage-data
    storageClassName: hdd-ext4
    rawBlockDevice:
      devicePath: /dev/sdb
    template:
      metadata:
        labels:
          xcore.scality.com/volume-type: data
  storage-data-2:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdc
  storage-data-3:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdd
  storage-data-4:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sde
  storage-data-5:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdf
  storage-data-6:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdg
  storage-data-7:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdh
  storage-data-8:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdi
  storage-data-9:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdj
  storage-data-10:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdk

  # Zenko volumes
  data-local:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 10Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: localData
          app.kubernetes.io/part-of: zenko

  data-zookeeper:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 5Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: zookeeper
          app.kubernetes.io/part-of: zenko

  data-kafka:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 20Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: kafka
          app.kubernetes.io/part-of: zenko

  data-mongodb:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 50Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: mongodb
          app.kubernetes.io/part-of: zenko

Base Section

ARTESCA installer deploys all ARTESCA Base services that will be used by other ARTESCA components. To deploy those services, provide a storage-class for Keycloak PostgreSQL database in the input file.

The base section from Installer should look like this (replacing placeholders with the values picked earlier):

base:
  keycloak:
    storageClassName: ssd-raid1-ext4

Storage Section

ARTESCA’s “storage” installation step deploys the Storage services with an operator. To trigger its deployment, describe the cluster in the input file, as follows:

  1. storage.spec: Contains the specification of the cluster to create, that includes the specifications for the network controller and the general specifications for the server (that applies to all servers).

    • protection.schema: the protection scheme list. There must be a single replication schema on a single server cluster. On a multiple server cluster, there must be at most one replication schema and at most one erasure code schema.

      • type values can be either replication or isa-l.

        • replication: replicates the values on all servers, implies the presence of the field copies.

        • isa-l: uses erasure coding over the network, implies the presence of the fields data and coding.

      • copies: value represent the number of copies over the network, only used with type = replication.

      • data: number of data parts over the network, only used with type = isa-l.

      • coding: number of coding parts over the network, only used with type = isa-l.

      • partsize: size of a part.

        Note

        Each large payload will be split into smaller chunks. The split threshold is partsize for a replication schema and data * partsize for an erasure code scheme. Each chunk is then replicated or erasure coded into partsize parts distributed to servers.

        Note

        The part size of last chunk may be smaller than partsize.

    • traits: list of pre-defined server configuration settings to apply. Possible values:

      • sync_ec or small_sync_ec: mandatory for all setups.

        Note

        • The small_sync_ec is for small setup (usualy VM setupts for tests).

      • small: for small setup (usualy VM setupts for tests).

      • low_overhead: reduce the local disk usage overhead on the servers.

      • [ ]: am empty list when no trait is set, this is the default value for multi-nodes cluster.

    • prometheus:

      • enable: true enables prometheus metrics export, false disables it.

      • serviceMonitorLabels: list of labels to assign to the serviceMonitor resource in order to be found by prometheus.

    • repair: network repair settings.

    • proxy: network proxy settings.

      • requestTimeout: timeout of each request made to a server.

      • slowRequestThreshold: timeout to flag a request as slow request.

      • replicationThreshold: any payload lower than this size (bytes) should be replicated.

      Note

      This is a size threshold to choose a replication or erasure code schema to protect data at the network level. At the server level, a size threshold of 65536 bytes specifies a replication for small parts and a local erasure code for large parts.

      • Replicas: number of pods running the network controller, defaults to 1.

    • version: the current version of the cluster to run.

      Note

      Images matching that version must be available on the cluster.

    • controller:

      • image:

        • forcedImageName: force the image name for the network controller.

        • version: force the version for the network controller.

        • pullPolicy: force the pod pull policy for the network controller.

    • advancedParametersForExtending: contains parameters used only for extending with new servers.

      • allowBalanceData: true enables balancing data between servers, false (default) disables it.

      • startMoveData: true starts data balance, false (default) keep data unballanced.

      • groupDensityRatio: modifies the metric that is defined as the average number of storage groups (SGs) per server. Possible values:

        • “0”: Do not create new SG.

        • “1” (default): keep the same metrics as before the extension by creating new SGs.

        • “1.5”: increase the metrics by 50% of the metric by creating new SGs.

        • “2”: double the metrics by creating new SGs.

  2. storage.servers List of servers for the current cluster. Each server entry in the list is described as follows:

    • diskMode: disk mode to use for that server.

    • nodeName: name of the node where that server will run.

      • image:

        • forcedImageName: force the image name for that server.

        • version: force the version for that server.

        • pullPolicy: force the pull policy for that server.

    • index: description of the disk used for that server’s index.

    • data: list of disks description for data storage.

      index and data have the same structure:

      • name: arbitrary name of that disk.

      • persistentVolume: persistent volume description. Cannot be used in conjonction with hostPath.

        • sotrageClassName: name of the storage class to use for that persistent volume.

        • selector:

          • matchLabel: list of labels (key-value pairs) to match a persistent volume for that disk.

      • hostPath: path on the host to use for that disk, instead of using a persistent volume. Cannot be used in conjonction with persistentVolume.

Bellow is an example of the storage section of the installer input file on a 3 nodes cluster, based on the configuration for HPE DL325.

storage:
  spec:
    protection:
    - schema:
        copies: 3
        partsize: 1024000
        type: replication
    traits: []
    prometheus:
      enable: true
    repair:
      type: kafka
      brokers:
      - xcore-kafka-headless:29092
      topic: repair
      partition: 1
      replica: 3
    proxy:
      requestTimeout: 10000
      slowRequestThreshold: 1000
    servers:
    - index:
        name: index
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: index
      data:
      - name: data-1
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-2
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-3
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-4
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-5
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-6
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-7
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-8
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-9
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-10
    - nodeName: node01
      index:
        name: index
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: index
      data:
      - name: data-1
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-2
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-3
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-4
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-5
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-6
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-7
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-8
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-9
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-10
    - nodeName: node02
      index:
        name: index
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: index
      data:
      - name: data-1
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-2
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-3
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-4
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-5
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-6
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-7
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-8
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-9
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-10

Data Section

The ARTESCA installer’s “data” step deploys the data services with all needed operators. To do so, describe the Zenko deployment in the input file, as follows:

  • crSpec, overrides for the spec field of the Zenko CustomResource

The data section from the Installer’s input file, which looks like this on a single server (HPE DL325):

data:
  crSpec:
    replicas: 1
    vault:
      enable: true
      iamIngress:
        hostname: iam.my-company.com
      stsIngress:
        hostname: sts.my-company.com
      replicas: 4
    internalCloudserver:
      enable: true
    cloudserver:
      replicas: 16
    scuba:
      api:
        ingress:
          hostname: utilization.my-company.com
    sorbet:
      enable: true
    management:
      provider: InCluster
      api:
        ingress:
          hostname: management.my-company.com
        allowFrom:
          - 172.16.0.0/12
          - 10.0.0.0/8
    ingress:
      workloadPlaneClass: 'nginx'
      controlPlaneClass: 'nginx-control-plane'
      annotations:
        nginx.ingress.kubernetes.io/proxy-body-size: 0m
        nginx.ingress.kubernetes.io/proxy-buffering: "off"
        nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
    mongodb:
      provider: KubeDB
      persistence:
        volumeClaimTemplate:
          size: 20G
          storageClassName: ssd-raid1-ext4
          selector:
            matchLabels:
              app.kubernetes.io/name: mongodb
              app.kubernetes.io/part-of: zenko
    redis:
      provider: Zenko
    kafka:
      provider: Managed
      persistence:
        volumeClaimTemplate:
          size: 10G
          storageClassName: ssd-raid1-ext4
          selector:
            matchLabels:
              app.kubernetes.io/name: kafka
              app.kubernetes.io/part-of: zenko
    zookeeper:
      provider: Managed
      persistence:
        volumeClaimTemplate:
          size: 5G
          storageClassName: ssd-raid1-ext4
          selector:
            matchLabels:
              app.kubernetes.io/name: zookeeper
              app.kubernetes.io/part-of: zenko
    localData:
      persistence:
        volumeClaimTemplate:
          size: 10G
          storageClassName: ssd-raid1-ext4
          selector:
            matchLabels:
              app.kubernetes.io/name: localData
              app.kubernetes.io/part-of: zenko
    initialConfiguration:
      locations:
      - name: us-east-1
        type: location-scality-hdclient-v2
        details:
          bootstrapList:
          - "local-storage-service-proxy.xcore.svc:18888"
      s3API:
        endpoints:
        - hostname: s3.my-company.com
          locationName: us-east-1

Full Installer Input File

The full Installer Input file should look like this (replacing placeholders with the values picked earlier, and potentially removing optional fields):

apiVersion: artesca.scality.com/v1alpha2
kind: InputConfiguration

# Platform configuration
platform:
  etcd_lv:
    vgName: vgartesca
    size: 5G

  bootstrap:
    networks:
      controlPlane:
        cidr: <CIDR-notation>
      workloadPlane:
        cidr: <CIDR-notation>
        mtu: <network-MTU>
      pods: <CIDR-notation>
      services: <CIDR-notation>
    proxies:
      http: <http://proxy-ip:proxy-port>
      https: <https://proxy-ip:proxy-port>
      no_proxy:
        - <host>
        - <ip/cidr>

  # This section is optional and only needed when you want to deploy
  # multiple nodes, it mostly lists the "extra nodes" SSH information.
  # The local node (bootstrap) does not need to be part of this list, except
  # if you want to override its name which defaults to its hostname.
  nodes:
    <local_node_name>: {}
    <node_2_name>:
      # SSH information to reach this node from Bootstrap node
      ssh:
        host: <node-ip-or-hostname>
        port: 22
        key: <ssh-key-path>
        user: root
        sudo: false
    <node_3_name>:
      ssh:
        host: <node-ip-or-hostname>
        port: 22
        key: <ssh-key-path>
        user: root
        sudo: false

  # This section is optional and only needed if you want to scale the infra
  # services, only needed when you run in multi node
  services:
    prometheus:
      spec:
        deployment:
          replicas: 3
    alertmanager:
      spec:
        deployment:
          replicas: 3
    loki:
      spec:
        deployment:
          replicas: 3

# Volumes configuration
volumes:
  # MetalK8s infra volumes
  prometheus:
    storageClassName: metalk8s
    lvmLogicalVolume:
      vgName: vgartesca
      size: 20Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: 'prometheus-operator-prometheus'

  alertmanager:
    storageClassName: metalk8s
    lvmLogicalVolume:
      vgName: vgartesca
      size: 2Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: 'prometheus-operator-alertmanager'

  loki:
    storageClassName: metalk8s
    lvmLogicalVolume:
      vgName: vgartesca
      size: 20Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: 'loki'

  # Keycloak volume
  keycloak-postgresql:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 10Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: keycloak-postgresql

  # XCore volumes
  storage-index:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 10Gi
    template:
      metadata:
        labels:
          xcore.scality.com/volume-type: index

  storage-data-1: &storage-data
    storageClassName: hdd-ext4
    rawBlockDevice:
      devicePath: /dev/sdb
    template:
      metadata:
        labels:
          xcore.scality.com/volume-type: data
  storage-data-2:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdc
  storage-data-3:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdd
  storage-data-4:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sde
  storage-data-5:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdf
  storage-data-6:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdg
  storage-data-7:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdh
  storage-data-8:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdi
  storage-data-9:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdj
  storage-data-10:
    <<: *storage-data
    rawBlockDevice:
      devicePath: /dev/sdk

  # Zenko volumes
  data-local:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 10Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: localData
          app.kubernetes.io/part-of: zenko

  data-zookeeper:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 5Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: zookeeper
          app.kubernetes.io/part-of: zenko

  data-kafka:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 20Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: kafka
          app.kubernetes.io/part-of: zenko

  data-mongodb:
    storageClassName: ssd-raid1-ext4
    lvmLogicalVolume:
      vgName: vgartesca
      size: 50Gi
    template:
      metadata:
        labels:
          app.kubernetes.io/name: mongodb
          app.kubernetes.io/part-of: zenko

# Base services configuration
base:
  keycloak:
    storageClassName: ssd-raid1-ext4

# Storage services configuration
storage:
  spec:
    protection:
    - schema:
        copies: 3
        partsize: 1024000
        type: replication
    traits: []
    prometheus:
      enable: true
    repair:
      type: kafka
      brokers:
      - xcore-kafka-headless:29092
      topic: repair
      partition: 1
      replica: 3
    proxy:
      requestTimeout: 10000
      slowRequestThreshold: 1000
    servers:
    - index:
        name: index
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: index
      data:
      - name: data-1
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-2
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-3
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-4
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-5
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-6
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-7
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-8
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-9
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-10
    - nodeName: node01
      index:
        name: index
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: index
      data:
      - name: data-1
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-2
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-3
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-4
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-5
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-6
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-7
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-8
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-9
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-10
    - nodeName: node02
      index:
        name: index
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: index
      data:
      - name: data-1
        persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-2
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-3
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-4
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-5
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-6
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-7
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-8
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-9
      - persistentVolume:
          storageClassName: nvme-ext4
          selector:
            matchLabels:
              xcore.scality.com/volume-type: data
        name: data-10

# Data services configuration
data:
  crSpec:
    replicas: 1
    vault:
      enable: true
      iamIngress:
        hostname: iam.my-company.com
      stsIngress:
        hostname: sts.my-company.com
      replicas: 4
    internalCloudserver:
      enable: true
    cloudserver:
      replicas: 16
    scuba:
      api:
        ingress:
          hostname: utilization.my-company.com
    sorbet:
      enable: true
    management:
      provider: InCluster
      api:
        ingress:
          hostname: management.my-company.com
        allowFrom:
          - 172.16.0.0/12
          - 10.0.0.0/8
    ingress:
      workloadPlaneClass: 'nginx'
      controlPlaneClass: 'nginx-control-plane'
      annotations:
        nginx.ingress.kubernetes.io/proxy-body-size: 0m
        nginx.ingress.kubernetes.io/proxy-buffering: "off"
        nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
    mongodb:
      provider: KubeDB
      persistence:
        volumeClaimTemplate:
          size: 20G
          storageClassName: ssd-raid1-ext4
          selector:
            matchLabels:
              app.kubernetes.io/name: mongodb
              app.kubernetes.io/part-of: zenko
    redis:
      provider: Zenko
    kafka:
      provider: Managed
      persistence:
        volumeClaimTemplate:
          size: 10G
          storageClassName: ssd-raid1-ext4
          selector:
            matchLabels:
              app.kubernetes.io/name: kafka
              app.kubernetes.io/part-of: zenko
    zookeeper:
      provider: Managed
      persistence:
        volumeClaimTemplate:
          size: 5G
          storageClassName: ssd-raid1-ext4
          selector:
            matchLabels:
              app.kubernetes.io/name: zookeeper
              app.kubernetes.io/part-of: zenko
    localData:
      persistence:
        volumeClaimTemplate:
          size: 10G
          storageClassName: ssd-raid1-ext4
          selector:
            matchLabels:
              app.kubernetes.io/name: localData
              app.kubernetes.io/part-of: zenko
    initialConfiguration:
      locations:
      - name: us-east-1
        type: location-scality-hdclient-v2
        details:
          bootstrapList:
          - "local-storage-service-proxy.xcore.svc:18888"
      s3API:
        endpoints:
        - hostname: s3.my-company.com
          locationName: us-east-1