Disable or Change Resource Limits for Data Services MongoDB

Problem Description

ARTESCA uses MongoDB within Data Services. If you encounter the symptoms or limits listed below, you may need to change or disable the existing resource limits.

  • ARTESCA’s memory is under-utilized in the long term.

  • ARTESCA has a non-standard deployment, requiring service reconfiguration.

Warning

ARTESCA follows the MongoDB’s best practices for production environments. Modifying resource limits can affect the service’s performance and stability.

Root Cause

ARTESCA enforces a default and dynamic configuration based on the cluster’s resources. This configuration is applied to the MongoDB statefulsets and enabled by default.

The resource allocation for both the MongoDB and Mongos services is configured to adhere to the recommended threshold, which is 50% of the cluster’s RAM minus 1GB. Specifically, ARTESCA allocates the cluster’s total memory resources as follows:

  • 16% for each Mongos data service.

  • 34% for each shard server data service.

  • 17% for each shard server cache service (which is 50% of the allocated memory for each shard server).

  • Kubernetes memory requests are configured to be 10% of the allocated memory limits.

Problem Resolution

Warning

Any custom configuration applied to MongoDB’s statefulsets will be overwritten during the next upgrade.

  1. Temporarily modify MongoDB resource limits:

    [artesca-os@bootstrap ~]$

    MEMORY_LIMIT_SHARD_DATA="48Gi" && \
MEMORY_REQUEST_SHARD_DATA="8Gi" && \
MEMORY_LIMIT_MONGOS="36Gi" && \
MEMORY_REQUEST_MONGOS="4Gi" && \
for shard in $(sudo kubectl --kubeconfig /etc/kubernetes/admin.conf get statefulsets -n zenko | grep 'data-db-mongodb-sharded-shard[0-9]' | awk '{print $1}'); do \
      sudo kubectl --kubeconfig /etc/kubernetes/admin.conf set resources sts/$shard -n zenko --limits=memory=$MEMORY_LIMIT_SHARD_DATA --requests=memory=$MEMORY_REQUEST_SHARD_DATA; \
done && \
sudo kubectl --kubeconfig /etc/kubernetes/admin.conf set resources sts/data-db-mongodb-sharded-mongos -n zenko --limits=memory=$MEMORY_LIMIT_MONGOS --requests=memory=$MEMORY_REQUEST_MONGOS

    Tip

    To remove any limit, set the values to "0".

  2. Adjust the MongoDB WiredTiger cache size:

    1. Edit the data-db-mongodb-sharded-shard<shard>-data statefulset:

      [artesca-os@bootstrap ~]$

      for shard in $(sudo kubectl --kubeconfig /etc/kubernetes/admin.conf get statefulsets -n zenko | grep 'data-db-mongodb-sharded-shard[0-9]' | awk '{print $1}'); do \
   sudo kubectl --kubeconfig /etc/kubernetes/admin.conf edit sts/$shard -n zenko; \
done

    2. Locate the spec.template.spec.containers[0].env section and update the following flag, where the unit is in Gigabytes:

      • Value: --wiredTigerCacheSizeGB=36

    Tip

    To remove any limit, remove the flag.