Recognize and Resolve Maximum Transmission Unit Issues¶

Problem Description¶

A Maximum Transmission Unit (MTU) issue occurs when the size of packets exceeds the maximum size that network devices can handle.

The following common symptoms can be noticed:

Performance issues: The network experiences noticeable slowdowns.
Packet loss: Packets are being dropped by switches, routers, or network adapters within the ARTESCA cluster nodes.
Connection instability: The network connections are unreliable or frequently disrupted.
Network adapter errors: Network adapters report a significantly increased number of errors.

If you notice one or more of these symptoms, follow the Investigation Steps to confirm the issue.

Investigation Steps¶

On ARTESCA nodes¶

Apply the Ping Method¶

The ping command can be used to help diagnose MTU issues.

This method consists in sending large, non-fragmented SMTP packets against the cluster nodes, and checking if the number of packets lost or network adapter errors increases.

Before proceeding, ensure that:

You run the ping command from a different device, because ARTESCA security policies block ICMP packets from being sent from cluster nodes.
The MTU values on all devices between the ARTESCA nodes and the device used for the ping method are large enough to support the packets without fragmentation.

Linux

[my-user@my-station ~]$

ping -c 10 -M do -s 8972 <NODE_IP>

PING <NODE_IP>: 8972 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
Request timeout for icmp_seq 6
Request timeout for icmp_seq 7
Request timeout for icmp_seq 8

-- <NODE_IP> ping statistics --
10 packets transmitted, 0 packets received, 100.0% packet loss

Windows

C:\>

ping -c 10 -f -l 8972 <NODE_IP>

PING <NODE_IP>: 8972 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
Request timeout for icmp_seq 6
Request timeout for icmp_seq 7
Request timeout for icmp_seq 8

-- <NODE_IP> ping statistics --
10 packets transmitted, 0 packets received, 100.0% packet loss

Because ARTESCA firewall rules block ICMP packets, you cannot receive ping responses from nodes. Continue with Check the Network Adapter Configuration and Statistics to get more information.

Check the Network Adapter Configuration and Statistics¶

Run the following command to get a wide overview of the setup and state of physical network adapters on ARTESCA nodes.

[artesca-os@<NODE_NAME> ~]$

PHY_NICS=$(ls -l /sys/class/net/ | grep pci | awk '{print $9}') && echo $PHY_NICS
for nic in $PHY_NICS; do echo "$nic -> TX_ERR=$(cat /sys/class/net/$nic/statistics/tx_errors), RX_ERR=$(cat /sys/class/net/$nic/statistics/rx_errors)"; done
for nic in $PHY_NICS; do echo "=========="; echo "-> $nic"; sudo ethtool -i $nic; sudo ethtool -g $nic; sudo ethtool -k $nic; sudo ethtool -S $nic; done

The output depends on the model of the network adapter and the driver in use.

In example 1, the network adapters use the bnxt_en (for Broadcom adapters) driver and they report an histogram of frames distributed by size, as well as an oversized frame metric.

In example 2, the network adapters use the igb (for Intel adapters) driver and they report a metric called rx_long_length_errors.

Example 1¶

-> ens1f0np0
    driver: bnxt_en
...
NIC statistics:
...
    [0]: rx_errors: 45082 <---| Reception errors - equals in quantity to frames having from 4096 to 9216 bytes.
...
    rx_64b_frames: 1109702590
    rx_65b_127b_frames: 26140224
    rx_128b_255b_frames: 184787935
    rx_256b_511b_frames: 4303029
    rx_512b_1023b_frames: 36365874
    rx_1024b_1518b_frames: 5605162165
    rx_1519b_2047b_frames: 0
    rx_2048b_4095b_frames: 0
    rx_4096b_9216b_frames: 45082 <---| Frames having from 4096 to 9216 bytes.
    rx_9217b_16383b_frames: 0
...
    rx_ovrsz_frames: 45082 <---| Oversized Frames - equals in quantity to frames having from 4096 to 9216 bytes.

Example 2¶

-> enp129s0f0
    driver: igb
...
NIC statistics:
...
     rx_long_length_errors: 45081

On Other Devices in the Network¶

Look for metrics that can indicate issues related to frames that exceed the MTU value.

Problem Resolution¶

Locate which devices to reconfigure and which MTU value to use.

The MTU value must be the same across all devices in the network, including switches, routers, and firewalls.

If consistent MTU values are not possible, ensure devices directly connected to the cluster share the same MTU value.
If you need to change the MTU value on the ARTESCA nodes, refer to Changing the MTU Value on ARTESCA Nodes.