documentation/networking.md

Bonding

Typical bonding configuration

modprobe i40e
modprobe bonding
ip link add dev bond0 type bond
ip link set dev eth4 down
ip link set dev eth5 down
ip link set dev eth6 down
ip link set dev eth7 down
ip link set dev bond0 down
echo 802.3ad >/sys/devices/virtual/net/bond0/bonding/mode
echo "layer3+4" >/sys/devices/virtual/net/bond0/bonding/xmit_hash_policy
ip link set dev eth4 master bond0
ip link set dev eth5 master bond0
ip link set dev eth6 master bond0
ip link set dev eth7 master bond0
ip link set dev eth4 up
ip link set dev eth5 up
ip link set dev eth6 up
ip link set dev eth7 up
ip link set dev bond0 up

Cisco Nexus

interface port-channel4
  description To-GlobalOS
  switchport mode trunk
  switchport trunk native vlan 2701
  switchport trunk allowed vlan 2584-2591,2701,2798,2901-2902
  no negotiate auto

interface Ethernet1/21
  description To-GlobalOS
  switchport mode trunk
  switchport trunk native vlan 2701
  switchport trunk allowed vlan 2584-2591,2701,2798,2901-2902
  channel-group 4 mode active

interface Ethernet1/22
  description To-GlobalOS
  switchport mode trunk
  switchport trunk native vlan 2701
  switchport trunk allowed vlan 2584-2591,2701,2798,2901-2902
  channel-group 4 mode active

Troubleshooting

Packetloss

• This kind of fast ping need to be applied only on ethernet links with MTU 1500. No packetloss should be present.

ping -c 1000 -i 0.01 -s1472 127.0.0.1
ping -c 1000 -i 0.01 -s1472 nearby.ip

• Check if any dropped counter are increasing

tc -s -d qdisc show

• Check SFP signal level (if SFP supports DDM/DOM)

ethtool -m eth4

You might need to tune icmp rate limit if too many people ping this host:

net.ipv4.icmp_msgs_per_sec = 10000
net.ipv4.icmp_ratelimit = 100000
net.ipv4.icmp_msgs_burst=1000

Making "fake ping"

docker-compose.yaml

version: "2.4"

services:
  fakeping:
    container_name: fakeping
    build: ./fakeping
    networks:
      mgmnt-net:
        ipv4_address: 10.0.252.19
    cap_add:
      - NET_ADMIN
    restart: always

networks:
  mgmnt-net:
    name: mgmnt-net
    driver: macvlan
    driver_opts:
      parent: eno1
    ipam:
      config:
        - subnet: "10.0.252.0/24"
          gateway: "10.0.252.1"

Dockerfile in fakeping directory

FROM ubuntu/kinetic:latest

RUN apt-get update && apt-get install -y iproute2 iputils-ping

ADD entrypoint2.sh /tmp
WORKDIR /root
ENTRYPOINT ["/tmp/entrypoint2.sh"]

File entrypoint2.sh

#!/bin/sh
tc qdisc del dev eth0 root
tc qdisc add dev eth0 root handle 1:0 htb default 10
tc class add dev eth0 parent 1:0 classid 1:10 htb rate 500Gbit ceil 500Mbit prio 0
tc qdisc add dev eth0 parent 1:10 handle 10: pfifo limit 10000
tc class add dev eth0 parent 1:0 classid 1:40 est 1sec 8sec htb rate 350Mbit ceil 350Mbit
tc qdisc add dev eth0 parent 1:40 handle 41: netem delay 40ms 10ms limit 90000
tc filter add dev eth0.11 protocol ip parent 1: prio 1 u32 match ip protocol 0x1 0xff flowid 1:40
# while loop
while true
do
    sleep 1
done