Shaping system on PPPoE

Packet flow and traversing thru rules from internet to user

Packet marking in firewall.cfg

First packet will be processed thru iptables, /etc/firewall.cfg

In case we want to apply mark to packet we will use ipset + skbinfo module
This is optional step and used only if we want some bypass class to match many ips or ports.

For example:

ipset add bypass 213.204.127.171 skbmark 0x12 #iptv
ipset add bypass 185.99.35.252/30 skbmark 0x12 #iptv

ipset add bypass 185.118.26.192/26 skbmark 0x13 # ggc-aoun
ipset add bypass 185.118.26.128/26 skbmark 0x13 # fna-aoun

This means if packet with source ip will go thru iptables it will be marked with 0x12 or 0x13.

Condition of iptables that apply this mark is:

iptables -t mangle -A PREROUTING -i bond0 -j SET --map-set bypass src --map-mark

Packet evaluation in filters of ppp interface to user

Next step is to evaluate packet in filters of ppp interface to user.
Filters are described in /etc/config.json, section "traffic_filters".

For example:

 "bypass4": {
   "prio": 14,
   "classid": 25,
   "criteria": "handle 0x14 fw"
  },
  "ping": {
   "prio": 5,
   "classid": 10,
   "criteria": "u32 match ip protocol 1 0xff"
  },
  "dns": {
   "prio": 5,
   "classid": 10,
   "criteria": "u32 match ip protocol 17 0xff match ip sport 53 0xffff"
  },

Take attention on "prio" field, this is order how packet evaluated, from small to big number.
If prio is same, then packet will be evaluated in order of appearance in config file (but better to not rely on that).
So packet will be evaluated first in "ping" filter, then in "dns" filter, then in "bypass4" filter.
ping and dns filters are self-explanatory, they will match icmp and dns packets and send them to classid 10.
bypass4 will verify if packet is marked with 0x14, if yes, then it will be sent to classid 25.

There is usually default filter rule

  "default": {
   "prio": 200,
   "classid": 2,
   "criteria": "u32 match u32 0 0"
  }

This rule will match all packets that were not matched by previous rules and send them to classid 2.
Classid is special and match account "default" speed set in radius, it is usually "direct traffic".

Packet classification in traffic shaper

Now after we have assigned classid to packet, we will go to traffic shaper and apply shaping rules.
Classes are defined in traffic_classes section of /etc/config.json.
Take attention that "bypass2" of filter and class in traffic_classes are not the same.

"traffic_classes": {
  "bypass1": {
   "classid": 10,
   "parent": 0,
   "qdisc": "pfifo limit 100"
  },

This is classid 10, parent 0, qdisc pfifo limit 100. Qdisc allows us to set different disciplines for class, some (not pfifo) might be more friendly for gaming, of we can increase queue size of bulk traffic to reduce packetloss if user are full.
Parent is special, it allows to set tree like in Mikrotik traffic shaping by trees.
In our case parent: 0 means that this class is root class, it is not child of any other class.
If we set parent class 1 it means burst traffic will be limited by parent class 1 (which is equal to account speed).

How traffic shaper speeds are set

Speeds are set in "services" of /etc/config.json.

 "services": {
  "#default": {
   "activate_classes": [
    "bypass1",
    "bypass2",
    "bypass3",
    "bypass4",
    "default"
   ],
   "cir_classes": [
    "%300",
    "%300",
    "%200",
    "%100",
    "%100"
   ],
   "burst_classes": [
    "%300",
    "%300",
    "%200",
    "%100",
    "%100"
   ],
   "modifiers_classes": [
    "",
    "",
    "",
    "",
    ""
   ],
   "activate_filters": [
    "bypass1",
    "bypass2",
    "bypass3",
    "bypass4",
    "ping",
    "dns",
    "default"
   ]
  }

Interesting part here is "cir_classes" and "burst_classes". They should match by sequence number to "activate_classes".
So for example "default" class will be set to cir 100% and burst 100% of account speed. (last value)
Possible values:

%XXX - percentage of account speed. So 5Mbit account with value %200 will have 10Mbit cir and burst.
#N - bypass N set by radius attribute Login-LAT-Node, for example "speed:1000/3000/6000" means #0 is 1000, #1 is 3000, #2 is 6000.
NNNN - fixed speed, for example 1000 is 1Mbit

So basically if you want to change bypass2 to bypass from radius, you need to change in cir_classes and burst_classes value to #, which by previous example will be 3000.

How to verify radius attributes

Find user ppp interface name, for example pppX, by running:

 accel-cmd show sessions|grep username

Then execute:

cat /var/run/radattr.pppX

This will show you all radius attributes for this user.

How to verify traffic classes of user

tc -s -d class show dev pppX

class htb 1:1 root rate 3072Kbit ceil 3072Kbit linklayer ethernet burst 1999999b/1mpu 0b cburst 1999999b/1mpu 0b level 7 
 Sent 76855085 bytes 74998 pkt (dropped 0, overlimits 2 requeues 0) 
 backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: 81273324 ctokens: 81273324

class htb 1:10 root leaf 10prio 0 quantum 30000 rate 9216Kbit ceil 9216Kbit linklayer ethernet burst 1999998b/1mpu 0b cburst 1999998b/1mpu 0b level 0 
 Sent 153094 bytes 838 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
 lended: 838 borrowed: 0 giants: 0
 tokens: 27125025 ctokens: 27125025

class htb 1:2 parent 1:1 leaf 2prio 0 quantum 30000 rate 3072Kbit ceil 3072Kbit linklayer ethernet burst 1999999b/1mpu 0b cburst 1999999b/1mpu 0b level 0 
 Sent 76855085 bytes 74998 pkt (dropped 114, overlimits 470 requeues 0) 
 backlog 0b 0p requeues 0
 lended: 71295 borrowed: 0 giants: 0
 tokens: 81273324 ctokens: 81273324

class htb 1:20 root leaf 20prio 0 quantum 30000 rate 6144Kbit ceil 6144Kbit linklayer ethernet burst 1999998b/1mpu 0b cburst 1999998b/1mpu 0b level 0 
 Sent 185321931 bytes 147757 pkt (dropped 67, overlimits 458 requeues 0) 
 backlog 0b 0p requeues 0
 lended: 145722 borrowed: 0 giants: 0
 tokens: 40660958 ctokens: 40660958

class htb 1:15 root leaf 15prio 0 quantum 30000 rate 9216Kbit ceil 9216Kbit linklayer ethernet burst 1999998b/1mpu 0b cburst 1999998b/1mpu 0b level 0 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: 27126734 ctokens: 27126734

class htb 1:25 root leaf 25prio 0 quantum 30000 rate 3072Kbit ceil 3072Kbit linklayer ethernet burst 1999999b/1mpu 0b cburst 1999999b/1mpu 0b level 0 
 Sent 48626547 bytes 37274 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
 lended: 37019 borrowed: 0 giants: 0
 tokens: 81372390 ctokens: 81372390

Classid match definition of classid in config.json.
If you execute command several times, take attention on counters, for example if you see value in backlog, it means that
traffic is being "buffered", so user use all speed for this class.
If counter in dropped is not zero, it means that user is sending more traffic than allowed by class and packets are dropped.
And sure you can match rate and ceil, which match cir and burst values.

7.1 KiB Raw Blame History