documentation/PPPOE_misc_tricks.md

# Misc tricks for GlobalOS

## trafw
Trafw allows to see bidirectional traffic statistics.

Usage: trafw interface1 "interface1 pcap filter" interface2 "interface2 pcap filter" timer

Example:
```
trafw ppp0 "inbound" ppp0 "outbound" 1
```
Will show inbound and outbound traffic on ppp0 every second.

## iptop

Usage: iptop interface "pcap filter" packets (dst|src) [p|b]

Example:
```
iptop eth0 "inbound" 10000 dst
```
Show top 20 destination ip address by rate. (top consumers)

## tcpdump and accel-cmd

To check user traffic you need to use tcpdump and accel-cmd.

To find out interface name of user you can use:
```
accel-cmd show sessions

pppoe-9 ~ # accel-cmd show sessions|more
 ifname  |    username     |    calling-sid    |       ip       | type  | comp | state  |   uptime    
---------+-----------------+-------------------+----------------+-------+------+--------+-------------
 ppp185  | user1           | 6c:3b:6b:73:33:11 | 172.17.16.185  | pppoe |      | active | 17.00:56:54 
 ppp305  | user2           | 50:0f:f5:40:22:22 | 172.17.17.49   | pppoe |      | active | 17.00:56:53 
 ppp318  | user3           | 6c:3b:6b:c4:11:33 | 172.17.17.62   | pppoe |      | active | 17.00:56:53 
```

Which means that user1 is connected to ppp185 interface, user2 to ppp305 and user3 to ppp318.
To check traffic of user1 you need to use:
```
tcpdump -ni ppp185 -vvv -c 100
```
Which means:
* -n - do not resolve ip addresses (we don't need it)
* -i ppp185 - listen on ppp185 interface
* -vvv - verbose output
* -c 100 - capture 100 packets and exit

## pcap filters

You can use various pcap filters in trafw, iptop and tcpdump.
For example if you want to see only traffic from subnet 8.8.8.0/24 and source port 53 you can use:
```
# to show total traffic for this filter
trafw ppp0 "src net 8.8.8.0/24 and src port 53" ppp0 "src net 8.8.8.0/24 and src port 53" 1
# to show top consumers of such traffic
iptop eth0 "src net 8.8.8.0/24 and src port 53" 10000 dst
# to capture such traffic and see each packet, only 100 packets
tcpdump -ni ppp185 -vvv -c 100 "src net 8.8.8.0/24 and src port 53"

More info, in articles:
https://iphelix.medium.com/packet-filtering-techniques-84fc3fc2ea3b

## restarting accel-pppd

If you need to restart accel-pppd you can use:
```
killall accel-pppd
```
then wait until process is killed, it might take up to few minutes on large pppoe.
To check if accel-pppd is running you can use:
```
ps aux|grep accel-pppd
```
If you see accel-pppd process you can start it with:
```
/usr/sbin/accel-pppd -c /etc/accel-ppp.conf -p /var/run/accel-ppp.pid -d
```

## Shell tricks

For example i want to filter all lines that begin with "fadi" in users list:
```
accel-cmd show sessions username|grep "^ fadi"
```
What does it mean and how it works?

* accel-cmd show sessions username - will show all sessions, but username field will be shown only
* | - pipe, it will pass output of previous command to next command
* grep "^ fadi" - will filter only lines that begin with " fadi", symbol ^ means beginning of line, space is needed to filter accel-specific output (it has space before username)

Now, for example i want to save this list and disconnect all users that begin with "fadi":
```
accel-cmd show sessions username|grep "^ fadi"|awk '{print $1}'|xargs -I {} accel-cmd terminate username {}
```

What does it mean and how it works?

* awk '{print $1}' - will print first field of each line, which is username, it will remove leading space
* xargs -I {} - will pass each line to next command, {} is placeholder for line(variable)
* accel-cmd terminate username {} - will disconnect user by username

## Real IPs routing to multiple PPPoE servers

In situation when you have more than one PPPoE server and you want to route real IPs to them you can use proxy ARP trick.
How it works?

1. You route real IPs to your NAT or separate server (anything that can do direct route to interface), for example real subnet a.b.c.0/24
2. On nat server you have interface facing PPPoE servers, for example bond0.1234, with ip 10.100.100.1/24. where pppoe servers are 10.100.100.10/24, 10.100.100.11/24 10.100.100.12/24. You add route:
```
ip route add a.b.c.0/24 dev bond0.1234
```
3. On pppoe servers you have interfaces facing pppoe server, for example eth4.999 10.100.100.10/24 on first one. You need to execute following commands:
```
sysctl -w net.ipv4.conf.eth4/999.proxy_arp=1

or if interface eth0:
sysctl -w net.ipv4.conf.eth0.proxy_arp=1
```

How this does work?
- When packet comes to your router, it is routed to NAT (or separate server) because of route.
- NAT server sees that packet is for a.b.c.0/24 and sends ARP request "WHERE IS a.b.c.1?" to bond0.1234 because it has route to a.b.c.0/24 over bond0.1234 directly.
- Each PPPoE server due proxy_arp entry will check, do i have a.b.c.1? If yes, it will respond with its own MAC address.
- NAT server will send packet to PPPoE server with a.b.c.1 and PPPoE server will process it.

This way you can route real IPs to multiple PPPoE servers without need of BGP or other routing protocols. It is simple and works well, but have one caveat, if old entry exist and user changed pppoe, it might need timeout (max 5min) so ARP entry expires.