# PPPoE management guide ## Adding new vlan interface ``` interface=bond0.2529,{"services":["zone1","Flexnet","ferrari2","fi","rp","pr","dotnet"],"service-blank":1} ``` Possible options: * services - list of PPPoE services to be added to the interface * service-blank - if set to 1, then user that didnt specify service can connect to this pppoe * service-any - if set to 1, then user that specified any service can connect to this pppoe After updating accel-ppp config, you need to run script that will create vlans and add vlans to accel without restart: ``` sh /etc/add_accel_vlans.sh ``` ## Monitoring vlan utilization ``` ~ # accel-cmd pppoe interface show interface: connections: state: service-name policy: ----------------------------------------------------------- bond0.2622 0 active 3 bond0.2529 2251 active 3 bond0.368 530 active 3 bond0.923 0 active 1 ``` You can also check macs on /proc/net/pppoe, and exclude only those you want for specific vlan using grep: ``` ~ # cat /proc/net/pppoe|grep bond0.368 ``` Also handy command: ``` accel-cmd show sessions username,inbound-if,service-name ``` ### Removing vlan interface from accel-ppp instance live ``` accel-cmd pppoe interface del bond0.2529 ``` After that you need to monitor using "interface show" command, pppoe will slowly kill all sessions and then remove interface from accel-ppp instance. ### Modifying vlan interface on accel-ppp instance live This is not easy operation and will cause extended downtime due need to kill all sessions on interface and then add it back. You need to use "interface del" and "interface add" commands. You can use raw interface del, wait until all users gone, then update accel-ppp.conf, and use shell script to add interface back. **TODO** I have in development new version of pppoe that will allow to modify interface without downtime. ### Monitoring CPU usage ``` mpstat -P ALL 1 ``` If you notice significant disabalance - please run ``` cpubalancerd --rebalance ``` **TODO** I have cpubalancerd in development that will do this automatically in daemon mode, still it is under testing ### How PPPoE session is established 1. Client sends PADI packet to server, for example: ``` tcpdump -ni bond0.2529 -vvv -e pppoed 21:32:27.763589 6c:3b:6b:68:XX:XX (oui Unknown) > Broadcast, ethertype PPPoE D (0x8863), length 64: PPPoE PADI [Host-Uniq 0x0031004F] [Service-Name "TR"] 21:32:27.779473 bc:e0:01:25:XX:XX (oui Unknown) > Broadcast, ethertype PPPoE D (0x8863), length 60: PPPoE PADI [Service-Name "id"] [Host-Uniq 0x00001020] ``` First packet is from client that wants to connect to TR service, second packet is from client that wants to connect to id service. It might be also [Service-Name] - empty service name, which is called service-blank in accel-ppp config. 2. Server answers with PADO packet, for example: ``` 21:34:10.705827 90:e2:ba:ae:XX:XX (oui Unknown) > 08:55:31:82:XX:XX (oui Unknown), ethertype PPPoE D (0x8863), length 72: PPPoE PADO [AC-Name "servername"] [Service-Name "pr"] [AC-Cookie 0xC8D61ACB51BB81F6A75F2097CF120F1DA1AC93E4C97ED3BA] [Host-Uniq 0x00000F30] ``` Server answers with PADO packet, that contains AC-Name, Service-Name, AC-Cookie and Host-Uniq. Most of this entries usually not important in troubleshooting. As you can see here - client sends PADI with Host-Uniq 0x500FF5C517B8000005020600, and server answers with PADO with same Host-Uniq. This is how PPPoE session is established. 3. Client sends PADR packet to server, for example: ``` 21:35:47.143621 6c:3b:6b:81:XX:XX (oui Unknown) > 90:e2:ba:ae:XX:XX (oui Unknown), ethertype PPPoE D (0x8863), length 80: PPPoE PADR [Host-Uniq 0x500FF5C517B8000005020600] [AC-Cookie 0x6F71CC054D13DCE911CC1341F1FBDCE7A16830B204F2BEF7] [AC-Name "united"] [Service-Name "pr"] ``` Client sends PADR packet with Host-Uniq and AC-Cookie that he received in PADO packet. If server receives PADR with correct Host-Uniq and AC-Cookie - it will establish PPPoE session. 4. Server sends PADS packet to client, for example: ``` 21:35:47.143936 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:XX:XX (oui Unknown), ethertype PPPoE D (0x8863), length 52: PPPoE PADS [ses 0x8a48] [AC-Name "united"] [Service-Name "pr"] [Host-Uniq 0x500FF5C517B8000005020600] ``` 5. PPPoE session is established, and client can send PPP packets to server. It is more complicated, so i will post whole handshake here of user "VeryGoodUsername": ``` 21:42:26.848209 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:xx:xx (oui Unknown), ethertype PPPoE S (0x8864), length 41: PPPoE [ses 0x1807] LCP (0xc021), length 21: LCP, Conf-Request (0x01), id 107, length 21 encoded length 19 (=Option(s) length 15) 0x0000: c021 016b 0013 Auth-Prot Option (0x03), length 5: CHAP, MD5 0x0000: c223 05 MRU Option (0x01), length 4: 1440 0x0000: 05a0 Magic-Num Option (0x05), length 6: 0x193a92d5 0x0000: 193a 92d5 21:42:26.868270 6c:3b:6b:81:xx:xx (oui Unknown) > 90:e2:ba:ae:XX:XX (oui Unknown), ethertype PPPoE S (0x8864), length 60: PPPoE [ses 0x1807] LCP (0xc021), length 16: LCP, Conf-Request (0x01), id 1, length 16 encoded length 14 (=Option(s) length 10) 0x0000: c021 0101 000e MRU Option (0x01), length 4: 1480 0x0000: 05c8 Magic-Num Option (0x05), length 6: 0x194d41e9 0x0000: 194d 41e9 21:42:26.868295 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:xx:xx (oui Unknown), ethertype PPPoE S (0x8864), length 36: PPPoE [ses 0x1807] LCP (0xc021), length 16: LCP, Conf-Ack (0x02), id 1, length 16 encoded length 14 (=Option(s) length 10) 0x0000: c021 0201 000e MRU Option (0x01), length 4: 1480 0x0000: 05c8 Magic-Num Option (0x05), length 6: 0x194d41e9 0x0000: 194d 41e9 21:42:29.848211 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:xx:xx (oui Unknown), ethertype PPPoE S (0x8864), length 41: PPPoE [ses 0x1807] LCP (0xc021), length 21: LCP, Conf-Request (0x01), id 107, length 21 encoded length 19 (=Option(s) length 15) 0x0000: c021 016b 0013 Auth-Prot Option (0x03), length 5: CHAP, MD5 0x0000: c223 05 MRU Option (0x01), length 4: 1440 0x0000: 05a0 Magic-Num Option (0x05), length 6: 0x193a92d5 0x0000: 193a 92d5 21:42:29.905959 6c:3b:6b:81:xx:xx (oui Unknown) > 90:e2:ba:ae:XX:XX (oui Unknown), ethertype PPPoE S (0x8864), length 60: PPPoE [ses 0x1807] LCP (0xc021), length 16: LCP, Conf-Request (0x01), id 1, length 16 encoded length 14 (=Option(s) length 10) 0x0000: c021 0101 000e MRU Option (0x01), length 4: 1480 0x0000: 05c8 Magic-Num Option (0x05), length 6: 0x194d41e9 0x0000: 194d 41e9 21:42:29.905980 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:xx:xx (oui Unknown), ethertype PPPoE S (0x8864), length 36: PPPoE [ses 0x1807] LCP (0xc021), length 16: LCP, Conf-Ack (0x02), id 1, length 16 encoded length 14 (=Option(s) length 10) 0x0000: c021 0201 000e MRU Option (0x01), length 4: 1480 0x0000: 05c8 Magic-Num Option (0x05), length 6: 0x194d41e9 0x0000: 194d 41e9 21:42:29.908534 6c:3b:6b:81:xx:xx (oui Unknown) > 90:e2:ba:ae:XX:XX (oui Unknown), ethertype PPPoE S (0x8864), length 60: PPPoE [ses 0x1807] LCP (0xc021), length 21: LCP, Conf-Ack (0x02), id 107, length 21 encoded length 19 (=Option(s) length 15) 0x0000: c021 026b 0013 Auth-Prot Option (0x03), length 5: CHAP, MD5 0x0000: c223 05 MRU Option (0x01), length 4: 1440 0x0000: 05a0 Magic-Num Option (0x05), length 6: 0x193a92d5 0x0000: 193a 92d5 21:42:29.908557 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:xx:xx (oui Unknown), ethertype PPPoE S (0x8864), length 52: PPPoE [ses 0x1807] CHAP (0xc223), length 32: CHAP, Challenge (0x01), id 1, Value c879a7f4e8b3272656f5e2faa4e1d3ad, Name accel-ppp 21:42:29.965063 6c:3b:6b:81:xx:xx (oui Unknown) > 90:e2:ba:ae:XX:XX (oui Unknown), ethertype PPPoE S (0x8864), length 60: PPPoE [ses 0x1807] LCP (0xc021), length 10: LCP, Echo-Request (0x09), id 0, length 10 encoded length 8 (=Option(s) length 4) 0x0000: c021 0900 0008 Magic-Num 0x194d41e9 21:42:29.965065 6c:3b:6b:81:xx:xx (oui Unknown) > 90:e2:ba:ae:XX:XX (oui Unknown), ethertype PPPoE S (0x8864), length 60: PPPoE [ses 0x1807] CHAP (0xc223), length 37: CHAP, Response (0x02), id 1, Value a29cc35e5fc2d30d1770f7a6a099f019, Name VeryGoodUsername 21:42:29.965080 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:xx:xx (oui Unknown), ethertype PPPoE S (0x8864), length 30: PPPoE [ses 0x1807] LCP (0xc021), length 10: LCP, Echo-Reply (0x0a), id 0, length 10 encoded length 8 (=Option(s) length 4) 0x0000: c021 0a00 0008 Magic-Num 0x193a92d5 21:42:30.966257 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:xx:xx (oui Unknown), ethertype PPPoE S (0x8864), length 47: PPPoE [ses 0x1807] CHAP (0xc223), length 27: CHAP, Fail (0x04), id 1, Msg Authentication failed 21:42:30.966267 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:xx:xx (oui Unknown), ethertype PPPoE S (0x8864), length 26: PPPoE [ses 0x1807] LCP (0xc021), length 6: LCP, Term-Request (0x05), id 109, length 6 encoded length 4 (=Option(s) length 0) 21:42:31.004958 6c:3b:6b:81:xx:xx (oui Unknown) > 90:e2:ba:ae:XX:XX (oui Unknown), ethertype PPPoE S (0x8864), length 66: PPPoE [ses 0x1807] LCP (0xc021), length 46: LCP, Term-Request (0x05), id 2, length 46 encoded length 44 (=Option(s) length 40) 0x0000: c021 0502 002c 21:42:31.004961 6c:3b:6b:81:xx:xx (oui Unknown) > 90:e2:ba:ae:XX:XX (oui Unknown), ethertype PPPoE S (0x8864), length 60: PPPoE [ses 0x1807] LCP (0xc021), length 6: LCP, Term-Ack (0x06), id 109, length 6 encoded length 4 (=Option(s) length 0) 21:42:31.004977 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:xx:xx (oui Unknown), ethertype PPPoE S (0x8864), length 26: PPPoE [ses 0x1807] LCP (0xc021), length 6: LCP, Term-Ack (0x06), id 2, length 6 encoded length 4 (=Option(s) length 0) ``` Most important message here is: ``` 21:42:30.966257 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:xx:xx (oui Unknown), ethertype PPPoE S (0x8864), length 47: PPPoE [ses 0x1807] CHAP (0xc223), length 27: CHAP, Fail (0x04), id 1, Msg Authentication failed ``` As you can see, CHAP authentication failed. This is because of wrong password, MAC mismatch, or duplicate session (check billing logs). ### Troubleshooting examples Verify if PPPoE handshake is working: The only fact is important is to trace sequence for specific user MAC if he sends correct PADI and receives correct PADO, and then sends PADR. For example: ``` tcpdump -ni bond0.2529 -vvv -e pppoed and ether host 6c:3b:6b:81:XX:XX 21:35:47.131362 6c:3b:6b:81:XX:XX (oui Unknown) > Broadcast, ethertype PPPoE D (0x8863), length 60: PPPoE PADI [Host-Uniq 0x500FF5C517B8000005020600] [Service-Name "pr"] 21:35:47.131408 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:XX:XX (oui Unknown), ethertype PPPoE D (0x8863), length 80: PPPoE PADO [AC-Name "united"] [Service-Name "pr"] [AC-Cookie 0x6F71CC054D13DCE911CC1341F1FBDCE7A16830B204F2BEF7] [Host-Uniq 0x500FF5C517B8000005020600] 21:35:47.143621 6c:3b:6b:81:XX:XX (oui Unknown) > 90:e2:ba:ae:XX:XX (oui Unknown), ethertype PPPoE D (0x8863), length 80: PPPoE PADR [Host-Uniq 0x500FF5C517B8000005020600] [AC-Cookie 0x6F71CC054D13DCE911CC1341F1FBDCE7A16830B204F2BEF7] [AC-Name "united"] [Service-Name "pr"] 21:35:47.143936 90:e2:ba:ae:XX:XX (oui Unknown) > 6c:3b:6b:81:XX:XX (oui Unknown), ethertype PPPoE D (0x8863), length 52: PPPoE PADS [ses 0x8a48] [AC-Name "united"] [Service-Name "pr"] [Host-Uniq 0x500FF5C517B8000005020600] ``` Here we see complete handshake, PADI, PADO, PADR, PADS. If you see this sequence - PPPoE handshake is working. ### Troubleshooting pppoe packets by tcpdump Monitor what PPPoE answer to client PADI: ``` tcpdump -nei bond0.372|grep PADO ``` Monitor traffic from specific mac ``` tcpdump -nei bond0.372 ether host 00:00:00:00:00:00 ``` ### Decoding values in accel-cmd pppoe interface show ``` interface: connections: state: service-name policy: ----------------------------------------------------------- bond0.2622 0 active 3 bond0.2529 2251 active 3 bond0.368 530 active 3 bond0.923 0 active 1 ``` * service-name policy is bitmask value - 1 means service-blank, 3 means service-blank+service-name-by-list,2 means service-name-by-list (only) /* Answer requests with empty (0 size) service name */ #define PPPOE_FLAG_ANSWER_BLANK (1 << 0) /* Answer to assigned service names */ #define PPPOE_FLAG_ANSWER_LIST (1 << 1) /* Answer to ANY service name */ #define PPPOE_FLAG_ANSWER_ANY (1 << 2) /* Answer to global service name list */ #define PPPOE_FLAG_ANSWER_GLOBAL (1 << 3)