forked from spinesystemspublic/documentation
Compare commits
76 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 nuclearcat
|
5a8aa4bb73 | ||
 Denys Fedoryshchenko
|
5052e0da70 | ||
|
Denys Fedoryshchenko
|
a250493255 | ||
|
nuclearcat
|
09064b7726 | ||
|
nuclearcat
|
668d44e673 | ||
|
nuclearcat
|
491edb0e68 | ||
|
nuclearcat
|
71ae6c8705 | ||
|
nuclearcat
|
c02b852f1f | ||
|
nuclearcat
|
cad757a8e2 | ||
|
nuclearcat
|
fd19b3bff3 | ||
|
nuclearcat
|
66f8632dde | ||
|
nuclearcat
|
e0c6f435fd | ||
|
nuclearcat
|
9d7b434374 | ||
|
nuclearcat
|
3c8c651572 | ||
|
nuclearcat
|
1679a3670d | ||
|
nuclearcat
|
1816bac83c | ||
|
nuclearcat
|
0c5303cc84 | ||
|
nuclearcat
|
9ed1c6ed90 | ||
|
nuclearcat
|
018ae870c9 | ||
|
nuclearcat
|
06c10f1e24 | ||
|
nuclearcat
|
55e2691635 | ||
|
nuclearcat
|
f48cd5be1e | ||
|
nuclearcat
|
d6f943f654 | ||
|
nuclearcat
|
d1204eab55 | ||
|
Denys Fedoryshchenko
|
09061e21ab | ||
|
nuclearcat
|
3b24dc9cfc | ||
|
nuclearcat
|
405a517466 | ||
|
Denys Fedoryshchenko
|
0bf2177669 | ||
|
Denys Fedoryshchenko
|
d522e1b82e | ||
|
Denys Fedoryshchenko
|
782eb80d85 | ||
|
Denys Fedoryshchenko
|
0625d7fda3 | ||
|
Denys Fedoryshchenko
|
e97c1b288b | ||
|
Denys Fedoryshchenko
|
86f86d8928 | ||
|
nuclearcat
|
75498d06c0 | ||
|
Denys Fedoryshchenko
|
d0a9340661 | ||
|
Denys Fedoryshchenko
|
2b40b78166 | ||
|
Denys Fedoryshchenko
|
14a22b5a54 | ||
|
Denys Fedoryshchenko
|
55e17dbf69 | ||
|
nuclearcat
|
327e5cd68a | ||
|
Denys Fedoryshchenko
|
fe0a55f97f | ||
|
nuclearcat
|
fcd42048f0 | ||
|
Denys Fedoryshchenko
|
8aab6de65b | ||
|
Denys Fedoryshchenko
|
416d41b623 | ||
|
Denys Fedoryshchenko
|
d6855c239e | ||
|
Denys Fedoryshchenko
|
17d44e3180 | ||
|
Denys Fedoryshchenko
|
cf2a34d69c | ||
|
Denys Fedoryshchenko
|
1978691400 | ||
|
Denys Fedoryshchenko
|
4f28e54dc2 | ||
|
nuclearcat
|
eda02fefd7 | ||
|
nuclearcat
|
50254593bb | ||
|
nuclearcat
|
58d6671ac4 | ||
|
Denys Fedoryshchenko
|
3729c2296b | ||
|
Denys Fedoryshchenko
|
fecb3b07a0 | ||
|
Denys Fedoryshchenko
|
7dea0dacd2 | ||
|
Denys Fedoryshchenko
|
78c1e6af60 | ||
|
Denys Fedoryshchenko
|
733baa90ae | ||
|
Denys Fedoryshchenko
|
1a337879e7 | ||
|
Denys Fedoryshchenko
|
bdc4ee573d | ||
|
Denys Fedoryshchenko
|
da4662e34d | ||
|
Denys Fedoryshchenko
|
58b67277c2 | ||
|
Denys Fedoryshchenko
|
c34f6ec7df | ||
|
Denys Fedoryshchenko
|
3560a73c92 | ||
|
Denys Fedoryshchenko
|
ec72a4178d | ||
|
Denys Fedoryshchenko
|
70da2d0e44 | ||
|
Denys Fedoryshchenko
|
8fc13c9b91 | ||
|
Denys Fedoryshchenko
|
3a64bfdfff | ||
|
Denys Fedoryshchenko
|
de020be72a | ||
|
Denys Fedoryshchenko
|
ca493005bb | ||
|
Denys Fedoryshchenko
|
c899d1aa22 | ||
|
Denys Fedoryshchenko
|
599ebd3c86 | ||
|
Denys Fedoryshchenko
|
73961dd660 | ||
|
nuclearcat
|
d976d2cc0b | ||
|
Denys Fedoryshchenko
|
02da072046 | ||
|
Denys Fedoryshchenko
|
3a39f87630 | ||
|
Denys Fedoryshchenko
|
afd8053c55 | ||
|
nuclearcat
|
d850079db1 |
@@ -1,3 +1,45 @@
|
||||
# GlobalOS API
|
||||
|
||||
## Setting up
|
||||
|
||||
To enable API, you need to add following file /etc/api.conf
|
||||
|
||||
```ini
|
||||
# set API key
|
||||
apikey=ZZZZZ
|
||||
# disable API key check
|
||||
# disableapikey=1
|
||||
|
||||
allowip=1.2.3.4,2.4.5.6
|
||||
```
|
||||
|
||||
API key should be present in header X-Auth-Token: ZZZZZ
|
||||
|
||||
## Hooks
|
||||
|
||||
### API hook: /hooks/counters
|
||||
|
||||
Available counters for a given interface
|
||||
GET parameters:
|
||||
iface - interface name
|
||||
source - source of data (tc, ip, ethtool)
|
||||
type - type of data (for source tc: qdisc, class, filter; for source ip: link, addr; for source ethtool: statistics, module)
|
||||
|
||||
Example:
|
||||
/hook/counters?iface=ppp100&source=ip&type=link
|
||||
|
||||
### API hook: /ppp/list
|
||||
|
||||
Optional GET params:
|
||||
&tag=XXXX - filter by radattr tag
|
||||
&tag=XXXX&value=YYYY - filter by radattr tag and value (full match)
|
||||
X-Auth-Token: XXXX
|
||||
Login-LAT-Service gaming
|
||||
curl -H "X-Auth-Token: XXXX" http://localhost:8080/ppp/list?tag=Login-LAT-Service&value=gaming
|
||||
|
||||
|
||||
|
||||
|
||||
Work in progress...
|
||||
```
|
||||
http.HandleFunc("/hooks/counters", counters_hook)
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
# Gaming service
|
||||
|
||||
## Configure pppoes
|
||||
|
||||
/etc/api.conf
|
||||
|
||||
```sh
|
||||
apikey=SecretTokenRandom
|
||||
allowip=10.168.100.
|
||||
```
|
||||
|
||||
allowip must be set to ip of your nat
|
||||
|
||||
## Configure radius
|
||||
|
||||
Configure for gaming service radius attribute "Login-LAT-Service" with value "gaming"
|
||||
|
||||
## Configure nat
|
||||
|
||||
* Run several instances of pppoe gaming pollers that will retrieve list of gaming users
|
||||
|
||||
```sh
|
||||
gamingclient -u http://pppoeip:apiport/ -k SecretTokenRandom
|
||||
```
|
||||
|
||||
apiport by default 9000, might be different in some cases
|
||||
|
||||
* Configure ipset for gaming users
|
||||
|
||||
```sh
|
||||
ipset create gamingclients hash:net timeout 120
|
||||
```
|
||||
|
||||
Also add relevant rules to nat users to different pool, for example:
|
||||
|
||||
```sh
|
||||
iptables -t nat -A POSTROUTING -m set --match-set gamingclients src -j SNAT --to-source $GAMING --persistent
|
||||
```
|
||||
|
||||
where $GAMING is ip of your gaming pool (1.2.3.1-1.2.3.10 for example)
|
||||
|
||||
### How it works?
|
||||
|
||||
* User login and get gaming service attribute from radius
|
||||
* API daemon will detect such user and put in special list available over API HTTP GET request
|
||||
* Gaming poller will retrieve list of users and put them in ipset each X seconds
|
||||
* NAT will detect users in ipset and will put them in gaming NAT pool by evaluating SNAT rule to gaming
|
||||
* If user will be disconnected, gaming poller will not readd user, and it will be removed from ipset after 120 seconds by ipset timeout
|
||||
+218
@@ -0,0 +1,218 @@
|
||||
# Networking
|
||||
|
||||
## Bonding
|
||||
|
||||
Traffic is distributed between interfaces using LACP protocol. It is recommended to use LACP on switch side too.
|
||||
It is also recommended to enable layer2+3 hashing for better traffic distribution.
|
||||
|
||||
```
|
||||
ip link set bond0 type bond xmit_hash_policy layer3+4
|
||||
```
|
||||
|
||||
|
||||
## collectd
|
||||
|
||||
### Forwarding collectd metrics to another collectd instance
|
||||
|
||||
You need to add on destination collectd instance:
|
||||
|
||||
```
|
||||
LoadPlugin network
|
||||
|
||||
<Plugin network>
|
||||
<Listen "LISTENIP" "25826">
|
||||
SecurityLevel Encrypt
|
||||
AuthFile "/etc/collectd/passwd"
|
||||
</Listen>
|
||||
</Plugin>
|
||||
```
|
||||
In /etc/colllectd/passwd you need to add user and password for source collectd instance:
|
||||
|
||||
```
|
||||
user:password
|
||||
user2:password2
|
||||
```
|
||||
|
||||
Source collectd instance:
|
||||
|
||||
```
|
||||
LoadPlugin network
|
||||
|
||||
<Plugin network>
|
||||
<Server "DESTINATIONIP" "25826">
|
||||
SecurityLevel Encrypt
|
||||
Username "user"
|
||||
Password "password"
|
||||
</Server>
|
||||
</Plugin>
|
||||
```
|
||||
|
||||
## Selecting right CPU, memory, disk, network card for your GlobalOS
|
||||
|
||||
### CPU
|
||||
Find CPU available on cpubenchmark.net website and check "Average CPU mark".
|
||||
|
||||
For PPPoE: You can estimate at 7000 points it might take about 4000 users. More points - more users, but growth is not linear.
|
||||
For example "Intel Xeon E3-1230 v3 @ 3.30GHz" has 6795 points.
|
||||
|
||||
For NAT: You can estimate at 7000 points it might take about 10Gbps.
|
||||
|
||||
Note, if you buy dual CPU server, you will not get 2x performance, it will be still significant boost, and you need to take into consideration, that each PCI-E is "local" for one CPU only, so you might need to install 2 network cards, on each CPU "side". Read about NUMA architecture and how it works.
|
||||
|
||||
### Memory
|
||||
For PPPoE it is recommended to have at least 8GB of RAM at least. Recommended to have 16-32GB, but not more. You might take into consideration some servers with too much memory might have slower memory speed.
|
||||
Do not use DDR3 memory/servers, it is too slow, unless you are ok with lower than expected number of users per server.
|
||||
|
||||
### Disk
|
||||
For our system it is totally fine to use USB drive, but internal SSD is preferable. System use very little of storage resources, not I/O bound, we know some USB flashes that lasted more than 6 years under 24/7 operation. Always keep nearby USB storage for backup/restore purposes.
|
||||
|
||||
### Network card
|
||||
We highly recommend Intel network cards. Several generations exist:
|
||||
|
||||
1) Well tested, but a bit old - X520, 82599 chipset (10Gbps), might have 1 or 2 SFP+ ports. Might accept any SFP+ modules.
|
||||
2) X7** series - 10Gbps/40Gbps, might have 1 or 2 SFP+ ports. Need to check compatibility with SFP+ modules, need Intel branded or modified firmware modules. Very high performance, but not tested extensively for PPPoE task.
|
||||
3) X9** series - 10Gbps/40Gbps, might have 1 or 2 SFP+ ports. Need to check compatibility with SFP+ modules, need Intel branded or modified firmware modules. Not tested yet.
|
||||
|
||||
Not recommended: Broadcom, Chelsio cards, they are known to start packetloss way before 10Gbps.
|
||||
|
||||
## Boot options
|
||||
|
||||
### EFI vs BIOS
|
||||
We support both EFI and BIOS boot modes. But we highly recommend to use EFI boot mode, as BIOS boot known to fail from time to time. Please disable "Secure boot" in BIOS settings.
|
||||
|
||||
## BIOS settings
|
||||
We recommend to do following tuning before deploying server to production:
|
||||
|
||||
* Disable "Secure boot" in BIOS settings.
|
||||
* Change CPU power(powersaving) mode to "OS control" if available, and to "Performance" if not.
|
||||
* TBD, i need to check BIOS of servers i have :)
|
||||
|
||||
## Other sources
|
||||
|
||||
* https://github.com/leandromoreira/linux-network-performance-parameters
|
||||
|
||||
|
||||
# IPMI
|
||||
|
||||
## IPMI setup
|
||||
|
||||
- Verify if your server has IPMI.
|
||||
|
||||
```
|
||||
modprobe ipmi_devintf
|
||||
modprobe ipmi_si
|
||||
```
|
||||
If second command fails, you might not have IPMI.
|
||||
Check if `ipmitool` is installed, if not, install it.
|
||||
Then verify if it works:
|
||||
```
|
||||
ipmitool sdr
|
||||
```
|
||||
|
||||
### IPMI network setup
|
||||
|
||||
First verify current settings:
|
||||
```
|
||||
ipmitool lan print
|
||||
```
|
||||
|
||||
Assign IP to IPMI interface:
|
||||
```
|
||||
ipmitool lan set 1 ipsrc static 1.2.3.4/24
|
||||
ipmitool lan set 1 defgw ipaddr 1.2.3.1
|
||||
ipmitool lan set 1 access on
|
||||
```
|
||||
|
||||
Set username and password. First you need to find channel with users.
|
||||
Try:
|
||||
```
|
||||
ipmitool user list
|
||||
ipmitool user list 1
|
||||
ipmitool user list 2
|
||||
etc to 8
|
||||
```
|
||||
When it will show you list of users, you can set username and password for this channel:
|
||||
|
||||
```
|
||||
ipmitool user set name 1 admin
|
||||
ipmitool user set password 1 password
|
||||
```
|
||||
TODO: Some boards doesnt allow to change username, and some have limits on password length.
|
||||
|
||||
|
||||
|
||||
# Monitoring more than basic metrics
|
||||
|
||||
## Temperatures
|
||||
|
||||
### IPMI
|
||||
|
||||
If IPMI present, you can use ipmitool to get temperatures:
|
||||
```
|
||||
ipmitool sensors
|
||||
```
|
||||
|
||||
If you don't have ipmitool please contact to obtain latest version of GlobalOS.
|
||||
|
||||
### lm-sensors
|
||||
|
||||
```
|
||||
modprobe i2c-dev
|
||||
modprobe coretemp
|
||||
modprobe k10temp
|
||||
modprobe pinctrl_lewisburg
|
||||
```
|
||||
|
||||
In `/sys/class/hwmon/` you will find entries for your sensors, search for one with name "coretemp" or "k10temp". for example
|
||||
|
||||
```
|
||||
grep "" /sys/class/hwmon/hwmon*/name
|
||||
/sys/class/hwmon/hwmon1/name:coretemp
|
||||
/sys/class/hwmon/hwmon2/name:pch_lewisburg
|
||||
|
||||
grep "" /sys/class/hwmon/hwmon1/temp*
|
||||
/sys/class/hwmon/hwmon1/temp10_crit:98000
|
||||
/sys/class/hwmon/hwmon1/temp10_crit_alarm:0
|
||||
/sys/class/hwmon/hwmon1/temp10_input:30000
|
||||
/sys/class/hwmon/hwmon1/temp10_label:Core 8
|
||||
/sys/class/hwmon/hwmon1/temp10_max:88000
|
||||
/sys/class/hwmon/hwmon1/temp11_crit:98000
|
||||
....
|
||||
|
||||
grep "" /sys/class/hwmon/hwmon2/temp*
|
||||
/sys/class/hwmon/hwmon2/device/temp:37000
|
||||
|
||||
```
|
||||
|
||||
`pch_lewisburg` is chipset temperature
|
||||
|
||||
|
||||
### Power consumption monitoring
|
||||
|
||||
- Power meter (ACPI interface)
|
||||
```
|
||||
Check in /sys/class/hwmon/ entries where device/name contains "power" or "energy".
|
||||
```
|
||||
|
||||
- Power meter (IPMI)
|
||||
```
|
||||
ipmitool sensor
|
||||
```
|
||||
|
||||
### Disk temperature
|
||||
|
||||
Note: Smartctl is available in latest GlobalOS versions.
|
||||
|
||||
```
|
||||
smartctl -a /dev/sda | grep Temperature
|
||||
```
|
||||
|
||||
## SFP/SFP+ modules
|
||||
|
||||
```
|
||||
ethtool -m ethX
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
+929
@@ -0,0 +1,929 @@
|
||||
# Juniper switch tips and tricks
|
||||
|
||||
## Initial setup
|
||||
|
||||
* Set hostname, root password, timezone, NTP server, SNMP community, etc
|
||||
|
||||
```
|
||||
set system host-name YOURSWITCHNAME
|
||||
set system root-authentication plain-text-password
|
||||
set system name-server x.x.x.x
|
||||
```
|
||||
|
||||
* Configure management interface or vlan.0 to have internet access
|
||||
|
||||
```
|
||||
set interfaces vlan unit 0 family inet address x.x.x.x/24
|
||||
set routing-options static route 0.0.0.0/0 next-hop x.x.x.y
|
||||
```
|
||||
|
||||
* Check if internet present
|
||||
|
||||
```
|
||||
ping www.google.com
|
||||
```
|
||||
|
||||
|
||||
* Download and upgrade to latest junos (Denys can assist to obtain latest versions)
|
||||
|
||||
```
|
||||
copy file https://url/jinstall...tgz /var/tmp/
|
||||
request system software add /var/tmp/jinstall...tgz
|
||||
request system reboot
|
||||
```
|
||||
|
||||
* Set NTP server
|
||||
|
||||
```
|
||||
set system ntp server x.x.x.x
|
||||
set system time-zone Asia/Beirut
|
||||
```
|
||||
|
||||
* Enable SSH access
|
||||
|
||||
```
|
||||
set system services ssh
|
||||
```
|
||||
|
||||
|
||||
## Setting options
|
||||
|
||||
ssh to the switch, you will see the prompt
|
||||
|
||||
```
|
||||
username@YOURSWITCHNAME>
|
||||
```
|
||||
To set certain options you need to go to configuration mode
|
||||
|
||||
```
|
||||
username@YOURSWITCHNAME> edit
|
||||
Entering configuration mode
|
||||
|
||||
[edit]
|
||||
username@YOURSWITCHNAME#
|
||||
```
|
||||
You can see prompt changed, now you can set options.
|
||||
|
||||
```
|
||||
[edit]
|
||||
username@YOURSWITCHNAME# set system host-name YOURSWITCHNAME
|
||||
[edit]
|
||||
username@YOURSWITCHNAME# set system root-authentication plain-text-password
|
||||
New password:
|
||||
Retype new password:
|
||||
|
||||
```
|
||||
etc etc
|
||||
|
||||
After you finished adding changes, you can verify them by issuing command
|
||||
|
||||
```
|
||||
[edit]
|
||||
username@YOURSWITCHNAME# show | compare
|
||||
```
|
||||
|
||||
It will show you the difference between current configuration and the one you are going to commit.
|
||||
|
||||
If you are happy with the changes, you can commit them in "confirmed" mode, similar to Mikrotik "safe mode"
|
||||
|
||||
```
|
||||
[edit]
|
||||
username@YOURSWITCHNAME# commit confirmed 5
|
||||
```
|
||||
This will commit the changes and will rollback them in 5 minutes if you don't confirm them. So if you made a mistake and you are locked out of the switch, you can still wait 5 minutes and it will rollback to previous configuration.
|
||||
But if everything is fine, you can confirm the changes by issuing command
|
||||
|
||||
```
|
||||
[edit]
|
||||
username@YOURSWITCHNAME# commit
|
||||
commit complete
|
||||
```
|
||||
You can directly go to commit step if you are absolutely sure what you are doing, and changes wont lock you out of the switch.
|
||||
|
||||
## Config backup
|
||||
|
||||
You can backup config by copying file /config/juniper.conf.gz using SCP (WinSCP for example).
|
||||
|
||||
## Interface names
|
||||
|
||||
In juniper interface names depends on port-type. For example xe-0/0/0 means 10G port, ge-0/0/0 means 1G, et-0/0/0 means 40G. So for example if you have configuration:
|
||||
|
||||
```
|
||||
set interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members CUSTOMER100
|
||||
set interfaces ge-0/0/0 unit 0 family ethernet-switching port mode trunk
|
||||
set interfaces xe-0/0/0 unit 0 family ethernet-switching vlan members CUSTOMER101
|
||||
set interfaces xe-0/0/0 unit 0 family ethernet-switching port mode trunk
|
||||
```
|
||||
If you plug 1G SFP in port 0/0/0 you will have vlan CUSTOMER100 activated, but if you plug 10G SFP+ - you will have vlan CUSTOMER101 activated.
|
||||
|
||||
## Professional style
|
||||
|
||||
It is professional to set on each interface description, so you can easily identify the interface and keep things in order. This includes also description of the VLANs. (irb.* or vlan.*)
|
||||
```
|
||||
set interfaces xe-0/0/0 description RESELLER-ABDOTALEB-PPPoE
|
||||
set interfaces vlan.1011 description RESELLER-ABDOTALEB-PPPoE-1
|
||||
set interfaces vlan.1012 description RESELLER-ABDOTALEB-PPPoE-2
|
||||
```
|
||||
You can shorten names your own way.
|
||||
|
||||
## Users management
|
||||
|
||||
*WARNING! Seems new junos doesn't allow to ssh user root*
|
||||
|
||||
### Changing root password
|
||||
|
||||
```
|
||||
set system root-authentication plain-text-password
|
||||
```
|
||||
NEVER set easy passwords, you might lose your device as soon as it is connected to internet.
|
||||
|
||||
### Adding new user
|
||||
|
||||
```
|
||||
set system login user USERNAME class super-user authentication plain-text-password
|
||||
```
|
||||
NEVER set easy passwords, you might lose your device as soon as it is connected to internet.
|
||||
|
||||
### Adding new user with SSH key
|
||||
|
||||
```
|
||||
set system login user USERNAME class super-user authentication ssh-rsa "USERKEY"
|
||||
```
|
||||
|
||||
## Firewall
|
||||
|
||||
### Set firewall for management interface
|
||||
|
||||
```
|
||||
set interface lo0 unit 0 family inet filter input MANAGEMENT
|
||||
set firewall family inet filter MANAGEMENT term MANAGEMENT from source-address x.x.x.x/32
|
||||
set firewall family inet filter MANAGEMENT term MANAGEMENT then accept
|
||||
set firewall family inet filter MANAGEMENT term default then discard
|
||||
```
|
||||
|
||||
Make sure to use commit confirmed, otherwise you might lock yourself out of the switch.
|
||||
You might need to add also BGP of Ogero IP addresses to the firewall, sometimes allow ICMP protocol for troubleshooting.
|
||||
*THIS IS IMPORTANT STEP TO SECURE THE SWITCH*
|
||||
|
||||
|
||||
## Monitoring and misc
|
||||
### Add SNMP community for monitoring
|
||||
|
||||
```
|
||||
set snmp community COMMUNITY authorization read-only
|
||||
```
|
||||
Please make complex name for COMMUNITY, as it can be guessed or bruteforced.
|
||||
|
||||
### Reduce load due frequent SNMP polling
|
||||
|
||||
(Not tested well yet)
|
||||
```
|
||||
filter-interfaces {
|
||||
interfaces {
|
||||
ipip;
|
||||
gre;
|
||||
jsrv;
|
||||
mtun;
|
||||
lsi;
|
||||
pimd;
|
||||
pime;
|
||||
tap;
|
||||
fti0;
|
||||
dsc;
|
||||
vme;
|
||||
"(a|t|x|g)e-[0-9]\/[0-9]+\/[0-9]+\.0$";
|
||||
gr-0/0/0;
|
||||
"irb$";
|
||||
}
|
||||
all-internal-interfaces;
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
### Add NTP server
|
||||
|
||||
```
|
||||
set system ntp server x.x.x.x
|
||||
set system time-zone Asia/Beirut
|
||||
```
|
||||
It is highly recommended to have NTP server, as it will help you to troubleshoot problems in the future, as logs will have correct time.
|
||||
|
||||
### Filtering license flood messages
|
||||
|
||||
```
|
||||
system {
|
||||
syslog {
|
||||
file messages {
|
||||
any notice;
|
||||
authorization info;
|
||||
match "!(.*BGP Routing Protocol usage requires a license.*)";
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
This will prevent early wearout of your flash memory, as license messages are written to the log every 10 seconds.
|
||||
|
||||
### syslog audit
|
||||
|
||||
It is recommended to send syslog to remote host, in case of incidents, offender cannot remove them:
|
||||
```
|
||||
system {
|
||||
syslog {
|
||||
host x.x.x.x {
|
||||
any notice;
|
||||
authorization info;
|
||||
match "!(.*BGP Routing Protocol usage requires a license.*)";
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Deactivate parts of config
|
||||
|
||||
You can "deactivate" parts of config, so it will be removed from config, but it will be still present in the system, so you can activate it back without reconfiguring it.
|
||||
|
||||
```
|
||||
deactivate interfaces xe-0/0/0
|
||||
or
|
||||
deactivate protocols bgp group OGERO
|
||||
or
|
||||
deactivate vlans CUSTOMER100
|
||||
```
|
||||
|
||||
to activate back:
|
||||
|
||||
```
|
||||
activate interfaces xe-0/0/0
|
||||
or
|
||||
activate protocols bgp group OGERO
|
||||
or
|
||||
activate vlans CUSTOMER100
|
||||
```
|
||||
|
||||
## Tuning
|
||||
|
||||
After installing switch it is recommended to do certain tuning (which i often forget) to avoid problems in the future.
|
||||
|
||||
### EX4500, EX4550
|
||||
|
||||
```
|
||||
chassis {
|
||||
fpc 0 {
|
||||
pic 0 {
|
||||
q-pic-large-buffer;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
Set the large buffer on the EX-series switches. (Doesn't work on all model), might help to avoid packetloss at high utilization.
|
||||
|
||||
```
|
||||
class-of-service {
|
||||
shared-buffer {
|
||||
percent 100;
|
||||
}
|
||||
}
|
||||
```
|
||||
Not sure this tuning useful, to be researched.
|
||||
|
||||
|
||||
|
||||
### All models
|
||||
|
||||
```
|
||||
interfaces {
|
||||
xe-0/0/0 {
|
||||
ether-options {
|
||||
no-flow-control;
|
||||
}
|
||||
```
|
||||
Disable flow control on all interfaces. Flow control in ISP environment is not recommended, it can cause packetloss and various network issues.
|
||||
|
||||
## VLANs
|
||||
|
||||
### Create L2 VLAN between 2 interfaces
|
||||
|
||||
Create VLAN name with associated VLAN ID
|
||||
Then set on interfaces the VLAN name and the port mode to trunk
|
||||
EX4600, QFX: interface-mode
|
||||
|
||||
```
|
||||
set vlans CUSTOMER100 vlan-id 100
|
||||
set interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members CUSTOMER100
|
||||
set interfaces ge-0/0/0 unit 0 family ethernet-switching port mode trunk
|
||||
set interfaces ge-0/0/1 unit 0 family ethernet-switching vlan members CUSTOMER100
|
||||
set interfaces ge-0/0/1 unit 0 family ethernet-switching port mode trunk
|
||||
|
||||
QFX, EX4600
|
||||
set vlans CUSTOMER100 vlan-id 100
|
||||
set interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members CUSTOMER100
|
||||
set interfaces ge-0/0/0 unit 0 family ethernet-switching interface-mode trunk
|
||||
set interfaces ge-0/0/1 unit 0 family ethernet-switching vlan members CUSTOMER100
|
||||
set interfaces ge-0/0/1 unit 0 family ethernet-switching interface-mode trunk
|
||||
```
|
||||
As you might notice, you can also set port mode to access, this means port will send traffic of vlan "untagged" (without vlan tag). If interface set to access, you cannot have more than one "member" VLAN.
|
||||
|
||||
### Create L3 VLAN
|
||||
|
||||
If you want to create a L3 VLAN, you need to create a VLAN name with id and set l3-interface, then set the interface with the IP address,
|
||||
and assign the VLAN to some interface.
|
||||
|
||||
Note, name of the VLAN interface is vlan.id on EX4500, EX4550, smaller series, and irb.id on EX4600, EX8200, EX9200, QFX Series. I am giving example for EX4500
|
||||
```
|
||||
set vlans CUSTOMER100 vlan-id 100
|
||||
set vlans CUSTOMER100 l3-interface vlan.100
|
||||
set interfaces vlan unit 100 family inet address 192.168.1.1/24
|
||||
set interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members CUSTOMER100
|
||||
set interfaces ge-0/0/0 unit 0 family ethernet-switching port mode access
|
||||
|
||||
QFX:
|
||||
set vlans CUSTOMER100 vlan-id 100
|
||||
set vlans CUSTOMER100 l3-interface vlan.100
|
||||
set interfaces irb unit 100 family inet address 192.168.1.1/24
|
||||
set interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members CUSTOMER100
|
||||
set interfaces ge-0/0/0 unit 0 family ethernet-switching interface-mode access
|
||||
```
|
||||
|
||||
### Create tagged with native VLAN
|
||||
|
||||
If you want to create a tagged VLAN with one VLAN passing as untagged, native VLAN, you need to set the following:
|
||||
|
||||
EX4500, EX4550
|
||||
```
|
||||
xe-0/0/1 {
|
||||
unit 0 {
|
||||
family ethernet-switching {
|
||||
port-mode trunk;
|
||||
vlan {
|
||||
members [ VLANTRUNK1 VLANTRUNK2 VLANTRUNK3 VLANSOMETHING ];
|
||||
}
|
||||
native-vlan-id OTHERVLAN;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
QFX, EX4600, EX8200, EX9200
|
||||
```
|
||||
et-0/0/50 {
|
||||
native-vlan-id 603;
|
||||
unit 0 {
|
||||
family ethernet-switching {
|
||||
interface-mode trunk;
|
||||
vlan {
|
||||
members [ VLAN1 VLANOTHER VLANTHIRD ];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
Attention to *native-vlan-id*. You might need to add native vlan also to members. (please check if it is mandatory and send PR to update docs)
|
||||
|
||||
## Interface
|
||||
|
||||
### Shutdown interface
|
||||
|
||||
```
|
||||
root@Juniper# set interfaces ge-0/0/1.0 disable << This is cisco equivalent of “shutdown”
|
||||
root@Juniper# delete interfaces ge-0/0/1.0 disable << This is cisco equivalent of “no shutdown”
|
||||
```
|
||||
|
||||
### Deactivate interface
|
||||
|
||||
Juniper have something unique, you can deactivate interface, which means it will be removed from configuration, but it will be still present in the system, so you can activate it back without reconfiguring it.
|
||||
|
||||
Deactivate is a Junos feature that deactivates configuration snippets and should NOT be confused with disabling or shutting down interfaces.
|
||||
|
||||
Deactivating a configuration on an interface will remove the effect of the configuration without removing it. However, it will keep the interface in UP state, which can impact routing and other functions for L2 and L3 protocols to detect the changes.
|
||||
|
||||
```
|
||||
deactivate interfaces xe-0/0/0
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Routing
|
||||
|
||||
### Static routing
|
||||
|
||||
```
|
||||
routing-options {
|
||||
static {
|
||||
route x.x.x.x/y next-hop z.z.z.z;
|
||||
}
|
||||
}
|
||||
```
|
||||
Add static route to x.x.x.x/y via z.z.z.z
|
||||
|
||||
### BGP
|
||||
|
||||
Typical setup with Ogero, you will have 2 BGP sessions, one for IPv4 and one for IPv6. You will receive default route from Ogero, and you will send your prefixes to Ogero.
|
||||
We will skip IPv6 for now.
|
||||
|
||||
```
|
||||
protocols {
|
||||
bgp {
|
||||
group OGERO {
|
||||
type external;
|
||||
local-address x.x.x.x;
|
||||
family inet {
|
||||
unicast;
|
||||
}
|
||||
neighbor y.y.y.y {
|
||||
peer-as 42020;
|
||||
authentication-key "$9$xxxxxxxxxxxxxxxxxxxxx";
|
||||
}
|
||||
export [ OGERO-EXPORT ];
|
||||
import [ OGERO-IMPORT ];
|
||||
}
|
||||
}
|
||||
}
|
||||
policy-options {
|
||||
policy-statement OGERO-EXPORT {
|
||||
term OGERO-EXPORT {
|
||||
from {
|
||||
route-filter a.a.a.a/24 exact;
|
||||
route-filter b.b.b.b/24 exact;
|
||||
}
|
||||
then accept;
|
||||
}
|
||||
term default {
|
||||
then reject;
|
||||
}
|
||||
}
|
||||
policy-statement OGERO-IMPORT {
|
||||
term OGERO-IMPORT {
|
||||
then accept;
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
```
|
||||
*IMPORTANT*: term default then reject will prevent route leaks, dont forget it.
|
||||
|
||||
* x.x.x.x is your peer IP address and y.y.y.y is Ogero IP address.
|
||||
* a.a.a.a/24 and b.b.b.b/24 are your prefixes you want to send to Ogero.
|
||||
|
||||
Also dont forget to set your ASN in routing-options
|
||||
|
||||
```
|
||||
set routing-options autonomous-system 12345
|
||||
```
|
||||
|
||||
You can reset BGP session by following command:
|
||||
|
||||
```
|
||||
clear bgp neighbor y.y.y.y
|
||||
```
|
||||
|
||||
Also you can temporary "remove" BGP configuration by deactivating it:
|
||||
|
||||
```
|
||||
deactivate protocols bgp group OGERO
|
||||
```
|
||||
|
||||
To activate back:
|
||||
|
||||
```
|
||||
activate protocols bgp group OGERO
|
||||
```
|
||||
|
||||
## Bonding
|
||||
|
||||
Don't forget to set number of bonding interfaces
|
||||
```
|
||||
chassis {
|
||||
aggregated-devices {
|
||||
ethernet {
|
||||
device-count 15;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then bonding itself:
|
||||
|
||||
```
|
||||
xe-0/0/6 {
|
||||
description IXMANAGER-eth3;
|
||||
ether-options {
|
||||
802.3ad ae6;
|
||||
}
|
||||
}
|
||||
xe-0/0/7 {
|
||||
description IXMANAGER-eth4;
|
||||
ether-options {
|
||||
802.3ad ae6;
|
||||
}
|
||||
}
|
||||
ae6 {
|
||||
description IXMANAGER;
|
||||
aggregated-ether-options {
|
||||
lacp {
|
||||
active;
|
||||
}
|
||||
}
|
||||
unit 0 {
|
||||
family ethernet-switching {
|
||||
port-mode access;
|
||||
vlan {
|
||||
members [ IXMANAGER ];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
## FBF
|
||||
|
||||
### EX4500, EX4550
|
||||
```
|
||||
routing-instances {
|
||||
route-tonat {
|
||||
instance-type forwarding;
|
||||
routing-options {
|
||||
static {
|
||||
route 0.0.0.0/0 next-hop 10.0.252.2;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
routing-options {
|
||||
interface-routes {
|
||||
rib-group inet fbf-group;
|
||||
}
|
||||
rib-groups {
|
||||
fbf-group {
|
||||
import-rib [ inet.0 route-tonat.inet.0 ];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
firewall {
|
||||
family inet {
|
||||
filter mainvlan-in {
|
||||
term bypass {
|
||||
from {
|
||||
destination-address {
|
||||
10.0.0.0/8;
|
||||
192.168.0.0/16;
|
||||
91.240.80.0/22;
|
||||
172.16.0.0/12;
|
||||
}
|
||||
}
|
||||
then accept;
|
||||
}
|
||||
term tonat {
|
||||
from {
|
||||
source-address {
|
||||
10.0.0.0/8;
|
||||
192.168.0.0/16;
|
||||
172.16.0.0/12;
|
||||
}
|
||||
}
|
||||
then {
|
||||
routing-instance route-tonat;
|
||||
}
|
||||
}
|
||||
term default {
|
||||
then accept;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
interfaces {
|
||||
vlan {
|
||||
unit 100 {
|
||||
family inet {
|
||||
filter {
|
||||
input mainvlan-in;
|
||||
}
|
||||
address x.x.x.x/24;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
### EX4600, QFX
|
||||
|
||||
Fundamental difference between EX4500, EX4550 and EX4600, QFX is that EX4500, EX4550 have "instance-type forwarding" and EX4600, QFX don't have it. So you need to use "instance-type virtual-router" instead.
|
||||
|
||||
Here is examples for EX4600, QFX, where we redirect traffic from users to nat, and have local bypass.
|
||||
|
||||
|
||||
One of them verified and i tested it works:
|
||||
|
||||
```
|
||||
routing-instances {
|
||||
route-tonat {
|
||||
instance-type virtual-router;
|
||||
routing-options {
|
||||
static {
|
||||
route 0.0.0.0/0 next-hop 10.0.250.2;
|
||||
}
|
||||
}
|
||||
interface irb.161;
|
||||
}
|
||||
}
|
||||
interfaces {
|
||||
irb {
|
||||
unit 161 {
|
||||
description VRF-TO-NAT;
|
||||
family inet {
|
||||
address 10.0.250.1/24;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
firewall {
|
||||
family inet {
|
||||
filter mainvlan-in {
|
||||
term bypass {
|
||||
from {
|
||||
destination-address {
|
||||
10.0.0.0/8;
|
||||
192.168.0.0/16;
|
||||
172.16.0.0/12;
|
||||
}
|
||||
}
|
||||
then accept;
|
||||
}
|
||||
term tonat {
|
||||
from {
|
||||
source-address {
|
||||
172.16.0.0/12;
|
||||
}
|
||||
}
|
||||
then {
|
||||
routing-instance route-tonat;
|
||||
}
|
||||
}
|
||||
term default {
|
||||
then accept;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
Note: interface irb.161 can be used only one-way, to send traffic to NAT, but not to receive it back. Receive back will be handled automatically, as NAT will decode back to fake ips and fake ips routes are present in default routing instance, so nothing special need to be done.
|
||||
Also after that you need to assign filter to interface where traffic from users is coming.
|
||||
|
||||
|
||||
Second one, i adapted it from from article, it should work and looks more elegant (doesnt need to lose interface for VRF)
|
||||
|
||||
```
|
||||
FBF-test {
|
||||
instance-type virtual-router;
|
||||
routing-options {
|
||||
static {
|
||||
route 0.0.0.0/0 next-hop 12.12.12.2;
|
||||
}
|
||||
instance-import FBF-export;
|
||||
}
|
||||
}
|
||||
policy-statement FBF-export {
|
||||
term 1 {
|
||||
from {
|
||||
instance master;
|
||||
route-filter 12.12.12.0/30 exact;
|
||||
}
|
||||
then accept;
|
||||
}
|
||||
term 2 {
|
||||
then reject;
|
||||
}
|
||||
}
|
||||
firewall {
|
||||
family inet {
|
||||
filter FBF-test {
|
||||
term 1 {
|
||||
from {
|
||||
destination-address {
|
||||
10.0.0.0/8;
|
||||
192.168.0.0/16;
|
||||
172.16.0.0/12;
|
||||
}
|
||||
}
|
||||
then accept;
|
||||
}
|
||||
term tonat {
|
||||
from {
|
||||
source-address {
|
||||
172.16.0.0/12;
|
||||
}
|
||||
}
|
||||
then {
|
||||
count FBF-count;
|
||||
routing-instance FBF-test;
|
||||
}
|
||||
}
|
||||
term 2 {
|
||||
then accept;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
Articles:
|
||||
|
||||
- https://supportportal.juniper.net/s/article/EX-QFX-Performing-filter-based-forwarding-in-ELS-devices?language=en_US
|
||||
|
||||
## Diagnostics
|
||||
|
||||
### SFP signal monitoring
|
||||
|
||||
If your SFP support DDMI, you can monitor the signal strength of the SFP. This is useful to see if the SFP is working properly or not.
|
||||
|
||||
```
|
||||
admin@CORE> show interfaces diagnostics optics xe-0/0/0
|
||||
Physical interface: xe-0/0/0
|
||||
Laser bias current : 8.370 mA
|
||||
Laser output power : 0.6260 mW / -2.03 dBm
|
||||
Module temperature : 40 degrees C / 104 degrees F
|
||||
Module voltage : 3.2510 V
|
||||
Receiver signal average optical power : 0.4357 mW / -3.61 dBm
|
||||
Laser bias current high alarm : Off
|
||||
Laser bias current low alarm : Off
|
||||
Laser bias current high warning : Off
|
||||
Laser bias current low warning : Off
|
||||
Laser output power high alarm : Off
|
||||
Laser output power low alarm : Off
|
||||
Laser output power high warning : Off
|
||||
Laser output power low warning : Off
|
||||
Module temperature high alarm : Off
|
||||
Module temperature low alarm : Off
|
||||
Module temperature high warning : Off
|
||||
Module temperature low warning : Off
|
||||
Module voltage high alarm : Off
|
||||
Module voltage low alarm : Off
|
||||
Module voltage high warning : Off
|
||||
Module voltage low warning : Off
|
||||
Laser rx power high alarm : Off
|
||||
Laser rx power low alarm : Off
|
||||
Laser rx power high warning : Off
|
||||
Laser rx power low warning : Off
|
||||
Laser bias current high alarm threshold : 100.000 mA
|
||||
Laser bias current low alarm threshold : 0.000 mA
|
||||
Laser bias current high warning threshold : 90.000 mA
|
||||
Laser bias current low warning threshold : 0.100 mA
|
||||
Laser output power high alarm threshold : 1.2580 mW / 1.00 dBm
|
||||
Laser output power low alarm threshold : 0.1990 mW / -7.01 dBm
|
||||
Laser output power high warning threshold : 1.0000 mW / 0.00 dBm
|
||||
Laser output power low warning threshold : 0.2510 mW / -6.00 dBm
|
||||
Module temperature high alarm threshold : 90 degrees C / 194 degrees F
|
||||
Module temperature low alarm threshold : -5 degrees C / 23 degrees F
|
||||
Module temperature high warning threshold : 85 degrees C / 185 degrees F
|
||||
Module temperature low warning threshold : 0 degrees C / 32 degrees F
|
||||
Module voltage high alarm threshold : 3.800 V
|
||||
Module voltage low alarm threshold : 2.700 V
|
||||
Module voltage high warning threshold : 3.700 V
|
||||
Module voltage low warning threshold : 2.800 V
|
||||
Laser rx power high alarm threshold : 1.0000 mW / 0.00 dBm
|
||||
Laser rx power low alarm threshold : 0.0501 mW / -13.00 dBm
|
||||
Laser rx power high warning threshold : 0.7943 mW / -1.00 dBm
|
||||
Laser rx power low warning threshold : 0.0631 mW / -12.00 dBm
|
||||
```
|
||||
Which means:
|
||||
|
||||
First quickly go thru list of alarms and warnings, if you see any of them, you have a problem with SFP or fiber.
|
||||
|
||||
Second check values, to have estimation.
|
||||
|
||||
* Receiver signal average optical power : 0.4357 mW / -3.61 dBm
|
||||
|
||||
Means the signal is good, and the SFP is working properly.
|
||||
|
||||
Usually signal should vary between " Laser rx power high warning threshold : 0.7943 mW / -1.00 dBm" and "Laser rx power low warning threshold : 0.0631 mW / -12.00 dBm". Which means if it is below -12dBm or above -1dBm, you have a problem. Too low signal usually means either problem with SFP degradation or fiber attenuation(damaged?). Too high signal usually means too much light is coming to the receiver, which can be caused by too high power SFP TX on other side + too short fiber for such type of SFP, this might cause receiver burnout.
|
||||
|
||||
You might also watch out for "borderline" values, which are close to the threshold, this might indicate that you have a problem, but it is not critical yet.
|
||||
|
||||
* Laser bias current : 8.370 mA
|
||||
|
||||
Similar, you need to be within bounds set by "bias current".
|
||||
|
||||
Watch out also for SFP temperatures, if it is too high, it might indicate that SFP is overheating, and likely will degrade soon.
|
||||
|
||||
### Check inventory
|
||||
|
||||
Usually it is used to verify list of plugged in SFPs, but it also shows other hardware.
|
||||
|
||||
```
|
||||
admin@CORE> show chassis hardware
|
||||
Hardware inventory:
|
||||
Item Version Part number Serial number Description
|
||||
Chassis ZZZZ EX4500-40F
|
||||
Routing Engine 0 REV 08 750-035702 ZZZZ EX4500-40F
|
||||
FPC 0 REV 08 750-035702 ZZZZ EX4500-40F
|
||||
CPU BUILTIN BUILTIN FPC CPU
|
||||
PIC 0 BUILTIN BUILTIN 40x 1/10GE
|
||||
Xcvr 0 REV 01 740-021308 I0512280053 SFP+-10G-SR
|
||||
Xcvr 1 NON-JNPR CI151116245 SFP-T
|
||||
Xcvr 2 REV 01 740-031980 F172JU01652 SFP+-10G-SR
|
||||
Xcvr 3 REV 01 740-031980 F172JU01651 SFP+-10G-SR
|
||||
Xcvr 4 REV 01 740-021308 I0512280054 SFP+-10G-SR
|
||||
Xcvr 5 REV 01 740-031981 F172JU02038 SFP+-10G-LR
|
||||
|
||||
```
|
||||
|
||||
### New SFP not working
|
||||
|
||||
This might happen on Junipers, while they work with most of SFP, some might be not compatible.
|
||||
|
||||
If you plugged in new SFP and it is not working, you can check if it is recognized by the switch. Check by following commands:
|
||||
|
||||
```
|
||||
show chassis hardware
|
||||
show interfaces diagnostics optics **-0/0/0
|
||||
show interface **-0/0/0 extensive
|
||||
|
||||
** should be replaced by interface name, for example xe-0/0/0, but you need to assume sometimes 1G, 10G not detected correctly, so you might need to check all possible types, like ge and xe.
|
||||
```
|
||||
|
||||
|
||||
For last command attention on lines like this:
|
||||
|
||||
```
|
||||
Physical interface: xe-0/0/0, Enabled, Physical link is Up
|
||||
|
||||
Link-level type: Ethernet, MTU: 1514, LAN-PHY mode, Speed: 10Gbps, Duplex: Full-Duplex, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Media type: Fiber
|
||||
```
|
||||
If physical link is Down it might also means SFP is non-standard, not detected correctly and not working.
|
||||
Check also if Speed, Media type match specification.
|
||||
|
||||
### Packetloss
|
||||
|
||||
If you suspect packetloss on some link, juniper have nice "flood" ping commands.
|
||||
```
|
||||
admin@CORE> ping 10.0.252.2 count 1000 rapid
|
||||
PING 10.0.252.2 (10.0.252.2): 56 data bytes
|
||||
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
||||
--- 10.0.252.2 ping statistics ---
|
||||
1000 packets transmitted, 1000 packets received, 0% packet loss
|
||||
round-trip min/avg/max/stddev = 0.572/2.286/13.992/2.207 ms
|
||||
```
|
||||
This will send 1000 packets as fast as possible, and will show you packetloss and RTT. While you can ignore RTT, as control plane dont have high priority, packetloss should not be present in any case.
|
||||
|
||||
### BGP troubleshooting
|
||||
|
||||
If you have BGP session with Ogero, and it is not working, you can check if you have BGP session established by following command:
|
||||
|
||||
```
|
||||
admin@CORE> show bgp summary
|
||||
Groups: 1 Peers: 1 Down peers: 0
|
||||
Table Tot Paths Act Paths Suppressed History Damp State Pending
|
||||
inet.0
|
||||
1 1 0 0 0 0
|
||||
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
|
||||
y.y.y.y 42020 0 0 0 0 1:01 1/1/1/0 0/0/0/0
|
||||
```
|
||||
If you see "State" column, it should be "Established", if it is not, you have a problem with BGP session.
|
||||
|
||||
You can check if you have BGP routes sent by following command:
|
||||
|
||||
```
|
||||
admin@CORE> show route advertising-protocol bgp y.y.y.y
|
||||
inet.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
|
||||
Prefix Nexthop MED Lclpref AS path
|
||||
* x.y.a.x/22 Self I
|
||||
* b.b.b.b/22 Self I
|
||||
```
|
||||
If you see your prefixes, it means you are sending them to Ogero.
|
||||
|
||||
You might use also "show bgp neighbor x.x.x.x" to see more detailed information about BGP session.
|
||||
|
||||
### MAC address table size
|
||||
|
||||
Keep an eye on EX4500 and EX4550 and don't exceed ~6000 MAC addresses, as it might cause problems with switch. This switch have 16k MAC addresses, but it have hash collision problems, so it is recommended to keep it below 6k.
|
||||
If you have more than 6000 MAC addresses, you might need to upgrade to better switch.
|
||||
There is a trick when you can disable MAC learning on some interfaces, but you have to meet one condition - only TWO interfaces should have this VLAN enabled, otherwise you will have unicast flood problem.
|
||||
Learning on VLAN can be disabled by following command:
|
||||
|
||||
```
|
||||
set vlans VLANNAME no-mac-learning
|
||||
```
|
||||
|
||||
Verifying MAC address table size:
|
||||
|
||||
```
|
||||
admin@CORE> show ethernet-switching table brief
|
||||
```
|
||||
Attention on header: "Ethernet-switching table: 477 entries, 239 learned, 0 persistent entries"
|
||||
|
||||
### Monitoring routing engine CPU
|
||||
|
||||
```
|
||||
admin@CORE> show chassis routing-engine
|
||||
```
|
||||
It is recommended to not have load average values more than 1.0 - 2.0.
|
||||
|
||||
### Searching particular user mac on ports
|
||||
|
||||
```
|
||||
admin@CORE> show ethernet-switching table | match MACADDRESS
|
||||
```
|
||||
MACADDRESS should be replaced by MAC address of the user, for example 00:11:22:33:44:55
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# Copying/cloning PPPoE
|
||||
|
||||
## Copy GlobalOS/Spinix
|
||||
|
||||
* Issue dmesg command, you can clean current log by using `dmesg -c`
|
||||
* Insert new flash into USB port
|
||||
* Issue dmesg command again, you should see new device, for example /dev/sdb
|
||||
* MAKE SURE THIS DEVICE IS NEW DEVICE! If you specify wrong one you might lose all data on it
|
||||
* Use gs install to install GlobalOS on the new storage
|
||||
|
||||
## Configuring networking
|
||||
|
||||
* TBD
|
||||
|
||||
## Copying PPPoE related software components
|
||||
|
||||
You might need to use scp to copy files from one device to another. (WinSCP for Windows, but keep attention on file permissions)
|
||||
|
||||
### Basic PPPoE components
|
||||
|
||||
* accel-ppp daemon itself is included in base image, so you don't need to copy it
|
||||
* copy /etc/accel-ppp.conf and adjust according to your needs, we will cover in detail later in another chapter
|
||||
* /etc/add_accel_vlans.sh - script to add VLANs to accel-ppp
|
||||
* Copy certain settings/commands from /etc/rc.local to your new rc.local
|
||||
* /etc/firewall.cfg - firewall configuration, copy it if you need it and dont forget to adjust it
|
||||
* /etc/config.json - shaper config
|
||||
* /usr/local/bin/burst2d - shaper daemon
|
||||
|
||||
### network.cfg adjustments
|
||||
|
||||
* sysctl -w net.ipv4.conf.eth4/999.proxy_arp=1
|
||||
Enable proxy-arp if you have real IP addresses on PPPoE shared between multiple pppoe, enable it on "WAN" interface of PPPoE
|
||||
|
||||
### shaper daemon startup
|
||||
|
||||
* Create directory for example /etc/service-burst2d
|
||||
* Create file /etc/service-burst2d/run with following content:
|
||||
```
|
||||
#!/bin/sh
|
||||
/usr/local/bin/burst2d
|
||||
```
|
||||
* Make it executable
|
||||
* add to /etc/rc.local
|
||||
```
|
||||
runsv /etc/service-burst2d &
|
||||
```
|
||||
|
||||
### accel-ppp.conf
|
||||
|
||||
* Allocate new subnet and route on your core router
|
||||
```
|
||||
[ip-pool]
|
||||
gw-ip-address=10.255.255.1
|
||||
attr=Framed-Pool
|
||||
X.X.X.X/MASK
|
||||
```
|
||||
* Update dae-server (CoA) entry with new PPPoE IP
|
||||
```
|
||||
dae-server=Z.Z.Z.Z:1700,NASSECRETSOMETHING
|
||||
```
|
||||
* Make sure radius entry is correct
|
||||
```
|
||||
server=X.X.X.X....
|
||||
```
|
||||
|
||||
### add_accel_vlans.sh
|
||||
|
||||
In old versions of script you might have fixed interface name, make sure it match interface where pppoe vlans are connected. For example eth4:
|
||||
```
|
||||
ip link add link eth4 name eth4.$VLANID type vlan id $VLANID
|
||||
```
|
||||
Change if necessary over all script
|
||||
|
||||
### /etc/hosts
|
||||
|
||||
Make sure hosts have correct entry for localhost
|
||||
```
|
||||
pppoe /etc # cat /etc/hosts
|
||||
127.0.0.1 localhost
|
||||
|
||||
```
|
||||
|
||||
### Preferable tuning for PPPoE
|
||||
|
||||
* Disable TSO, GRO, GSO for better shaper precision on ethernet and bonding interfaces
|
||||
```
|
||||
ethtool -K eth4 tso off gro off gso off
|
||||
...
|
||||
```
|
||||
* Make sure gc_thresh* tuned for PPPoE
|
||||
```
|
||||
echo 2560 > /proc/sys/net/ipv4/neigh/default/gc_thresh1
|
||||
echo 5120 > /proc/sys/net/ipv4/neigh/default/gc_thresh2
|
||||
echo 10240 > /proc/sys/net/ipv4/neigh/default/gc_thresh3
|
||||
```
|
||||
* conntrack max entries
|
||||
```
|
||||
sysctl -w net.nf_conntrack_max=2000000
|
||||
```
|
||||
+96
-3
@@ -1,23 +1,41 @@
|
||||
# PPPoE management guide
|
||||
|
||||
## Config system
|
||||
|
||||
* *IMPORTANT* To save configuration like in cisco you need to issue command "gs save" (Cisco equivalent - write mem)
|
||||
* Config directory /etc/ is saved in /mnt/flash/config.tgz or /mnt/flash/secure2/config.tgz
|
||||
|
||||
## VLAN operations
|
||||
|
||||
### Adding new vlan interface
|
||||
|
||||
You need to add following line in /etc/accel-ppp.conf
|
||||
```
|
||||
interface=bond0.2529,{"services":["zone1","Flexnet","ferrari2","fi","rp","pr","dotnet"],"service-blank":1}
|
||||
interface=bond0.2529,{"services":["zone1","Flexnet","IDM","fi","rp","pr","dotnet"],"service-blank":1}
|
||||
```
|
||||
Possible options:
|
||||
* services - list of PPPoE services to be added to the interface
|
||||
* services - list of PPPoE services to be added to the interface, if you dont want any omit this parameter
|
||||
* service-blank - if set to 1, then user that didnt specify service can connect to this pppoe
|
||||
* service-any - if set to 1, then user that specified any service can connect to this pppoe
|
||||
* service-any - if set to 1, then user that specified any service can connect to this pppoe, if not needed - omit this parameter
|
||||
|
||||
After updating accel-ppp config, you need to run script that will create vlans and add vlans to accel without restart:
|
||||
```
|
||||
sh /etc/add_accel_vlans.sh
|
||||
```
|
||||
|
||||
You can verify vlan is created by using following command:
|
||||
```
|
||||
ip link show bond0.2529
|
||||
```
|
||||
Make sure it is activated in accel-ppp:
|
||||
```
|
||||
accel-cmd pppoe interface show
|
||||
```
|
||||
Also you can check mac addresses on /proc/net/pppoe, and exclude only those you want for specific vlan using grep:
|
||||
```
|
||||
grep bond0.2529 /proc/net/pppoe
|
||||
```
|
||||
|
||||
### Monitoring vlan utilization
|
||||
|
||||
```
|
||||
@@ -316,7 +334,82 @@ Detected PPPoE server at interface bond0.2529
|
||||
```
|
||||
As you can see there are 3 other PPPoE servers in the network, and they are all detected by pppoe-discovery. They might cause conflicts during PPPoE session establishment, so it is recommended to find and remove them.
|
||||
|
||||
## Hardware issues
|
||||
|
||||
### Reduced size PCI slot
|
||||
```
|
||||
[ 5.584995] ixgbe 0000:05:00.0: Multiqueue Enabled: Rx Queue count = 32, Tx Queue count = 32 XDP Queue count = 0
|
||||
[ 5.592552] ixgbe 0000:05:00.0: 16.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x4 link at 0000:00:02.0 (capable of 32.000 Gb/s with 5.0 GT/s PCIe x8 link)
|
||||
[ 5.608961] ixgbe 0000:05:00.0: MAC: 2, PHY: 14, SFP+: 3, PBA No: Unknown
|
||||
[ 5.617719] ixgbe 0000:05:00.0: xx:xx:xx:xx:xx:xx
|
||||
[ 5.632680] ixgbe 0000:05:00.0: Intel(R) 10 Gigabit Network Connection
|
||||
[ 5.792953] ixgbe 0000:05:00.1: Multiqueue Enabled: Rx Queue count = 32, Tx Queue count = 32 XDP Queue count = 0
|
||||
[ 5.802226] ixgbe 0000:05:00.1: 16.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x4 link at 0000:00:02.0 (capable of 32.000 Gb/s with 5.0 GT/s PCIe x8 link)
|
||||
[ 5.821176] ixgbe 0000:05:00.1: MAC: 2, PHY: 14, SFP+: 4, PBA No: Unknown
|
||||
[ 5.830715] ixgbe 0000:05:00.1: xx:xx:xx:xx:xx:xx
|
||||
[ 5.841368] ixgbe 0000:05:00.1: Intel(R) 10 Gigabit Network Connection
|
||||
[ 6.041356] ixgbe 0000:05:00.0: registered PHC device on eth4
|
||||
```
|
||||
If you see message about "PCIe bandwidth, limited" this means your slot is likely have limited lanes and cannot reach full capacity. In case of X520 cards, this means about 15Gbit total bandwidth limit (in+out).
|
||||
Try to change slot or make sure riser have support for all PCIe lanes.
|
||||
|
||||
## Misc tuning
|
||||
|
||||
### Neighbour table overflow
|
||||
|
||||
If you see in dmesg:
|
||||
```
|
||||
[ 803.123253] neighbour: arp_cache: neighbor table overflow!
|
||||
```
|
||||
Apply following tuning:
|
||||
```
|
||||
echo 2560 > /proc/sys/net/ipv4/neigh/default/gc_thresh1
|
||||
echo 5120 > /proc/sys/net/ipv4/neigh/default/gc_thresh2
|
||||
echo 10240 > /proc/sys/net/ipv4/neigh/default/gc_thresh3
|
||||
```
|
||||
|
||||
## Ogero DSL
|
||||
|
||||
To run Ogero DSL L2TP you need to add following to accel-ppp.conf:
|
||||
```
|
||||
[l2tp]
|
||||
verbose=255
|
||||
dictionary=/usr/share/accel-ppp/l2tp/dictionary
|
||||
host-name=TO-OGERO
|
||||
dir300_quirk=1
|
||||
#secret=ISP@mplsDSL
|
||||
dataseq=allow
|
||||
reorder-timeout=0
|
||||
avp_permissive=1
|
||||
```
|
||||
Sure you need to enable l2tp plugin and configure rest of options similar to pppoe.
|
||||
|
||||
## Installing burst2d daemon on new pppoe
|
||||
|
||||
* Unpack [tgz archive](https://git.spinesystems.solutions/spinesystemspublic/binaries/src/branch/master/burst2d/burst2d.tgz)
|
||||
* put burst2d binary to /usr/local/bin
|
||||
* put burst2d.cfg to /etc from previous installation
|
||||
* put libraries to /usr/local/lib
|
||||
* copy /etc/config.json from previous installation
|
||||
|
||||
Create and run service (runsv)
|
||||
```
|
||||
mkdir -p /etc/svc/burst2d
|
||||
echo -ne "#!/bin/sh\n/usr/local/bin/burst2d\n" >/etc/svc/burst2d/run
|
||||
chmod +x /etc/svc/burst2d/run
|
||||
runsv /etc/svc/burst2d &
|
||||
```
|
||||
Add to rc.local last line to start on boot
|
||||
|
||||
* Verify any user, if shapers present by:
|
||||
```
|
||||
tc -s -d class show dev pppX
|
||||
```
|
||||
|
||||
|
||||
### References
|
||||
|
||||
* [TCPDUMP examples ] (https://hackertarget.com/tcpdump-examples/)
|
||||
* [More tcpdump docs] (https://danielmiessler.com/study/tcpdump/)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
# Misc tricks for GlobalOS
|
||||
|
||||
## trafw
|
||||
Trafw allows to see bidirectional traffic statistics.
|
||||
|
||||
Usage: trafw interface1 "interface1 pcap filter" interface2 "interface2 pcap filter" timer
|
||||
|
||||
Example:
|
||||
```
|
||||
trafw ppp0 "inbound" ppp0 "outbound" 1
|
||||
```
|
||||
Will show inbound and outbound traffic on ppp0 every second.
|
||||
|
||||
## iptop
|
||||
|
||||
Usage: iptop interface "pcap filter" packets (dst|src) [p|b]
|
||||
|
||||
Example:
|
||||
```
|
||||
iptop eth0 "inbound" 10000 dst
|
||||
```
|
||||
Show top 20 destination ip address by rate. (top consumers)
|
||||
|
||||
## tcpdump and accel-cmd
|
||||
|
||||
To check user traffic you need to use tcpdump and accel-cmd.
|
||||
|
||||
To find out interface name of user you can use:
|
||||
```
|
||||
accel-cmd show sessions
|
||||
|
||||
pppoe-9 ~ # accel-cmd show sessions|more
|
||||
ifname | username | calling-sid | ip | type | comp | state | uptime
|
||||
---------+-----------------+-------------------+----------------+-------+------+--------+-------------
|
||||
ppp185 | user1 | 6c:3b:6b:73:33:11 | 172.17.16.185 | pppoe | | active | 17.00:56:54
|
||||
ppp305 | user2 | 50:0f:f5:40:22:22 | 172.17.17.49 | pppoe | | active | 17.00:56:53
|
||||
ppp318 | user3 | 6c:3b:6b:c4:11:33 | 172.17.17.62 | pppoe | | active | 17.00:56:53
|
||||
```
|
||||
|
||||
Which means that user1 is connected to ppp185 interface, user2 to ppp305 and user3 to ppp318.
|
||||
To check traffic of user1 you need to use:
|
||||
```
|
||||
tcpdump -ni ppp185 -vvv -c 100
|
||||
```
|
||||
Which means:
|
||||
* -n - do not resolve ip addresses (we don't need it)
|
||||
* -i ppp185 - listen on ppp185 interface
|
||||
* -vvv - verbose output
|
||||
* -c 100 - capture 100 packets and exit
|
||||
|
||||
## pcap filters
|
||||
|
||||
You can use various pcap filters in trafw, iptop and tcpdump.
|
||||
For example if you want to see only traffic from subnet 8.8.8.0/24 and source port 53 you can use:
|
||||
```
|
||||
# to show total traffic for this filter
|
||||
trafw ppp0 "src net 8.8.8.0/24 and src port 53" ppp0 "src net 8.8.8.0/24 and src port 53" 1
|
||||
# to show top consumers of such traffic
|
||||
iptop eth0 "src net 8.8.8.0/24 and src port 53" 10000 dst
|
||||
# to capture such traffic and see each packet, only 100 packets
|
||||
tcpdump -ni ppp185 -vvv -c 100 "src net 8.8.8.0/24 and src port 53"
|
||||
|
||||
More info, in articles:
|
||||
https://iphelix.medium.com/packet-filtering-techniques-84fc3fc2ea3b
|
||||
|
||||
## restarting accel-pppd
|
||||
|
||||
If you need to restart accel-pppd you can use:
|
||||
```
|
||||
killall accel-pppd
|
||||
```
|
||||
then wait until process is killed, it might take up to few minutes on large pppoe.
|
||||
To check if accel-pppd is running you can use:
|
||||
```
|
||||
ps aux|grep accel-pppd
|
||||
```
|
||||
If you see accel-pppd process you can start it with:
|
||||
```
|
||||
/usr/sbin/accel-pppd -c /etc/accel-ppp.conf -p /var/run/accel-ppp.pid -d
|
||||
```
|
||||
|
||||
## Shell tricks
|
||||
|
||||
For example i want to filter all lines that begin with "fadi" in users list:
|
||||
```
|
||||
accel-cmd show sessions username|grep "^ fadi"
|
||||
```
|
||||
What does it mean and how it works?
|
||||
|
||||
* accel-cmd show sessions username - will show all sessions, but username field will be shown only
|
||||
* | - pipe, it will pass output of previous command to next command
|
||||
* grep "^ fadi" - will filter only lines that begin with " fadi", symbol ^ means beginning of line, space is needed to filter accel-specific output (it has space before username)
|
||||
|
||||
Now, for example i want to save this list and disconnect all users that begin with "fadi":
|
||||
```
|
||||
accel-cmd show sessions username|grep "^ fadi"|awk '{print $1}'|xargs -I {} accel-cmd terminate username {}
|
||||
```
|
||||
|
||||
What does it mean and how it works?
|
||||
|
||||
* awk '{print $1}' - will print first field of each line, which is username, it will remove leading space
|
||||
* xargs -I {} - will pass each line to next command, {} is placeholder for line(variable)
|
||||
* accel-cmd terminate username {} - will disconnect user by username
|
||||
|
||||
## Real IPs routing to multiple PPPoE servers
|
||||
|
||||
In situation when you have more than one PPPoE server and you want to route real IPs to them you can use proxy ARP trick.
|
||||
How it works?
|
||||
|
||||
1. You route real IPs to your NAT or separate server (anything that can do direct route to interface), for example real subnet a.b.c.0/24
|
||||
2. On nat server you have interface facing PPPoE servers, for example bond0.1234, with ip 10.100.100.1/24. where pppoe servers are 10.100.100.10/24, 10.100.100.11/24 10.100.100.12/24. You add route:
|
||||
```
|
||||
ip route add a.b.c.0/24 dev bond0.1234
|
||||
```
|
||||
3. On pppoe servers you have interfaces facing pppoe server, for example eth4.999 10.100.100.10/24 on first one. You need to execute following commands:
|
||||
```
|
||||
sysctl -w net.ipv4.conf.eth4/999.proxy_arp=1
|
||||
|
||||
or if interface eth0:
|
||||
sysctl -w net.ipv4.conf.eth0.proxy_arp=1
|
||||
```
|
||||
|
||||
How this does work?
|
||||
- When packet comes to your router, it is routed to NAT (or separate server) because of route.
|
||||
- NAT server sees that packet is for a.b.c.0/24 and sends ARP request "WHERE IS a.b.c.1?" to bond0.1234 because it has route to a.b.c.0/24 over bond0.1234 directly.
|
||||
- Each PPPoE server due proxy_arp entry will check, do i have a.b.c.1? If yes, it will respond with its own MAC address.
|
||||
- NAT server will send packet to PPPoE server with a.b.c.1 and PPPoE server will process it.
|
||||
|
||||
This way you can route real IPs to multiple PPPoE servers without need of BGP or other routing protocols. It is simple and works well, but have one caveat, if old entry exist and user changed pppoe, it might need timeout (max 5min) so ARP entry expires.
|
||||
+250
@@ -195,3 +195,253 @@ If you execute command several times, take attention on counters, for example if
|
||||
traffic is being "buffered", so user use all speed for this class.<br>
|
||||
If counter in dropped is not zero, it means that user is sending more traffic than allowed by class and packets are dropped.<br>
|
||||
And sure you can match rate and ceil, which match cir and burst values.<br>
|
||||
|
||||
# burst2d /etc/burst2d.cfg
|
||||
|
||||
This is burst daemon system settings file.
|
||||
Available options:
|
||||
```
|
||||
[global]
|
||||
static_config=/etc/config.json
|
||||
#tc=/usr/local/sbin/tc
|
||||
htb_extra=burst 4000000 cburst 4000000
|
||||
upload_shaper=0
|
||||
filter_swap=0
|
||||
```
|
||||
- static_config - path to config.json, where shaper definitions are done.
|
||||
- tc - optional, used in case we need to use different version of tc
|
||||
- htb_extra - extra options for HTB, for example to set microburst values to reduce CPU load
|
||||
- upload_shaper if set to 0 - upload shaper is disabled, 1 - enabled
|
||||
- filter_swap - swap how Radius speed attributes identified. Values 0 or 1. If you see download speed incorrectly set as upload (and upload as download), change variable to opposite
|
||||
|
||||
# burst2d config.json options
|
||||
|
||||
## Example config.json
|
||||
```json
|
||||
{
|
||||
"defaultservice": "#default",
|
||||
"traffic_classes": {
|
||||
"fna" : { "classid": 10, "parent": 1, "qdisc": "pie limit 100 target 10ms" },
|
||||
"ggc" : { "classid": 11, "parent": 1, "qdisc": "pfifo limit 100" },
|
||||
"bypass" : { "classid": 12, "parent": 0, "qdisc": "pie limit 100 target 10ms" },
|
||||
"default" : { "classid": 2, "parent": 1, "qdisc": "pie limit 100 target 100ms" }
|
||||
},
|
||||
"traffic_filters": {
|
||||
"fna" : { "prio": 10, "classid": 10, "criteria": "u32 match ip src 185.22.34.0/24" },
|
||||
"ggc" : { "prio": 10, "classid": 11, "criteria": "u32 match ip src 91.240.80.224/27" },
|
||||
"ping" : { "prio": 5, "classid": 12, "criteria": "u32 match ip protocol 1 0xff" },
|
||||
"dns" : { "prio": 5, "classid": 12, "criteria": "u32 match ip protocol 17 0xff match ip sport 53 0xffff" },
|
||||
"proxy" : { "prio": 6, "classid": 12, "criteria": "handle 1 fw" },
|
||||
"default" : { "prio": 200, "classid": 2, "criteria": "u32 match u32 0 0" }
|
||||
},
|
||||
"classifiers": {
|
||||
"fuped": { "speed": "<512", "setservice": "#fuped", "decision": "final" },
|
||||
"bigvip": { "speed": ">100000", "tag":"vip", "setservice": "#plainflat" },
|
||||
"big": { "speed": ">100000", "setservice": "#plainflat", "decision": "final" },
|
||||
"eveningshapedfna": { "timerange":"17-24", "setservice": "#default" },
|
||||
"noshapedfna": { "timerange":"0-17", "setservice": "#defaultnofnashape" }
|
||||
},
|
||||
"services": {
|
||||
"#default" : {
|
||||
"activate_classes": ["fna", "ggc", "bypass", "default"],
|
||||
"cir_classes": ["%25", "10000", "10000", "%1"],
|
||||
"burst_classes": ["%50", "10000", "10000", "%100"],
|
||||
"modifiers_classes": ["", "pfifo limit 1000", "", ""],
|
||||
"activate_filters": ["fna", "ggc", "ping", "dns", "proxy", "default"]
|
||||
},
|
||||
"#defaultnofnashape" : {
|
||||
"activate_classes": ["ggc", "bypass", "default"],
|
||||
"cir_classes": ["10000", "10000", "%1"],
|
||||
"burst_classes": ["10000", "10000", "%100"],
|
||||
"modifiers_classes": ["pfifo limit 1000", "", ""],
|
||||
"activate_filters": ["ggc", "ping", "dns", "proxy", "default"]
|
||||
},
|
||||
"#fuped" : {
|
||||
"activate_classes": ["bypass", "ggc", "default"],
|
||||
"cir_classes": ["2000","1000", "%1"],
|
||||
"burst_classes": ["2000","1000", "%100"],
|
||||
"modifiers_classes": ["", "", ""],
|
||||
"activate_filters": ["ping", "proxy", "dns", "ggc", "default"]
|
||||
},
|
||||
"#plainflat" : {
|
||||
"activate_classes": ["default"],
|
||||
"cir_classes": ["%100"],
|
||||
"burst_classes": ["%100"],
|
||||
"modifiers_classes": [""],
|
||||
"activate_filters": ["default"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Radius attributes
|
||||
* Login-LAT-Service tag1,tag2,tag3 - set tags for user
|
||||
* Login-LAT-Node bypass:N1/N2/N3... - set bypass speed for user, for example bypass:1000/3000/6000 means bypass0 is 1000, bypass1 is 3000, bypass2 is 6000, which is set as #0, #1, #2 in cir_classes and burst_classes
|
||||
|
||||
## Config.json structure
|
||||
|
||||
|
||||
* defaultservice - default service to use if no classifier matched
|
||||
* traffic_classes - classes to use in traffic shaper
|
||||
* classid - classid to use, this classid will be used in filters, so when filter match it will route traffic to this class
|
||||
* parent - parent classid, this means burst will be limited by parent class, or it can be set as 0 if this is root class and no burst expected (cir should be equal to burst)
|
||||
* qdisc - qdisc to use, this is queue discipline, it can be pfifo, pie, fq_codel, etc.
|
||||
* traffic_filters - filters to match traffic and decide which class will handle it
|
||||
* prio - priority of filter, lower number means higher priority
|
||||
* classid - classid to route traffic if filter matched
|
||||
* criteria - criteria to match, it can be u32 match, handle, etc.
|
||||
* classifiers - classifiers to match user and set service. WARNING: if you have complex classifiers, you should set them in order of priority using prio field. Prio evaluated from 0 to last classifier, so if you have classifier with prio 0, it will be evaluated first, then 1, etc.
|
||||
* speed - speed to match, it can be <, >, =, <=, >=, <>, etc.
|
||||
* setservice - service to set if classifier matched
|
||||
* decision - decision to make if classifier matched, it can be final, continue, etc.
|
||||
* timerange - timerange to match, it can be 0-24, 17-24, etc, so you can set different service for different time range
|
||||
* tag VALUE - check if tag is present (Login-LAT-Service tag1,tag2,tag3)
|
||||
* notag VALUE - check if tag is not present (Login-LAT-Service tag1,tag2,tag3)
|
||||
* isreal 0/1 - check if realip is 0 or not
|
||||
* isfup 0/1 - check if bypass_speed[0] is equal to speed
|
||||
* brasname VALUE - check if brasname is equal to VALUE (set in /etc/burst2d.conf brasname=xxx, default value is "default")
|
||||
* services - services to use in traffic shaper
|
||||
* activate_classes - classes to activate in this service
|
||||
* cir_classes - cir values for classes, it can be %XXX, #N, NNNN, where %XXX is percentage of account speed, #N is bypass N, NNNN is fixed speed. For example 10000 is 10Mbit, %200 is 2x account speed, etc.
|
||||
* burst_classes - burst values for classes, it can be %XXX, #N, NNNN
|
||||
* modifiers_classes - modifiers for classes, it can be "", "pfifo limit 1000", etc.
|
||||
* activate_filters - filters to activate in this service
|
||||
|
||||
## Example of changing config.json
|
||||
|
||||
Let's say we want to introduce new service for users where all CDN bypass(fna and ggc) is grouped together and set to 10Mbit, and we want to set it to users with tag "onebypass".
|
||||
New config will look as following:
|
||||
```json
|
||||
{
|
||||
"defaultservice": "#default",
|
||||
"traffic_classes": {
|
||||
"fna" : { "classid": 10, "parent": 1, "qdisc": "pie limit 100 target 10ms" },
|
||||
"ggc" : { "classid": 11, "parent": 1, "qdisc": "pfifo limit 100" },
|
||||
"bypass" : { "classid": 12, "parent": 0, "qdisc": "pie limit 100 target 10ms" },
|
||||
"default" : { "classid": 2, "parent": 1, "qdisc": "pie limit 100 target 100ms" }
|
||||
},
|
||||
"traffic_filters": {
|
||||
"fna" : { "prio": 10, "classid": 10, "criteria": "u32 match ip src 185.22.34.0/24" },
|
||||
"ggc" : { "prio": 10, "classid": 11, "criteria": "u32 match ip src 91.240.80.224/27" },
|
||||
"ping" : { "prio": 5, "classid": 12, "criteria": "u32 match ip protocol 1 0xff" },
|
||||
"dns" : { "prio": 5, "classid": 12, "criteria": "u32 match ip protocol 17 0xff match ip sport 53 0xffff" },
|
||||
"proxy" : { "prio": 6, "classid": 12, "criteria": "handle 1 fw" },
|
||||
"fnaonebypass" : { "prio": 10, "classid": 12, "criteria": "u32 match ip src 185.22.34.0/24" },
|
||||
"ggconebypass" : { "prio": 10, "classid": 12, "criteria": "u32 match ip src 91.240.80.224/27" },
|
||||
"default" : { "prio": 200, "classid": 2, "criteria": "u32 match u32 0 0" }
|
||||
},
|
||||
"classifiers": {
|
||||
"fuped": { "speed": "<512", "setservice": "#fuped", "decision": "final", "prio": 0 },
|
||||
"onebypass": { "tag":"onebypass", "setservice": "#onebypass", "decision": "final", "prio": 1 },
|
||||
"bigvip": { "speed": ">100000", "tag":"vip", "setservice": "#plainflat", "prio": 2 },
|
||||
"big": { "speed": ">100000", "setservice": "#plainflat", "decision": "final", "prio": 3 },
|
||||
"eveningshapedfna": { "timerange":"17-24", "setservice": "#default", "prio": 4 },
|
||||
"noshapedfna": { "timerange":"0-17", "setservice": "#defaultnofnashape", "prio": 5 },
|
||||
},
|
||||
"services": {
|
||||
"#default" : {
|
||||
"activate_classes": ["fna", "ggc", "bypass", "default"],
|
||||
"cir_classes": ["%25", "10000", "10000", "%1"],
|
||||
"burst_classes": ["%50", "10000", "10000", "%100"],
|
||||
"modifiers_classes": ["", "pfifo limit 1000", "", ""],
|
||||
"activate_filters": ["fna", "ggc", "ping", "dns", "proxy", "default"]
|
||||
},
|
||||
"#defaultnofnashape" : {
|
||||
"activate_classes": ["ggc", "bypass", "default"],
|
||||
"cir_classes": ["10000", "10000", "%1"],
|
||||
"burst_classes": ["10000", "10000", "%100"],
|
||||
"modifiers_classes": ["pfifo limit 1000", "", ""],
|
||||
"activate_filters": ["ggc", "ping", "dns", "proxy", "default"]
|
||||
},
|
||||
"#fuped" : {
|
||||
"activate_classes": ["bypass", "ggc", "default"],
|
||||
"cir_classes": ["2000","1000", "%1"],
|
||||
"burst_classes": ["2000","1000", "%100"],
|
||||
"modifiers_classes": ["", "", ""],
|
||||
"activate_filters": ["ping", "proxy", "dns", "ggc", "default"]
|
||||
},
|
||||
"#plainflat" : {
|
||||
"activate_classes": ["default"],
|
||||
"cir_classes": ["%100"],
|
||||
"burst_classes": ["%100"],
|
||||
"modifiers_classes": [""],
|
||||
"activate_filters": ["default"]
|
||||
},
|
||||
"#onebypass" : {
|
||||
"activate_classes": ["bypass", "default"],
|
||||
"cir_classes": ["10000", "%100"],
|
||||
"burst_classes": ["10000", "%100"],
|
||||
"modifiers_classes": ["", ""],
|
||||
"activate_filters": ["fnaonebypass", "ggconebypass", "default"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
So what are changed?
|
||||
* Added new filters fnaonebypass and ggconebypass to match fna and ggc traffic to class 12, which is bypass class
|
||||
* Added new classifier onebypass to match users with tag "onebypass" and set service to #onebypass
|
||||
* Added new service #onebypass to activate only 2 classes, bypass and default, and set speed of bypass to 10Mbit, and "default" (which is direct traffic) to 100% of account speed
|
||||
|
||||
## How this rules evaluated when shaper find such user?
|
||||
* First it will read radius attributes and find all tags
|
||||
* Then it will start evaluating classifiers in order of prio. First one is fuped, it will check if user speed is less than 512, if yes, it will set service to #fuped and stop evaluating classifiers. But we are not, so it will continue. Next rule is onebypass, it will check if user has tag "onebypass", if yes, it will set service to #onebypass and stop evaluating classifiers. This is our case, so it will set service to #onebypass and stop evaluating classifiers.
|
||||
* Then it will read service #onebypass and activate classes bypass and default, set cir of bypass to 10Mbit and default to 100% of account speed.
|
||||
* Also it will activate filters fnaonebypass and ggconebypass, so all fna and ggc traffic will be matched to bypass class.
|
||||
|
||||
To verify if user is matched to correct service, you can use tc command to show classes of user, for example:
|
||||
```
|
||||
tc -s -d class show dev pppX
|
||||
```
|
||||
Where pppX is user ppp interface name of user identified by accel-cmd show sessions|grep username for example. And you can see two classes, one for fna + ggc traffic, and one for all other traffic.
|
||||
|
||||
## I have large list of ips to match, how to do it?
|
||||
|
||||
For example such situation might be with iptv traffic, where you have large list of ips to match, or speedtest servers, etc.
|
||||
You need to add following changes to your firewall first, note that you need to place each line in proper place, not as is. E.g. ipset create added after lines where we flush and destroy old ipset rules and etc.
|
||||
```
|
||||
ipset create bypass hash:net skbinfo
|
||||
|
||||
for i in $(cat /etc/speedtest.txt | awk '{ print $1; }') ; do ipset -A bypass $i skbmark 0x15 ; done
|
||||
|
||||
iptables -t mangle -A PREROUTING -i bond0 -j SET --map-set bypass src --map-mark
|
||||
```
|
||||
|
||||
in config.json you can add filter speedtest:
|
||||
```
|
||||
"speedtest": {
|
||||
"prio": 5,
|
||||
"classid": 10,
|
||||
"criteria": "handle 0x15 fw"
|
||||
},
|
||||
```
|
||||
|
||||
|
||||
First line will create ipset named bypass with hash:net type, which is used to store list of ips and together with skbinfo module to store skbmark value which can be matched by "handle 0x15 fw" in filters.
|
||||
Second line will read file /etc/speedtest.txt and add each ip to ipset bypass with skbmark 0x15. You can change 0x15 to any other value. This file should contain list of ips, one per line.
|
||||
Third line will add iptables rule to match all packets with source ip in ipset bypass and set skbmark to 0x15.
|
||||
|
||||
## Upload shaper
|
||||
|
||||
Settings in burst2d.cfg:
|
||||
|
||||
- `upload_shaper=0` unlimited upload
|
||||
- `upload_shaper=1` upload set as in radius
|
||||
|
||||
### Verifying upload shaper
|
||||
|
||||
`tc -s -d filter show dev pppX ingress`
|
||||
|
||||
```
|
||||
filter parent ffff: protocol all pref 49152 u32 chain 0
|
||||
filter parent ffff: protocol all pref 49152 u32 chain 0 fh 800: ht divisor 1
|
||||
filter parent ffff: protocol all pref 49152 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 terminal flowid not_in_hw (rule hit 29 success 29)
|
||||
match 00000000/00000000 at 0 (success 29 )
|
||||
police 0x3 rate 20Mbit burst 10000Kb mtu 65000b action drop overhead 0b linklayer ethernet
|
||||
ref 1 bind 1 installed 867 sec used 0 sec firstused 840 sec
|
||||
|
||||
Sent 4640 bytes 29 pkts (dropped 0, overlimits 0)
|
||||
```
|
||||
As you can see it is set to 20Mbit with burstability in bucket to 10Mbyte.
|
||||
|
||||
Upload shaper, when enabled, is single for all traffic and set as upload speed in billing(radius). Bypass classes and such stuff is not relevant to upload shaper.
|
||||
@@ -0,0 +1,826 @@
|
||||
# DoH Server
|
||||
|
||||
## Note!
|
||||
|
||||
This setup verified partially only. It is slightly better than blocking, but likely DOH clients will refuse to work due certificate mismatch.
|
||||
|
||||
## Installing/updating unbound
|
||||
|
||||
1. Update typical docker image `docker pull alpinelinux/unbound`
|
||||
2. You might beed to do in some cases `docker-compose build --no-cache` in directory of unbound docker-compose.yml
|
||||
3. Generate certificates, same directory where unbound.conf is located
|
||||
|
||||
```bash
|
||||
openssl genrsa -out key.pem 4096
|
||||
openssl req -new -x509 -key key.pem -out cert.pem -days 3650 \
|
||||
-subj "/C=US/ST=State/L=City/O=Organization/CN=spinesystems.solutions"
|
||||
```
|
||||
4. Add the following to unbound.conf to `server:` section
|
||||
|
||||
```
|
||||
server:
|
||||
...
|
||||
interface: 0.0.0.0@443
|
||||
tls-service-key: "/etc/unbound/key.pem"
|
||||
tls-service-pem: "/etc/unbound/cert.pem"
|
||||
```
|
||||
|
||||
5. Run the container `docker-compose up -d`
|
||||
|
||||
Your unbound ready now to accept DoH requests on port 443
|
||||
|
||||
## IPTables on nat/pppoe
|
||||
|
||||
|
||||
```
|
||||
ipset create dohservers hash:net family inet
|
||||
|
||||
# Cloudflare DNS (1.1.1.1, 1.0.0.1)
|
||||
ipset add dohservers 1.1.1.1/32
|
||||
ipset add dohservers 1.0.0.1/32
|
||||
#ipset add dohservers 2606:4700:4700::1111/128
|
||||
#ipset add dohservers 2606:4700:4700::1001/128
|
||||
|
||||
# Google DNS (8.8.8.8, 8.8.4.4)
|
||||
ipset add dohservers 8.8.8.8/32
|
||||
ipset add dohservers 8.8.4.4/32
|
||||
#ipset add dohservers 2001:4860:4860::8888/128
|
||||
#ipset add dohservers 2001:4860:4860::8844/128
|
||||
|
||||
# Quad9 DNS (9.9.9.9, 149.112.112.112)
|
||||
ipset add dohservers 9.9.9.9/32
|
||||
ipset add dohservers 149.112.112.112/32
|
||||
#ipset add dohservers 2620:fe::fe/128
|
||||
#ipset add dohservers 2620:fe::9/128
|
||||
|
||||
# NextDNS (45.90.28.0 - 45.90.31.255)
|
||||
ipset add dohservers 45.90.28.0/22
|
||||
#ipset add dohservers 2a07:a8c0::/29
|
||||
|
||||
# AdGuard DNS (94.140.14.14, 94.140.15.15)
|
||||
ipset add dohservers 94.140.14.14/32
|
||||
ipset add dohservers 94.140.15.15/32
|
||||
#ipset add dohservers 2a10:50c0::ad1:ff/128
|
||||
#ipset add dohservers 2a10:50c0::ad2:ff/128
|
||||
|
||||
# OpenDNS (Cisco Umbrella) (208.67.222.222, 208.67.220.220)
|
||||
ipset add dohservers 208.67.222.222/32
|
||||
ipset add dohservers 208.67.220.220/32
|
||||
#ipset add dohservers 2620:119:35::35/128
|
||||
#ipset add dohservers 2620:119:53::53/128
|
||||
|
||||
# DNS.SB (185.222.222.222, 185.184.222.222)
|
||||
ipset add dohservers 185.222.222.222/32
|
||||
ipset add dohservers 185.184.222.222/32
|
||||
#ipset add dohservers 2a09::/32
|
||||
|
||||
# cloudflare-dns.com
|
||||
dig +short cloudflare-dns.com A | xargs -n1 ipset add dohservers
|
||||
# dns.google
|
||||
dig +short dns.google A | xargs -n1 ipset add dohservers
|
||||
# dns9.quad9.net
|
||||
dig +short dns9.quad9.net A | xargs -n1 ipset add dohservers
|
||||
|
||||
|
||||
# Here goes other ipset stuff and maybe iptables rules...
|
||||
|
||||
iptables -t nat -A PREROUTING -p tcp --dport 443 -m set --match-set dohservers dst -j DNAT --to-destination 10.0.100.6:443
|
||||
```
|
||||
|
||||
## More complete DOH server list
|
||||
|
||||
Thanks to Khodor for contribution!
|
||||
|
||||
```
|
||||
1.1.1.1
|
||||
1.0.0.1
|
||||
1.1.1.2
|
||||
1.0.0.2
|
||||
1.1.1.3
|
||||
1.0.0.3
|
||||
104.16.132.229
|
||||
104.16.133.229
|
||||
168.235.81.167
|
||||
176.56.236.175
|
||||
176.103.130.131
|
||||
176.103.130.130
|
||||
176.103.130.132
|
||||
176.103.130.134
|
||||
37.252.185.229
|
||||
206.189.215.75
|
||||
104.24.120.142
|
||||
104.24.121.142
|
||||
108.61.201.119
|
||||
139.59.48.222
|
||||
104.16.249.249
|
||||
104.16.248.249
|
||||
199.58.81.218
|
||||
23.92.29.236
|
||||
104.28.0.106
|
||||
104.28.1.106
|
||||
8.8.4.4
|
||||
8.8.8.8
|
||||
185.95.218.42
|
||||
185.95.218.43
|
||||
185.222.222.222
|
||||
185.184.222.222
|
||||
46.101.66.244
|
||||
172.64.108.27
|
||||
172.64.109.27
|
||||
45.77.124.64
|
||||
45.32.253.116
|
||||
104.236.178.232
|
||||
89.234.186.112
|
||||
45.90.28.0
|
||||
45.90.30.0
|
||||
193.17.47.1
|
||||
185.43.135.1
|
||||
136.144.215.158
|
||||
118.126.68.223
|
||||
118.89.110.78
|
||||
47.96.179.163
|
||||
145.100.185.15
|
||||
145.100.185.16
|
||||
174.138.29.175
|
||||
45.77.180.10
|
||||
185.216.27.142
|
||||
217.169.20.23
|
||||
217.169.20.22
|
||||
172.65.3.223
|
||||
188.60.252.16
|
||||
149.112.112.112
|
||||
9.9.9.9
|
||||
149.112.112.9
|
||||
9.9.9.10
|
||||
149.112.112.10
|
||||
9.9.9.11
|
||||
149.112.112.11
|
||||
9.9.9.12
|
||||
149.112.112.12
|
||||
9.9.9.13
|
||||
149.112.112.13
|
||||
43.132.150.250
|
||||
103.254.108.74
|
||||
51.38.83.141
|
||||
51.38.82.198
|
||||
206.189.142.179
|
||||
94.237.80.211
|
||||
185.228.168.9
|
||||
185.228.169.9
|
||||
185.228.168.10
|
||||
185.228.169.11
|
||||
185.228.168.168
|
||||
185.228.169.168
|
||||
139.162.131.245
|
||||
76.76.2.1
|
||||
76.76.2.2
|
||||
76.76.2.3
|
||||
76.76.2.4
|
||||
76.76.2.5
|
||||
76.76.10.1
|
||||
76.76.10.2
|
||||
76.76.10.3
|
||||
76.76.10.4
|
||||
76.76.10.5
|
||||
51.89.22.36
|
||||
164.132.45.112
|
||||
172.64.105.36
|
||||
172.64.104.36
|
||||
195.30.94.28
|
||||
125.77.154.35
|
||||
104.27.164.27
|
||||
104.27.165.27
|
||||
47.101.136.37
|
||||
114.115.240.175
|
||||
119.29.107.85
|
||||
118.24.208.197
|
||||
103.2.57.5
|
||||
103.2.57.6
|
||||
174.138.21.128
|
||||
172.104.93.80
|
||||
146.148.56.78
|
||||
185.157.233.92
|
||||
210.17.9.228
|
||||
96.113.151.141
|
||||
96.113.151.142
|
||||
96.113.151.143
|
||||
96.113.151.147
|
||||
96.113.151.148
|
||||
96.113.151.149
|
||||
96.113.151.150
|
||||
167.114.220.125
|
||||
149.56.228.45
|
||||
5.2.75.75
|
||||
45.79.120.233
|
||||
45.67.219.208
|
||||
185.213.26.187
|
||||
45.132.75.16
|
||||
45.91.95.12
|
||||
45.132.74.167
|
||||
185.175.56.133
|
||||
193.29.62.196
|
||||
103.73.64.132
|
||||
80.156.145.201
|
||||
95.216.229.153
|
||||
51.15.124.208
|
||||
104.168.247.138
|
||||
45.153.187.96
|
||||
185.235.81.1
|
||||
185.235.81.2
|
||||
185.235.81.3
|
||||
185.235.81.4
|
||||
185.235.81.5
|
||||
185.235.81.6
|
||||
146.112.41.2
|
||||
146.112.41.3
|
||||
146.112.41.4
|
||||
208.67.222.222
|
||||
208.67.220.220
|
||||
208.67.222.123
|
||||
208.67.220.123
|
||||
208.67.220.2
|
||||
208.67.222.2
|
||||
88.198.161.8
|
||||
116.203.35.255
|
||||
116.203.70.156
|
||||
174.68.248.77
|
||||
185.26.126.37
|
||||
178.62.214.105
|
||||
35.231.247.227
|
||||
116.203.115.192
|
||||
5.45.107.88
|
||||
46.239.223.80
|
||||
51.158.147.50
|
||||
93.177.65.183
|
||||
146.255.56.98
|
||||
176.9.93.198
|
||||
176.9.1.117
|
||||
13.89.120.251
|
||||
40.76.112.230
|
||||
116.203.179.248
|
||||
198.251.90.114
|
||||
198.251.90.89
|
||||
209.141.34.95
|
||||
199.195.251.84
|
||||
104.244.78.231
|
||||
104.18.2.55
|
||||
104.18.3.55
|
||||
104.18.26.128
|
||||
104.18.27.128
|
||||
149.112.121.10
|
||||
149.112.122.10
|
||||
149.112.121.20
|
||||
149.112.122.20
|
||||
149.112.121.30
|
||||
149.112.122.30
|
||||
74.82.42.42
|
||||
223.5.5.5
|
||||
223.6.6.6
|
||||
130.59.31.251
|
||||
130.59.31.248
|
||||
45.90.57.121
|
||||
95.216.212.177
|
||||
78.46.244.143
|
||||
139.162.112.47
|
||||
192.53.175.149
|
||||
8.26.56.26
|
||||
8.20.247.20
|
||||
139.59.55.13
|
||||
104.24.122.53
|
||||
104.24.123.53
|
||||
209.250.241.25
|
||||
149.154.153.153
|
||||
185.233.106.232
|
||||
185.233.107.4
|
||||
3.0.59.48
|
||||
54.169.103.244
|
||||
172.67.153.154
|
||||
104.28.13.215
|
||||
104.28.12.215
|
||||
176.9.199.158
|
||||
46.227.200.54
|
||||
46.227.200.55
|
||||
185.134.196.54
|
||||
185.134.197.54
|
||||
46.227.200.52
|
||||
46.227.203.52
|
||||
185.134.196.52
|
||||
80.241.218.68
|
||||
159.69.114.157
|
||||
91.239.100.100
|
||||
130.225.244.166
|
||||
130.226.161.34
|
||||
185.38.24.52
|
||||
198.180.150.12
|
||||
89.233.43.71
|
||||
45.76.113.31
|
||||
139.99.222.72
|
||||
45.90.28.1
|
||||
45.90.28.2
|
||||
45.90.28.3
|
||||
45.90.28.4
|
||||
45.90.28.5
|
||||
45.90.28.6
|
||||
45.90.28.7
|
||||
45.90.28.8
|
||||
45.90.28.9
|
||||
45.90.28.10
|
||||
45.90.28.11
|
||||
45.90.28.12
|
||||
45.90.28.13
|
||||
45.90.28.14
|
||||
45.90.28.15
|
||||
45.90.28.16
|
||||
45.90.28.17
|
||||
45.90.28.18
|
||||
45.90.28.19
|
||||
45.90.28.20
|
||||
45.90.28.21
|
||||
45.90.28.22
|
||||
45.90.28.23
|
||||
45.90.28.24
|
||||
45.90.28.25
|
||||
45.90.28.26
|
||||
45.90.28.27
|
||||
45.90.28.28
|
||||
45.90.28.29
|
||||
45.90.28.30
|
||||
45.90.28.31
|
||||
45.90.28.32
|
||||
45.90.28.33
|
||||
45.90.28.34
|
||||
45.90.28.35
|
||||
45.90.28.36
|
||||
45.90.28.37
|
||||
45.90.28.38
|
||||
45.90.28.39
|
||||
45.90.28.40
|
||||
45.90.28.41
|
||||
45.90.28.42
|
||||
45.90.28.43
|
||||
45.90.28.44
|
||||
45.90.28.45
|
||||
45.90.28.46
|
||||
45.90.28.47
|
||||
45.90.28.48
|
||||
45.90.28.49
|
||||
45.90.28.50
|
||||
45.90.28.51
|
||||
45.90.28.52
|
||||
45.90.28.53
|
||||
45.90.28.54
|
||||
45.90.28.55
|
||||
45.90.28.56
|
||||
45.90.28.57
|
||||
45.90.28.58
|
||||
45.90.28.59
|
||||
45.90.28.60
|
||||
45.90.28.61
|
||||
45.90.28.62
|
||||
45.90.28.63
|
||||
45.90.28.64
|
||||
45.90.28.65
|
||||
45.90.28.66
|
||||
45.90.28.67
|
||||
45.90.28.68
|
||||
45.90.28.69
|
||||
45.90.28.70
|
||||
45.90.28.71
|
||||
45.90.28.72
|
||||
45.90.28.73
|
||||
45.90.28.74
|
||||
45.90.28.75
|
||||
45.90.28.76
|
||||
45.90.28.77
|
||||
45.90.28.78
|
||||
45.90.28.79
|
||||
45.90.28.80
|
||||
45.90.28.81
|
||||
45.90.28.82
|
||||
45.90.28.83
|
||||
45.90.28.84
|
||||
45.90.28.85
|
||||
45.90.28.86
|
||||
45.90.28.87
|
||||
45.90.28.88
|
||||
45.90.28.89
|
||||
45.90.28.90
|
||||
45.90.28.91
|
||||
45.90.28.92
|
||||
45.90.28.93
|
||||
45.90.28.94
|
||||
45.90.28.95
|
||||
45.90.28.96
|
||||
45.90.28.97
|
||||
45.90.28.98
|
||||
45.90.28.99
|
||||
45.90.28.100
|
||||
45.90.28.101
|
||||
45.90.28.102
|
||||
45.90.28.103
|
||||
45.90.28.104
|
||||
45.90.28.105
|
||||
45.90.28.106
|
||||
45.90.28.107
|
||||
45.90.28.108
|
||||
45.90.28.109
|
||||
45.90.28.110
|
||||
45.90.28.111
|
||||
45.90.28.112
|
||||
45.90.28.113
|
||||
45.90.28.114
|
||||
45.90.28.115
|
||||
45.90.28.116
|
||||
45.90.28.117
|
||||
45.90.28.118
|
||||
45.90.28.119
|
||||
45.90.28.120
|
||||
45.90.28.121
|
||||
45.90.28.122
|
||||
45.90.28.123
|
||||
45.90.28.124
|
||||
45.90.28.125
|
||||
45.90.28.126
|
||||
45.90.28.127
|
||||
45.90.28.128
|
||||
45.90.28.129
|
||||
45.90.28.130
|
||||
45.90.28.131
|
||||
45.90.28.132
|
||||
45.90.28.133
|
||||
45.90.28.134
|
||||
45.90.28.135
|
||||
45.90.28.136
|
||||
45.90.28.137
|
||||
45.90.28.138
|
||||
45.90.28.139
|
||||
45.90.28.140
|
||||
45.90.28.141
|
||||
45.90.28.142
|
||||
45.90.28.143
|
||||
45.90.28.144
|
||||
45.90.28.145
|
||||
45.90.28.146
|
||||
45.90.28.147
|
||||
45.90.28.148
|
||||
45.90.28.149
|
||||
45.90.28.150
|
||||
45.90.28.151
|
||||
45.90.28.152
|
||||
45.90.28.153
|
||||
45.90.28.154
|
||||
45.90.28.155
|
||||
45.90.28.156
|
||||
45.90.28.157
|
||||
45.90.28.158
|
||||
45.90.28.159
|
||||
45.90.28.160
|
||||
45.90.28.161
|
||||
45.90.28.162
|
||||
45.90.28.163
|
||||
45.90.28.164
|
||||
45.90.28.165
|
||||
45.90.28.166
|
||||
45.90.28.167
|
||||
45.90.28.168
|
||||
45.90.28.169
|
||||
45.90.28.170
|
||||
45.90.28.171
|
||||
45.90.28.172
|
||||
45.90.28.173
|
||||
45.90.28.174
|
||||
45.90.28.175
|
||||
45.90.28.176
|
||||
45.90.28.177
|
||||
45.90.28.178
|
||||
45.90.28.179
|
||||
45.90.28.180
|
||||
45.90.28.181
|
||||
45.90.28.182
|
||||
45.90.28.183
|
||||
45.90.28.184
|
||||
45.90.28.185
|
||||
45.90.28.186
|
||||
45.90.28.187
|
||||
45.90.28.188
|
||||
45.90.28.189
|
||||
45.90.28.190
|
||||
45.90.28.191
|
||||
45.90.28.192
|
||||
45.90.28.193
|
||||
45.90.28.194
|
||||
45.90.28.195
|
||||
45.90.28.196
|
||||
45.90.28.197
|
||||
45.90.28.198
|
||||
45.90.28.199
|
||||
45.90.28.200
|
||||
45.90.28.201
|
||||
45.90.28.202
|
||||
45.90.28.203
|
||||
45.90.28.204
|
||||
45.90.28.205
|
||||
45.90.28.206
|
||||
45.90.28.207
|
||||
45.90.28.208
|
||||
45.90.28.209
|
||||
45.90.28.210
|
||||
45.90.28.211
|
||||
45.90.28.212
|
||||
45.90.28.213
|
||||
45.90.28.214
|
||||
45.90.28.215
|
||||
45.90.28.216
|
||||
45.90.28.217
|
||||
45.90.28.218
|
||||
45.90.28.219
|
||||
45.90.28.220
|
||||
45.90.28.221
|
||||
45.90.28.222
|
||||
45.90.28.223
|
||||
45.90.28.224
|
||||
45.90.28.225
|
||||
45.90.28.226
|
||||
45.90.28.227
|
||||
45.90.28.228
|
||||
45.90.28.229
|
||||
45.90.28.230
|
||||
45.90.28.231
|
||||
45.90.28.232
|
||||
45.90.28.233
|
||||
45.90.28.234
|
||||
45.90.28.235
|
||||
45.90.28.236
|
||||
45.90.28.237
|
||||
45.90.28.238
|
||||
45.90.28.239
|
||||
45.90.28.240
|
||||
45.90.28.241
|
||||
45.90.28.242
|
||||
45.90.28.243
|
||||
45.90.28.244
|
||||
45.90.28.245
|
||||
45.90.28.246
|
||||
45.90.28.247
|
||||
45.90.28.248
|
||||
45.90.28.249
|
||||
45.90.28.250
|
||||
45.90.28.251
|
||||
45.90.28.252
|
||||
45.90.28.253
|
||||
45.90.28.254
|
||||
45.90.30.1
|
||||
45.90.30.2
|
||||
45.90.30.3
|
||||
45.90.30.4
|
||||
45.90.30.5
|
||||
45.90.30.6
|
||||
45.90.30.7
|
||||
45.90.30.8
|
||||
45.90.30.9
|
||||
45.90.30.10
|
||||
45.90.30.11
|
||||
45.90.30.12
|
||||
45.90.30.13
|
||||
45.90.30.14
|
||||
45.90.30.15
|
||||
45.90.30.16
|
||||
45.90.30.17
|
||||
45.90.30.18
|
||||
45.90.30.19
|
||||
45.90.30.20
|
||||
45.90.30.21
|
||||
45.90.30.22
|
||||
45.90.30.23
|
||||
45.90.30.24
|
||||
45.90.30.25
|
||||
45.90.30.26
|
||||
45.90.30.27
|
||||
45.90.30.28
|
||||
45.90.30.29
|
||||
45.90.30.30
|
||||
45.90.30.31
|
||||
45.90.30.32
|
||||
45.90.30.33
|
||||
45.90.30.34
|
||||
45.90.30.35
|
||||
45.90.30.36
|
||||
45.90.30.37
|
||||
45.90.30.38
|
||||
45.90.30.39
|
||||
45.90.30.40
|
||||
45.90.30.41
|
||||
45.90.30.42
|
||||
45.90.30.43
|
||||
45.90.30.44
|
||||
45.90.30.45
|
||||
45.90.30.46
|
||||
45.90.30.47
|
||||
45.90.30.48
|
||||
45.90.30.49
|
||||
45.90.30.50
|
||||
45.90.30.51
|
||||
45.90.30.52
|
||||
45.90.30.53
|
||||
45.90.30.54
|
||||
45.90.30.55
|
||||
45.90.30.56
|
||||
45.90.30.57
|
||||
45.90.30.58
|
||||
45.90.30.59
|
||||
45.90.30.60
|
||||
45.90.30.61
|
||||
45.90.30.62
|
||||
45.90.30.63
|
||||
45.90.30.64
|
||||
45.90.30.65
|
||||
45.90.30.66
|
||||
45.90.30.67
|
||||
45.90.30.68
|
||||
45.90.30.69
|
||||
45.90.30.70
|
||||
45.90.30.71
|
||||
45.90.30.72
|
||||
45.90.30.73
|
||||
45.90.30.74
|
||||
45.90.30.75
|
||||
45.90.30.76
|
||||
45.90.30.77
|
||||
45.90.30.78
|
||||
45.90.30.79
|
||||
45.90.30.80
|
||||
45.90.30.81
|
||||
45.90.30.82
|
||||
45.90.30.83
|
||||
45.90.30.84
|
||||
45.90.30.85
|
||||
45.90.30.86
|
||||
45.90.30.87
|
||||
45.90.30.88
|
||||
45.90.30.89
|
||||
45.90.30.90
|
||||
45.90.30.91
|
||||
45.90.30.92
|
||||
45.90.30.93
|
||||
45.90.30.94
|
||||
45.90.30.95
|
||||
45.90.30.96
|
||||
45.90.30.97
|
||||
45.90.30.98
|
||||
45.90.30.99
|
||||
45.90.30.100
|
||||
45.90.30.101
|
||||
45.90.30.102
|
||||
45.90.30.103
|
||||
45.90.30.104
|
||||
45.90.30.105
|
||||
45.90.30.106
|
||||
45.90.30.107
|
||||
45.90.30.108
|
||||
45.90.30.109
|
||||
45.90.30.110
|
||||
45.90.30.111
|
||||
45.90.30.112
|
||||
45.90.30.113
|
||||
45.90.30.114
|
||||
45.90.30.115
|
||||
45.90.30.116
|
||||
45.90.30.117
|
||||
45.90.30.118
|
||||
45.90.30.119
|
||||
45.90.30.120
|
||||
45.90.30.121
|
||||
45.90.30.122
|
||||
45.90.30.123
|
||||
45.90.30.124
|
||||
45.90.30.125
|
||||
45.90.30.126
|
||||
45.90.30.127
|
||||
45.90.30.128
|
||||
45.90.30.129
|
||||
45.90.30.130
|
||||
45.90.30.131
|
||||
45.90.30.132
|
||||
45.90.30.133
|
||||
45.90.30.134
|
||||
45.90.30.135
|
||||
45.90.30.136
|
||||
45.90.30.137
|
||||
45.90.30.138
|
||||
45.90.30.139
|
||||
45.90.30.140
|
||||
45.90.30.141
|
||||
45.90.30.142
|
||||
45.90.30.143
|
||||
45.90.30.144
|
||||
45.90.30.145
|
||||
45.90.30.146
|
||||
45.90.30.147
|
||||
45.90.30.148
|
||||
45.90.30.149
|
||||
45.90.30.150
|
||||
45.90.30.151
|
||||
45.90.30.152
|
||||
45.90.30.153
|
||||
45.90.30.154
|
||||
45.90.30.155
|
||||
45.90.30.156
|
||||
45.90.30.157
|
||||
45.90.30.158
|
||||
45.90.30.159
|
||||
45.90.30.160
|
||||
45.90.30.161
|
||||
45.90.30.162
|
||||
45.90.30.163
|
||||
45.90.30.164
|
||||
45.90.30.165
|
||||
45.90.30.166
|
||||
45.90.30.167
|
||||
45.90.30.168
|
||||
45.90.30.169
|
||||
45.90.30.170
|
||||
45.90.30.171
|
||||
45.90.30.172
|
||||
45.90.30.173
|
||||
45.90.30.174
|
||||
45.90.30.175
|
||||
45.90.30.176
|
||||
45.90.30.177
|
||||
45.90.30.178
|
||||
45.90.30.179
|
||||
45.90.30.180
|
||||
45.90.30.181
|
||||
45.90.30.182
|
||||
45.90.30.183
|
||||
45.90.30.184
|
||||
45.90.30.185
|
||||
45.90.30.186
|
||||
45.90.30.187
|
||||
45.90.30.188
|
||||
45.90.30.189
|
||||
45.90.30.190
|
||||
45.90.30.191
|
||||
45.90.30.192
|
||||
45.90.30.193
|
||||
45.90.30.194
|
||||
45.90.30.195
|
||||
45.90.30.196
|
||||
45.90.30.197
|
||||
45.90.30.198
|
||||
45.90.30.199
|
||||
45.90.30.200
|
||||
45.90.30.201
|
||||
45.90.30.202
|
||||
45.90.30.203
|
||||
45.90.30.204
|
||||
45.90.30.205
|
||||
45.90.30.206
|
||||
45.90.30.207
|
||||
45.90.30.208
|
||||
45.90.30.209
|
||||
45.90.30.210
|
||||
45.90.30.211
|
||||
45.90.30.212
|
||||
45.90.30.213
|
||||
45.90.30.214
|
||||
45.90.30.215
|
||||
45.90.30.216
|
||||
45.90.30.217
|
||||
45.90.30.218
|
||||
45.90.30.219
|
||||
45.90.30.220
|
||||
45.90.30.221
|
||||
45.90.30.222
|
||||
45.90.30.223
|
||||
45.90.30.224
|
||||
45.90.30.225
|
||||
45.90.30.226
|
||||
45.90.30.227
|
||||
45.90.30.228
|
||||
45.90.30.229
|
||||
45.90.30.230
|
||||
45.90.30.231
|
||||
45.90.30.232
|
||||
45.90.30.233
|
||||
45.90.30.234
|
||||
45.90.30.235
|
||||
45.90.30.236
|
||||
45.90.30.237
|
||||
45.90.30.238
|
||||
45.90.30.239
|
||||
45.90.30.240
|
||||
45.90.30.241
|
||||
45.90.30.242
|
||||
45.90.30.243
|
||||
45.90.30.244
|
||||
45.90.30.245
|
||||
45.90.30.246
|
||||
45.90.30.247
|
||||
45.90.30.248
|
||||
45.90.30.249
|
||||
45.90.30.250
|
||||
45.90.30.251
|
||||
45.90.30.252
|
||||
45.90.30.253
|
||||
45.90.30.254
|
||||
```
|
||||
+100
@@ -0,0 +1,100 @@
|
||||
## Preparing hardware for the GlobalOS
|
||||
|
||||
- CPU:
|
||||
- CPU control on frequency should be given to the OS
|
||||
- Enable all settings to performance, avoid "power saving" , "efficient" or "balanced"
|
||||
- Memory:
|
||||
- Make sure memory sticks propagated properly per-CPU and to utilize channels properly. Check server or motherboard manual for more information.
|
||||
- DIMM frequency should be set to the maximum supported by the RAM. We can verify that by running `dmidecode -t memory` and checking the frequency of the DIMMs.
|
||||
- Make sure to disable "Memory Interleaving" in the BIOS for NUMA systems.
|
||||
- Boot settings:
|
||||
- Disable "Secure Boot"
|
||||
- Enable "EFI" boot mode (CSM supported, but not stable)
|
||||
- Make USB stick or your boot media boot first. Install will be done from USB stick
|
||||
- Disable PXE boot and PXE options from the BIOS
|
||||
- Network:
|
||||
- Preferable to disable NIC you are not planning to use
|
||||
- iLOM/DRAC/BMC/IPMI
|
||||
- Make sure to configure it to be accessible from the network, it is very useful for recovery and monitoring
|
||||
- Network cards
|
||||
- Make sure you have enough lanes in your slot. Some PCIe risers have 8x lane slot, but only 4x are propagated on the motherboard. Motherboard manual will have the information, but we can also check that on boot.
|
||||
- If you are installing multiple network cards, and you have dual CPU, make sure to install them in different NUMA nodes.
|
||||
- Make sure you have spare SFP/DAC, preferable different brands if you didn't tested them with your network cards.
|
||||
- Boot storage/media
|
||||
- If you are using USB stick, make sure it is geniune/not fake. You can check that by: ValiDrive https://www.grc.com/validrive.htm , best one is H2testw https://h2testw.org/
|
||||
|
||||
## HP Gen9 servers bug (single core only)
|
||||
|
||||
Some HP servers might not detect all CPU cores, and show only one, then you need to do changes in BIOS:
|
||||
|
||||
1. F9 to enter the BIOS
|
||||
2. Go to System Configuration
|
||||
3. Go to BIOS/Platform Configuration (RBSU)
|
||||
4. Go to Processor Options
|
||||
5. Go to Processor x2APIC Support
|
||||
6. Select Disable
|
||||
7. To save the changes you must press “F10” and then the key “Y” to save.
|
||||
8. Hit ESC couple of times to exit
|
||||
|
||||
## HP Proliant Gen9 disabling network boot
|
||||
|
||||
It is recommended to disable PXE boot, unless you are using PXE version of GlobalOS.
|
||||
|
||||
Prerequisite
|
||||
|
||||
Boot Mode is set to UEFI Mode.
|
||||
|
||||
Procedure
|
||||
|
||||
1. From the System Utilities screen, select System Configuration > BIOS/Platform Configuration (RBSU) > Network Options > Network Boot Options > PCIe Slot Network Boot and press Enter.
|
||||
2. Select a PCIe slot entry and press Enter.
|
||||
3. Select a setting and press Enter.
|
||||
PXE Boot—Enables PXE boot.
|
||||
Disabled—Disables PXE boot.
|
||||
4. Press F10.
|
||||
|
||||
## How to verify PCIe slot
|
||||
|
||||
If you install card in 4x lane PCIe slot, while card designed for 8x PCIe, most likely your card wont be able to reach max bandwidth and will drop packets due PCIe bus bandwidth limit.
|
||||
|
||||
```
|
||||
pppoe01 ~ # cat /sys/class/net/eth3/device/current_link_width
|
||||
8
|
||||
pppoe01 ~ # cat /sys/class/net/eth3/device/max_link_width
|
||||
8
|
||||
```
|
||||
This means card working on maximum PCI lanes
|
||||
|
||||
```
|
||||
pppoe04 ~ # cat /sys/class/net/eth5/device/current_link_width
|
||||
4
|
||||
pppoe04 ~ # cat /sys/class/net/eth5/device/max_link_width
|
||||
8
|
||||
```
|
||||
And here you can see card working in PCIe 4x lanes slot, while it supports 8x.
|
||||
|
||||
## HP Proliant Gen9 CPU dynamic frequency settings
|
||||
|
||||
1. **Access the System Utilities:**
|
||||
- Power on or reboot the server.
|
||||
- During the boot process, when prompted, press the F9 key to launch the System Utilities.
|
||||
|
||||
2. **Navigate to the Power Profile Setting:**
|
||||
- Within the System Utilities, select "System Configuration."
|
||||
- Choose "BIOS/Platform Configuration (RBSU)."
|
||||
- Navigate to "Power Management."
|
||||
- Select "Power Profile" and press Enter.
|
||||
|
||||
3. **Select the Desired Power Profile:**
|
||||
- You'll see a list of available power profiles.
|
||||
- Choose the profile: Custom.
|
||||
- Press Enter to confirm your selection.
|
||||
|
||||
4. **Go back to the main menu**
|
||||
|
||||
5. **From the System Utilities screen:**
|
||||
- select System Configuration, then BIOS/Platform Configuration (RBSU) > Power Management > Power Regulator and press Enter.
|
||||
- Select: "OS Control Mode" —Processors run in their maximum power and performance state at all times, unless the OS enables a power management policy.
|
||||
- Press Enter to confirm your selection.
|
||||
|
||||
6. **Press F10 to save your changes and exit.**
|
||||
+189
@@ -0,0 +1,189 @@
|
||||
# GlobalOS introduction
|
||||
|
||||
GlobalOS is custom Linux distribution build from scratch. It is based on Linux kernel and uses BusyBox+glibc as userland. It is designed to be small, fast and secure, to be used in network appliances, embedded systems and virtual machines.
|
||||
|
||||
## Major differences from other distributions
|
||||
|
||||
- **No package manager**. GlobalOS is built from scratch and does not use any package manager. It is built as a single image (but contains multiple files) that contains all the necessary software. You can compile and add software to the running system, and install in /usr/local, but it will not be managed by any package manager.
|
||||
- **No init system**. GlobalOS uses a custom init system that is built into the kernel. It is a simple init system that starts all the necessary services and then waits for the system to be shut down. It does not support runlevels.
|
||||
- **No bash/fish**. GlobalOS does not have traditional shell like bash or fish. It uses a custom shell that is built into the busybox - ash. It is a POSIX compliant shell that is very small and fast.
|
||||
- **No systemd**. GlobalOS does not use systemd. It uses a custom init system that is built into the kernel.
|
||||
- **No X11, desktop environment**. GlobalOS does not have X11 or any other graphical environment. It is designed to be used in headless systems. It does provide web interface for configuration for some services.
|
||||
- **Custom persistence**. GlobalOS uses a custom persistence mechanism. It means in most of cases if you edit files in /etc, you need to issue `gs save` command to save the changes to the disk. This is done to prevent data loss in case of power failure or other unexpected shutdowns.
|
||||
- **Custom configuration**. GlobalOS uses a custom configuration system. It uses a multiple files, different from traditional distributions. Please check documentation for more information.
|
||||
|
||||
## Introduction to GlobalOS
|
||||
|
||||
Let's assume you have a fresh GlobalOS installation. You can login to the system using ssh with key authentication, using the default user `root`. Note: your ssh key might be installed by installer or certified engineer.
|
||||
|
||||
```bash
|
||||
ssh root@globalos.ip
|
||||
```
|
||||
|
||||
After you login, you will see the following prompt (will vary depending on the version and hostname):
|
||||
|
||||
```bash
|
||||
System RC (Firmware rev9.2)
|
||||
t-pppoe1 ~ #
|
||||
```
|
||||
|
||||
This is the GlobalOS shell prompt. You can issue commands here. You can use `exit` or `Ctrl+D` to exit the shell.
|
||||
|
||||
### View current network configuration
|
||||
|
||||
You can view the current network configuration using following commands:
|
||||
|
||||
- ip link show
|
||||
- ip addr show
|
||||
- ip route show
|
||||
|
||||
`ip link show` will show you the network interfaces available on the system. `ip addr show` will show you the IP addresses assigned to the interfaces. `ip route show` will show you the routing table.
|
||||
|
||||
Additional tools available for advanced network configuration are:
|
||||
- ip neigh (ARP table)
|
||||
- ip rule (routing rules, policy routing, similar to PBR in Cisco or FBF in Juniper)
|
||||
- ethtool (NIC configuration, link status, etc.)
|
||||
|
||||
You can get familiar with these tools by reading their documentation:
|
||||
- iproute2: https://wiki.linuxfoundation.org/networking/iproute2 https://www.man7.org/linux/man-pages/man8/ip-route.8.html (follow also links to other iproute2 man subpages)
|
||||
- ethtool: https://man7.org/linux/man-pages/man8/ethtool.8.html
|
||||
|
||||
### Edit configuration files
|
||||
|
||||
GlobalOS uses a custom configuration system. It uses multiple files for configuration, instead of a single file like in traditional distributions. The configuration files are stored in `/etc` directory. You can edit the configuration files using the `nano` editor. You can also use `cat`, `echo`, `sed`, `awk`, etc. to edit the files.
|
||||
Network configuration file is `/etc/network.cfg`, this is shell script executed only once at boot time. Firewall (legacy iptables) configuration is in `/etc/firewall.cfg`, this is shell script executed every time firewall is reloaded. Your can run it multiple times, but make sure rules are idempotent, e.g. you clean first all chains, and then add rules.
|
||||
`resolv.conf` is in `/etc/resolv.conf`, specify DNS servers here. You can also use `/etc/hosts` to specify static DNS entries.
|
||||
|
||||
So how for example `network.cfg` looks like? Here is an example:
|
||||
|
||||
```bash
|
||||
#!/bin/sh
|
||||
modprobe i40e
|
||||
modprobe ixgbe
|
||||
modprobe mlx4_core
|
||||
modprobe mlx5_core
|
||||
|
||||
ip link set dev eth0 up
|
||||
ip link set dev eth1 up
|
||||
ip link set dev eth2 up
|
||||
ip addr add 1.2.3.2/30 dev eth0
|
||||
|
||||
ip route replace default via 1.2.3.1 dev eth0
|
||||
```
|
||||
As you can see, it is a shell script that is executed at boot time. It loads the necessary kernel modules (some advanced NICs require kernel modules to be loaded), brings up the interfaces, assigns IP addresses and sets the default route.
|
||||
But as file is not idempotent, if you execute it multiple times, you will get errors. So you need to make sure that you clean up the configuration first, before applying it again.
|
||||
If you need to change configuration at runtime without reboot, you can use `ip` command to change the configuration, verify it, then edit `/etc/network.cfg` and save it with `gs save` command.
|
||||
|
||||
For example, to change IP address of eth0 to 2.2.2.2/30, you can use following commands:
|
||||
|
||||
```bash
|
||||
ip addr del dev eth0 1.2.3.2/30; ip addr add dev eth0 2.2.2.2/30; ip route replace default via 2.2.2.1 dev eth0
|
||||
```
|
||||
|
||||
Here you have certain risk, that if you make a mistake, you will lose connection to the system, it is better to ensure having either console access or out-of-band management.
|
||||
Then if you are sure that configuration is correct, you can edit `/etc/network.cfg` and save it with `gs save` command.
|
||||
Change lines:
|
||||
`ip addr add 1.2.3.2/30 dev eth0` to `ip addr add 2.2.2.2/30 dev eth0`
|
||||
`ip route replace default via 1.2.3.1 dev eth0` to `ip route replace default via 2.2.2.1 dev eth0`
|
||||
|
||||
Then save the configuration with `gs save` command.
|
||||
|
||||
### Save configuration
|
||||
|
||||
GlobalOS uses a custom persistence mechanism. It means in most of cases if you edit files in /etc, you need to issue `gs save` command to save the changes to the disk. This is done to prevent data loss in case of power failure or other unexpected shutdowns. Configuration will be saved to the flash disk, `/mnt/flash/secure2/config.tgz` file.
|
||||
|
||||
### Apply configuration
|
||||
|
||||
* /etc/network.cfg - require full reboot
|
||||
* /etc/accel-ppp.conf - restart accel-pppd daemon, in some cases accel-cmd reload is enough
|
||||
* /etc/nft.cfg - nft -f /etc/nft.cfg
|
||||
* /etc/config.json - killall burst2d
|
||||
* /etc/burst2d.cfg - killall burst2d
|
||||
* /etc/firewall.cfg - sh /etc/firewall.cfg
|
||||
|
||||
For restarting accel you can use following snippet:
|
||||
```
|
||||
killall accel-pppd; while true; do
|
||||
pid=$(pgrep accel-pppd)
|
||||
if [ -z "$pid" ]; then
|
||||
accel-pppd -c /etc/accel-ppp.conf -d
|
||||
break
|
||||
fi
|
||||
echo "Waiting for accel-pppd to finish"
|
||||
sleep 1
|
||||
done
|
||||
```
|
||||
|
||||
|
||||
### Network diagnostics and troubleshooting
|
||||
|
||||
If you have problems with network configuration, you can use following tools to diagnose the problem:
|
||||
- ping
|
||||
- iperf
|
||||
- tcpdump
|
||||
- traceroute
|
||||
- trafw
|
||||
- iptop
|
||||
- curl
|
||||
- dig
|
||||
- pppoe-discovery
|
||||
- conntrack
|
||||
|
||||
- `ping` is used to check if the host is reachable, and can be used to verify link reliability.
|
||||
For example `ping -q -s1472 -i0.0001 -c10000 1.2.3.1` will send full sized packets (MTU 1500) to the gateway, with interval of 0.0001ms, and count of 10000 packets. It is useful to check if there is any packet loss on the link. It is similar to Cisco rapid ping.
|
||||
- `iperf` is used to check the bandwidth between two hosts. You need to run iperf server on one host, and iperf client on the other host. For example `iperf -s` on one host, and `iperf -c IP.of.server` on the other host.
|
||||
- `tcpdump` is used to capture packets on the interface. You can use it to diagnose network problems, like missing packets, wrong packets, etc. For example `tcpdump -i eth0 -n -s65000 not port 22` will capture packets on eth0 interface, without resolving IP addresses, with full packet size, and not capturing ssh traffic (which is likely how you are connected to the system).
|
||||
- `traceroute` is used to trace the route to the destination. It is useful to diagnose routing problems. For example `traceroute 8.8.8.8` will show you the route to Google DNS server.
|
||||
- `trafw` is used to show current load on the interfaces, filtered using pcap filter. TODO: add example
|
||||
- `iptop` is used to show current load on the interfaces per dst/src subnets, filtered using iptables filter. TODO: add example
|
||||
- `curl` is used to download files from the internet. It is useful to check if the internet connection is working. For example `curl -I http://www.google.com` will show you the headers of the google.com website.
|
||||
- `dig` is used to query DNS servers. It is useful to check if the DNS servers are working. For example `dig www.google.com` will show you the IP address of the google.com website.
|
||||
- `pppoe-discovery` is used to discover PPPoE servers. It is useful to check if the PPPoE server is reachable in network, or if there is malicious PPPoE server in the network. For example `pppoe-discovery -I eth0` will show you the PPPoE servers on the eth0 interface. (it will not show pppoe server running on same server)
|
||||
- `conntrack` is used to show the connection tracking table. It is useful to check if the connections are being tracked correctly. For example `conntrack -L` will show you the connection tracking table. Also you can flush the table with `conntrack -F`.
|
||||
|
||||
### Directory structure
|
||||
|
||||
GlobalOS has a custom directory structure. It is different from traditional distributions. Here is the directory structure:
|
||||
|
||||
- `/bin`,`/sbin`, other first level directories, except `/usr`, `/mnt/flash` - special directories from kernel initramfs. You need to be carefuly, as changes and new files will use RAM, so you can run out of memory. Changes will not be saved.
|
||||
- `/mnt/flash` - contains the flash disk, where the configuration is saved and where the software is stored. You can write to this directory, and changes will be saved.
|
||||
- `/usr/*` - except `/usr/local`, contains the rest of the binaries, like `ping`, `iperf`, `tcpdump`, etc., this directory might be overwritten during runtime, but changes will not be saved
|
||||
|
||||
### BRAS(PPPoE/L2TP/IPoE/SSTP/PPTP) configuration
|
||||
|
||||
Some specific files are related to BRAS configuration:
|
||||
|
||||
- `/etc/accel-ppp.conf` - configuration file for accel-ppp, which implements PPPoE, L2TP, IPoE, SSTP, PPTP, etc. Mostly you can read the documentation of accel-ppp to configure it, but our version has some customizations, so here i will provide key differences.
|
||||
|
||||
```
|
||||
interface=eth6.3197,{"services":["SERVICENAME1","SERVICENAME2","CSP"],"service-blank":1,"acname":"HALBA-CSP"}
|
||||
```
|
||||
We support json format for services, so you can specify multiple services in one line, specific to each vlan. You can also specify AC-Name, which is used in PPPoE discovery.
|
||||
Also few other options are supported, like `service-blank`, which will allow to connect to the BRAS without specifying service name, `service-any` which will allow to connect to the BRAS with any service name.
|
||||
If you set `service-blank` to 1, this means PPPoE clients without service-name can connect to the BRAS. If you set it to 1 - configuring service name on client become mandatory.
|
||||
This might be useful if you are entering network of reseller or other ISP, and you want to give them trial, without disrupting existing clients, so you can set specific service name for them, and set `service-blank` to 0, so users without service name will not be able to connect.
|
||||
|
||||
TBD: Need to test live update of pppoe config without restard by "update" command i added to accel-ppp.
|
||||
|
||||
- `/etc/burst2d.cfg` - configuration file for burst2d, which implements shaper daemon system configuration.
|
||||
|
||||
```
|
||||
[global]
|
||||
static_config=/etc/config.json
|
||||
#tc=/usr/local/sbin/tc
|
||||
htb_extra=burst 4000000 cburst 4000000
|
||||
upload_shaper=0
|
||||
filter_swap=1
|
||||
#rx_ratio=1.0
|
||||
```
|
||||
|
||||
- `static_config` - path to the configuration file, which is in json format, and contains the configuration of the shaper itself.
|
||||
- `htb_extra` - additional options for htb qdisc, like burst and cburst, which are used to allow burst of traffic, and then limit it to the configured rate.
|
||||
- `upload_shaper` - if set to 1, will enable upload shaper, which is used to limit the upload speed of the clients. It might be disabled to increase speed of shaper updates and reduce CPU usage.
|
||||
- `filter_swap` - if set to 1, will swap the filters, so the filters will be applied in reverse order. Sometimes billing specify Upload/Download filters in reverse order, so this option is useful to apply them in correct order.
|
||||
- `rx_ratio` - ratio of download to download speed, if you want to apply some ratio to billing configured speed. For example if user have 2Mbps and `rx_ratio` is set to 0.5, then user will have 1Mbps download speed.
|
||||
|
||||
|
||||
- `/etc/config.json` - configuration file for burst2d, which implements shaper daemon system configuration. Please check `PPPOE_shaper.md` for more information.
|
||||
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
* To retrieve ASN list use get_asndb.sh
|
||||
* convert_tsv.py ip2asn-v4.tsv ip2asn-v4.csv
|
||||
Executable
+89
@@ -0,0 +1,89 @@
|
||||
#!/usr/bin/env python3
|
||||
# Convert a TSV file to a asn.txt
|
||||
# Source:
|
||||
# range_start range_end AS_number country_code AS_description
|
||||
# Result:
|
||||
# subnet/mask AS_number AS_description country_code
|
||||
# 1.0.0.0/24 13335 eetc US
|
||||
# 1.0.4.0/22 38803 etc AU
|
||||
|
||||
import sys
|
||||
import ipaddress
|
||||
import csv
|
||||
|
||||
def calculate_subnets(line_list):
|
||||
subnets = []
|
||||
while True:
|
||||
# Calculate from diff of range_start and range_end subnet size
|
||||
# 1.1.1.0 1.1.2.0 = 256
|
||||
subnet1_uint32 = int(ipaddress.IPv4Address(line_list[0]))
|
||||
subnet2_uint32 = int(ipaddress.IPv4Address(line_list[1]))
|
||||
subnet_size = subnet2_uint32 - subnet1_uint32
|
||||
# calculate bitmask
|
||||
subnet_mask = 32 - subnet_size.bit_length()
|
||||
# is subnet not exact size of 2^n? verify using subnet_mask
|
||||
p2 = (2**(32-subnet_mask))-1
|
||||
if subnet_size != p2:
|
||||
#print(f'Error: {line} is not a power of 2 {subnet_size} {subnet_mask} {p2}')
|
||||
# Calculate next subnet
|
||||
subnet_mask = subnet_mask + 1
|
||||
subnets.append(f'{line_list[0]}/{subnet_mask}')
|
||||
#print(f'Adding {line_list[0]}/{subnet_mask}')
|
||||
# Calculate next subnet
|
||||
subnet1_uint32 = subnet1_uint32 + (2**(32-subnet_mask))
|
||||
printable_subnet = str(ipaddress.IPv4Address(subnet1_uint32))
|
||||
#print(f'Next subnet {printable_subnet}')
|
||||
# if next subnet is not in range, break
|
||||
if subnet1_uint32 > subnet2_uint32:
|
||||
print(f'Error: Subnet {subnet1_uint32} is not in range {subnet2_uint32}')
|
||||
break
|
||||
line_list[0] = str(ipaddress.IPv4Address(subnet1_uint32))
|
||||
else:
|
||||
# Add subnet to subnets
|
||||
subnets.append(f'{line_list[0]}/{subnet_mask}')
|
||||
break
|
||||
return subnets
|
||||
|
||||
|
||||
def main():
|
||||
# argv[1] is the input file
|
||||
# argv[2] is the output file
|
||||
if len(sys.argv) != 3:
|
||||
print("Usage: convert_tsv.py <input_file> <output_file>")
|
||||
sys.exit(1)
|
||||
|
||||
# Open the input file
|
||||
try:
|
||||
input_file = open(sys.argv[1], 'r')
|
||||
except IOError:
|
||||
print("Error: Cannot open input file")
|
||||
sys.exit(1)
|
||||
|
||||
# Open the output file
|
||||
try:
|
||||
output_file = csv.writer(open(sys.argv[2], 'w'), delimiter=',',doublequote=True,quoting=csv.QUOTE_ALL)
|
||||
except IOError:
|
||||
print("Error: Cannot open output file")
|
||||
sys.exit(1)
|
||||
|
||||
# Read the input file line by line
|
||||
for line in input_file:
|
||||
# strip the newline at the end of the line
|
||||
line = line.rstrip('\n')
|
||||
# Split the line into a list by tabs
|
||||
line_list = line.split('\t')
|
||||
subnets = calculate_subnets(line_list)
|
||||
for subnet in subnets:
|
||||
#print(line_list[2] + '\t' + line_list[4] + '\t' + line_list[3] + '\t' + subnet)
|
||||
# CSV
|
||||
output_file.writerow([subnet, line_list[2], line_list[4], line_list[3]])
|
||||
|
||||
|
||||
# Write the output file
|
||||
#output_file.write(str(subnet) + '\t' + line_list[2] + '\t' + line_list[4] + '\t' + line_list[3] + '\n')
|
||||
|
||||
|
||||
input_file.close()
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Executable
+24
@@ -0,0 +1,24 @@
|
||||
#!/bin/sh
|
||||
# Retrieve the latest ASN database from github
|
||||
# curl https://raw.githubusercontent.com/ipverse/asn-ip/master/as/1234/ipv4-aggregated.txt
|
||||
|
||||
# Set the path to the directory where the ASN database will be stored to script_dir
|
||||
script_dir=$(dirname "$0")
|
||||
cd $script_dir
|
||||
curl https://raw.githubusercontent.com/ipverse/asn-info/master/as.csv > as-info.csv
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "ASN-Info database updated successfully"
|
||||
else
|
||||
echo "ASN-Info database update failed"
|
||||
fi
|
||||
|
||||
# https://iptoasn.com/
|
||||
# https://iptoasn.com/data/ip2asn-v4.tsv.gz
|
||||
curl https://iptoasn.com/data/ip2asn-v4.tsv.gz | gunzip > ip2asn-v4.tsv
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "ASN database updated successfully"
|
||||
else
|
||||
echo "ASN database update failed"
|
||||
fi
|
||||
|
||||
#range_start range_end AS_number country_code AS_description
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
# Using netflow
|
||||
|
||||
Our system support two modes of operation for netflow data collection:
|
||||
|
||||
1. Netflow conntrack: This mode uses the conntrack events generated by the kernel to collect netflow data. This mode is the default mode of operation, doesn't need any special rules in iptables.
|
||||
This mode primarily used for LE (Law Enforcement) purposes, to provide information on requests of ISF/Army, when they provide for you IP addresses + ports and ask for the username.
|
||||
It might be used with any other netflow collector, or https://gitlab.com/nuclearcat1/leconnlog
|
||||
|
||||
2. Netflow full: This mode uses the iptables rules to collect netflow data. This mode is more resource intensive, but provides more detailed information about the traffic. This mode is used for the billing or statistical purposes, when you need to know the amount of traffic consumed by each user. Requires special iptables rules to be added to the system.
|
||||
|
||||
## Supported standards
|
||||
|
||||
Our system supports the following netflow standards:
|
||||
|
||||
1. Netflow v5
|
||||
2. Netflow v9
|
||||
3. IPFIX (v10)
|
||||
|
||||
## Netflow conntrack
|
||||
|
||||
To enable netflow conntrack mode, you need to set the following options in the config file:
|
||||
|
||||
```
|
||||
insmod /tmp/ipt_NETFLOW.ko natevents=1 destination="10.168.101.3:2056" protocol=9
|
||||
sysctl -w net.netfilter.nf_conntrack_events=1
|
||||
```
|
||||
|
||||
Where:
|
||||
|
||||
- `natevents=1` - enables the netflow conntrack mode
|
||||
- `destination=` - specifies the destination for the netflow data. The format is `IP:PORT`
|
||||
- `protocol=9` - specifies the netflow protocol version. The supported values are 5, 9, 10
|
||||
|
||||
You can verify that the module is loaded by running the following command:
|
||||
|
||||
```
|
||||
lsmod | grep ipt_NETFLOW
|
||||
```
|
||||
|
||||
And verify statistics by running the following command:
|
||||
|
||||
```
|
||||
cat /proc/net/stat/ipt_netflow
|
||||
```
|
||||
|
||||
## Netflow full
|
||||
|
||||
To enable netflow full mode, you need to set the following options in the config file:
|
||||
|
||||
```
|
||||
insmod /tmp/ipt_NETFLOW.ko natevents=0 destination="10.168.101.3:2056" protocol=9
|
||||
```
|
||||
Then set in appropriate place traffic that should be accounted, for example
|
||||
```
|
||||
iptables -A FORWARD -i ppp+ -j NETFLOW
|
||||
iptables -A FORWARD -o ppp+ -j NETFLOW
|
||||
```
|
||||
|
||||
Where options are the same as for the netflow conntrack mode, as well as diagnostics commands.
|
||||
|
||||
## Netflow and nftables compatibility
|
||||
|
||||
The netflow module is partially compatible with nftables. You need to take special care about sequence of rules, as the netflow module doesn't support all the nftables features (for example flowtable), and nft rules must not override the iptables-nft rules.
|
||||
|
||||
|
||||
+105
@@ -1,3 +1,28 @@
|
||||
## How to get ASN report from sflowtools
|
||||
|
||||
1. Find your sflowdecode directory with docker-compose.yaml file, cd to it.
|
||||
2. Run `docker-compose pull` to pull the latest images.
|
||||
3. Run `docker-compose up -d` to start the containers.
|
||||
4. Run `docker-compose exec sflowdecode /bin/bash` to get the bash shell.
|
||||
5. `cd /sflowdecode`
|
||||
6. `ls -la` - fine appropriate sflow csv file to process. It has format like `20250225-06.csv`, time is GMT.
|
||||
7. Run for example: `traffstat --traffic 20250225-06.csv`
|
||||
8. You will get report in terminal, similar to this:
|
||||
```
|
||||
Total traffic 15424 Mbps
|
||||
Top 10 ASNs:
|
||||
AS20940 AVG_BW:4347 Mbps PERC:28%
|
||||
AS32934 AVG_BW:3255 Mbps PERC:21%
|
||||
AS15169 AVG_BW:3208 Mbps PERC:20%
|
||||
AS2906 AVG_BW:551 Mbps PERC:3%
|
||||
AS35197 AVG_BW:521 Mbps PERC:3%
|
||||
AS16509 AVG_BW:442 Mbps PERC:2%
|
||||
AS13335 AVG_BW:369 Mbps PERC:2%
|
||||
AS60068 AVG_BW:337 Mbps PERC:2%
|
||||
AS396982 AVG_BW:186 Mbps PERC:1%
|
||||
AS62041 AVG_BW:137 Mbps PERC:0%
|
||||
```
|
||||
|
||||
## Bonding
|
||||
Typical bonding configuration
|
||||
|
||||
@@ -23,6 +48,30 @@ ip link set dev eth7 up
|
||||
ip link set dev bond0 up
|
||||
```
|
||||
|
||||
### Cisco Nexus
|
||||
```
|
||||
interface port-channel4
|
||||
description To-GlobalOS
|
||||
switchport mode trunk
|
||||
switchport trunk native vlan 2701
|
||||
switchport trunk allowed vlan 2584-2591,2701,2798,2901-2902
|
||||
no negotiate auto
|
||||
|
||||
interface Ethernet1/21
|
||||
description To-GlobalOS
|
||||
switchport mode trunk
|
||||
switchport trunk native vlan 2701
|
||||
switchport trunk allowed vlan 2584-2591,2701,2798,2901-2902
|
||||
channel-group 4 mode active
|
||||
|
||||
interface Ethernet1/22
|
||||
description To-GlobalOS
|
||||
switchport mode trunk
|
||||
switchport trunk native vlan 2701
|
||||
switchport trunk allowed vlan 2584-2591,2701,2798,2901-2902
|
||||
channel-group 4 mode active
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Packetloss
|
||||
@@ -50,3 +99,59 @@ net.ipv4.icmp_ratelimit = 100000
|
||||
net.ipv4.icmp_msgs_burst=1000
|
||||
```
|
||||
|
||||
## Making "fake ping"
|
||||
docker-compose.yaml
|
||||
```
|
||||
version: "2.4"
|
||||
|
||||
services:
|
||||
fakeping:
|
||||
container_name: fakeping
|
||||
build: ./fakeping
|
||||
networks:
|
||||
mgmnt-net:
|
||||
ipv4_address: 10.0.252.19
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
restart: always
|
||||
|
||||
networks:
|
||||
mgmnt-net:
|
||||
name: mgmnt-net
|
||||
driver: macvlan
|
||||
driver_opts:
|
||||
parent: eno1
|
||||
ipam:
|
||||
config:
|
||||
- subnet: "10.0.252.0/24"
|
||||
gateway: "10.0.252.1"
|
||||
```
|
||||
|
||||
Dockerfile in fakeping directory
|
||||
```
|
||||
FROM ubuntu/kinetic:latest
|
||||
|
||||
RUN apt-get update && apt-get install -y iproute2 iputils-ping
|
||||
|
||||
ADD entrypoint2.sh /tmp
|
||||
WORKDIR /root
|
||||
ENTRYPOINT ["/tmp/entrypoint2.sh"]
|
||||
```
|
||||
|
||||
File entrypoint2.sh
|
||||
```
|
||||
#!/bin/sh
|
||||
tc qdisc del dev eth0 root
|
||||
tc qdisc add dev eth0 root handle 1:0 htb default 10
|
||||
tc class add dev eth0 parent 1:0 classid 1:10 htb rate 500Gbit ceil 500Mbit prio 0
|
||||
tc qdisc add dev eth0 parent 1:10 handle 10: pfifo limit 10000
|
||||
tc class add dev eth0 parent 1:0 classid 1:40 est 1sec 8sec htb rate 350Mbit ceil 350Mbit
|
||||
tc qdisc add dev eth0 parent 1:40 handle 41: netem delay 40ms 10ms limit 90000
|
||||
tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip protocol 0x1 0xff flowid 1:40
|
||||
# while loop
|
||||
while true
|
||||
do
|
||||
sleep 1
|
||||
done
|
||||
|
||||
```
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
#!/bin/sh
|
||||
cat /etc/accel-ppp.conf | grep "^interface=" | awk -F ',' '{ print $1; }' | awk -F '=' '{ print $2; }' | awk -F '.' '{ print $2; }' | while read VLANID; do
|
||||
echo Adding VLAN $VLANID
|
||||
ip link add link bond0 name bond0.$VLANID type vlan id $VLANID
|
||||
ip link set dev bond0.$VLANID up
|
||||
done
|
||||
|
||||
cat /etc/accel-ppp.conf | grep "^interface=" | awk -F '=' '{ print $2; }' | while read IFACCEL; do
|
||||
# add to accel live
|
||||
echo ${IFACCEL}
|
||||
accel-cmd "pppoe interface add ${IFACCEL}"
|
||||
done
|
||||
@@ -0,0 +1,15 @@
|
||||
#!/bin/sh
|
||||
# eth2, bond0, or whatever
|
||||
IFNAME=""
|
||||
|
||||
cat /etc/accel-ppp.conf | grep "^interface=" | awk -F ',' '{ print $1; }' | awk -F '=' '{ print $2; }' | awk -F '.' '{ print $2; }' | while read VLANID; do
|
||||
echo Adding VLAN $VLANID
|
||||
ip link add link ${IFNAME} name ${IFNAME}.$VLANID type vlan id $VLANID
|
||||
ip link set dev ${IFNAME}.$VLANID up
|
||||
done
|
||||
|
||||
cat /etc/accel-ppp.conf | grep "^interface=" | awk -F '=' '{ print $2; }' | while read IFACCEL; do
|
||||
# add to accel live
|
||||
echo Adding to ACCEL ${IFACCEL}
|
||||
accel-cmd "pppoe interface add ${IFACCEL}"
|
||||
done
|
||||
@@ -0,0 +1,90 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
# Ensure script is run as root
|
||||
if [[ $EUID -ne 0 ]]; then
|
||||
echo "This script must be run with sudo or as root."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 1) Remove any old Docker and Docker Compose packages
|
||||
echo "Removing old Docker packages..."
|
||||
set +e
|
||||
apt-get remove -y \
|
||||
docker docker-engine docker.io docker-ce docker-ce-cli \
|
||||
containerd runc \
|
||||
docker-compose
|
||||
set -e
|
||||
# Note: The above command may fail if Docker is not installed, which is fine.
|
||||
# The script will continue to install the latest version.
|
||||
|
||||
# 2) Update package index
|
||||
echo "Updating APT package index..."
|
||||
apt-get update -y
|
||||
|
||||
# 3) Install prerequisite packages
|
||||
echo "Installing prerequisites..."
|
||||
apt-get install -y \
|
||||
ca-certificates \
|
||||
curl \
|
||||
gnupg \
|
||||
lsb-release
|
||||
|
||||
# 4) Add Docker’s official GPG key
|
||||
echo "Adding Docker’s GPG key..."
|
||||
install -m 0755 -d /etc/apt/keyrings
|
||||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
|
||||
| gpg --dearmor -o /etc/apt/keyrings/docker.gpg
|
||||
chmod a+r /etc/apt/keyrings/docker.gpg
|
||||
|
||||
# ubuntu or debian?
|
||||
KIND=$(lsb_release -is | tr '[:upper:]' '[:lower:]')
|
||||
if [[ "$KIND" != "ubuntu" && "$KIND" != "debian" ]]; then
|
||||
echo "This script is intended for Ubuntu or Debian systems only."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 5) Set up the Docker APT repository
|
||||
echo \
|
||||
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
|
||||
https://download.docker.com/linux/$KIND \
|
||||
$(lsb_release -cs) stable" \
|
||||
| tee /etc/apt/sources.list.d/docker.list > /dev/null
|
||||
|
||||
# 6) Update the package index again
|
||||
echo "Updating APT package index with Docker repo..."
|
||||
apt-get update -y
|
||||
|
||||
# 7) Install the latest Docker Engine, CLI, containerd, and Compose plugin
|
||||
echo "Installing Docker Engine, CLI, containerd, and Docker Compose plugin..."
|
||||
apt-get install -y \
|
||||
docker-ce \
|
||||
docker-ce-cli \
|
||||
containerd.io \
|
||||
docker-compose-plugin
|
||||
|
||||
# 8) (Optional) Install legacy docker-compose binary
|
||||
# Uncomment below if you still need the standalone 'docker-compose' command
|
||||
# echo "Installing legacy docker-compose binary..."
|
||||
# COMPOSE_LATEST=$(curl -s https://api.github.com/repos/docker/compose/releases/latest \
|
||||
# | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/')
|
||||
# curl -L "https://github.com/docker/compose/releases/download/${COMPOSE_LATEST}/docker-compose-$(uname -s)-$(uname -m)" \
|
||||
# -o /usr/local/bin/docker-compose
|
||||
# chmod +x /usr/local/bin/docker-compose
|
||||
|
||||
# 9) Add current user to the 'docker' group (so you can run docker without sudo)
|
||||
USER_NAME=${SUDO_USER:-$(whoami)}
|
||||
echo "Adding user '$USER_NAME' to docker group..."
|
||||
usermod -aG docker "$USER_NAME"
|
||||
|
||||
# 10) Enable and start Docker
|
||||
echo "Enabling and starting Docker service..."
|
||||
systemctl enable docker
|
||||
systemctl start docker
|
||||
|
||||
# 11) Verify installation
|
||||
echo "Verifying Docker installation..."
|
||||
docker --version
|
||||
docker compose version || echo "Docker Compose plugin installed; use 'docker compose'."
|
||||
|
||||
echo "Done! Please log out and back in (or reboot) for group changes to take effect."
|
||||
Reference in New Issue
Block a user