forked from spinesystemspublic/documentation
Denys Fedoryshchenko
92 lines
3.5 KiB
Markdown
92 lines
3.5 KiB
Markdown
# Networking
|
|
|
|
## Bonding
|
|
|
|
Traffic is distributed between interfaces using LACP protocol. It is recommended to use LACP on switch side too.
|
|
It is also recommended to enable layer2+3 hashing for better traffic distribution.
|
|
|
|
```
|
|
ip link set bond0 type bond xmit_hash_policy layer3+4
|
|
```
|
|
|
|
|
|
## collectd
|
|
|
|
### Forwarding collectd metrics to another collectd instance
|
|
|
|
You need to add on destination collectd instance:
|
|
|
|
```
|
|
LoadPlugin network
|
|
|
|
<Plugin network>
|
|
<Listen "LISTENIP" "25826">
|
|
SecurityLevel Encrypt
|
|
AuthFile "/etc/collectd/passwd"
|
|
</Listen>
|
|
</Plugin>
|
|
```
|
|
In /etc/colllectd/passwd you need to add user and password for source collectd instance:
|
|
|
|
```
|
|
user:password
|
|
user2:password2
|
|
```
|
|
|
|
Source collectd instance:
|
|
|
|
```
|
|
LoadPlugin network
|
|
|
|
<Plugin network>
|
|
<Server "DESTINATIONIP" "25826">
|
|
SecurityLevel Encrypt
|
|
Username "user"
|
|
Password "password"
|
|
</Server>
|
|
</Plugin>
|
|
```
|
|
|
|
## Selecting right CPU, memory, disk, network card for your GlobalOS
|
|
|
|
### CPU
|
|
Find CPU available on cpubenchmark.net website and check "Average CPU mark".
|
|
|
|
For PPPoE: You can estimate at 7000 points it might take about 4000 users. More points - more users, but growth is not linear.
|
|
For example "Intel Xeon E3-1230 v3 @ 3.30GHz" has 6795 points.
|
|
|
|
For NAT: You can estimate at 7000 points it might take about 10Gbps.
|
|
|
|
Note, if you buy dual CPU server, you will not get 2x performance, it will be still significant boost, and you need to take into consideration, that each PCI-E is "local" for one CPU only, so you might need to install 2 network cards, on each CPU "side". Read about NUMA architecture and how it works.
|
|
|
|
### Memory
|
|
For PPPoE it is recommended to have at least 8GB of RAM at least. Recommended to have 16-32GB, but not more. You might take into consideration some servers with too much memory might have slower memory speed.
|
|
Do not use DDR3 memory/servers, it is too slow, unless you are ok with lower than expected number of users per server.
|
|
|
|
### Disk
|
|
For our system it is totally fine to use USB drive, but internal SSD is preferable. System use very little of storage resources, not I/O bound, we know some USB flashes that lasted more than 6 years under 24/7 operation. Always keep nearby USB storage for backup/restore purposes.
|
|
|
|
### Network card
|
|
We highly recommend Intel network cards. Several generations exist:
|
|
|
|
1) Well tested, but a bit old - X520, 82599 chipset (10Gbps), might have 1 or 2 SFP+ ports. Might accept any SFP+ modules.
|
|
2) X7** series - 10Gbps/40Gbps, might have 1 or 2 SFP+ ports. Need to check compatibility with SFP+ modules, need Intel branded or modified firmware modules. Very high performance, but not tested extensively for PPPoE task.
|
|
3) X9** series - 10Gbps/40Gbps, might have 1 or 2 SFP+ ports. Need to check compatibility with SFP+ modules, need Intel branded or modified firmware modules. Not tested yet.
|
|
|
|
Not recommended: Broadcom, Chelsio cards, they are known to start packetloss way before 10Gbps.
|
|
|
|
## Boot options
|
|
|
|
### EFI vs BIOS
|
|
We support both EFI and BIOS boot modes. But we highly recommend to use EFI boot mode, as BIOS boot known to fail from time to time. Please disable "Secure boot" in BIOS settings.
|
|
|
|
## BIOS settings
|
|
We recommend to do following tuning before deploying server to production:
|
|
|
|
* Disable "Secure boot" in BIOS settings.
|
|
* Change CPU power(powersaving) mode to "OS control" if available, and to "Performance" if not.
|
|
* TBD, i need to check BIOS of servers i have :)
|
|
|
|
## Other sources
|
|
|
|
* https://github.com/leandromoreira/linux-network-performance-parameters |