Networking on ousfifty's blog

How I Create Seamless Network Using OSPF on Routeros and Shellcrash on PVE

Thu, 29 Jan 2026 00:58:18 +0800

Note
Please be advised that this article is written for RouterOS 7.20.1 and ShellCrash 1.9.4beta6 on 2026-01-29, both operated as KVM instance in PVE. Contents might vary for different version of RouterOS and ShellCrash.

Introduction

Previously I used DHCP gateway nomination to let devices in my network use proxy. Although it successfully enables me to use proxy seamlessly, I can’t expand this experience to all devices, since sometimes ShellCrash may really “crashed” and would affect normal network traffic. I have heard OSPF feature for a long time, so recently I set up OSPF using RouterOS according to DeepRouter’s article. This article is to record my experience and share my way to enhance the robustness.

ShellCrash and Bird Configuration

Although it is fine to deploy ShellCrash and BIRD in separate instance, it is highly recommended to deploy in the same one.

ShellCrash installation process is ignored, just make sure you are using MIX in DNS configuration(press 2(feature setting) - 2(DNS Setting)).

Check IPv4 and IPv6 forward configuration

run sudo sysctl -p and check if the following are properly configured:

net.ipv4.ip_forward=1
net.ipv6.conf.all.forwarding=1

If not, edit /etc/sysctl.conf.

BIRD

We will use !chnroutes to distinguish chn ips.

Make sure you doesn’t enable/start bird service after installation. There are several things to do.

For Debian:

sudo apt update
sudo apt install bird2 make

bird.conf

Edit /etc/bird/bird.conf and change as below.

make sure:

router id is your ShellCrash/BIRD instance IP
The interface in both ospf v2 and v3 is set to the interface this instance used to connect to the gateway(the LAN port). You may check using ip a or ip link, and find the one OTHER THAN lo.

log syslog all;

router id 192.168.114.51; #This line change to instance IP

protocol device {
 scan time 60;
}

protocol kernel {
 ipv4 {
 import none;
 export none;
 };
}
protocol kernel {
 ipv6 {
 import none;
 export none;
 };
}

protocol static {
 ipv4;
 include "/etc/bird/routes4.conf";
}
protocol static {
 ipv6;
 include "/etc/bird/routes6.conf";
}

protocol ospf v2 {
 ipv4 {
 export all;
 };
 area 0.0.0.0 {
 interface "ens18" { # Change this to your networking interface
 type pointopoint;
 };
 };
}

protocol ospf v3 {
 ipv6 {
 export all;
 };
 area 0.0.0.0 {
 interface "ens18" { # Change this to your networking interface
 type pointopoint;
 };
 };
}

Generating routes

git clone https://github.com/ous50/nchnroutes.git
cd nchnroutes
make

If you have trouble connecting to github.com, use a mirror instead:

git clone https://hk.gh-proxy.org/https://github.com/ous50/nchnroutes.git
cd nchnroutes
make

You are pleased to continue if the following output is shown:

python3 produce.py
sudo mv routes4.conf /etc/bird/routes4.conf
sudo mv routes6.conf /etc/bird/routes6.conf
sudo birdc configure
BIRD 2.0.12 ready.
Reading configuration from /etc/bird/bird.conf
Reconfigured

RouterOS Configuration

You need to set up bypassing routing table first to prevent looping instantly after OSPF neighbor is found.

Bypass ShellCrash looping and setting up exceptions

We are going to make another routing table to bypass the traffic from ShellCrash to prevent loop.

/routing table
add disabled=no fib name=bypass-shellcrash
/routing rule
add action=lookup-only-in-table comment="Bypass Shellcrash Gateway using Routing Mark" disabled=no routing-mark=bypass-shellcrash table=bypass-shellcrash

Migrate your routes from main to bypass-shellcrash:

/ip route
add disabled=no distance=1 dst-address=<Your LAN with CIDR> gateway=lan routing-table=bypass-shellcrash scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=<Your Internet Gateway> routing-table=bypass-shellcrash scope=30 suppress-hw-offload=no target-scope=10

Firewall tagging

Make sure:

Change the dst-address to your LAN, preserving the !.
Change the src-mac-address to the ShellCrash networking interface port MAC address(check using ip link in shellcrash instance or directly from PVE)

/ip firewall mangle
add action=mark-routing chain=prerouting comment=bypass-shellcrash-gateway dst-address=!192.168.0.0/23 new-routing-mark=bypass-shellcrash src-mac-address=11:45:14:19:19:81

OSPF config

By default you do not need to change anything in this script.

/routing ospf instance
add disabled=no name=shellcrash
add disabled=no name=shellcrash-v6 version=3
/routing ospf area
add disabled=no instance=shellcrash name=ospf-area-v4
add disabled=no instance=shellcrash-v6 name=ospf-area-v6
/routing ospf interface-template
add area=ospf-area-v4 cost=10 disabled=no priority=32 type=ptp
add area=ospf-area-v6 cost=10 disabled=no priority=32 type=ptp

Tricks

Robust DNS setting

Due to the fake-ip used by ShellCrash, you should set ShellCrash’s DNS as an upstream.

In this case we assume nominating RouterOS’s DNS to clients and use the following RouterOS script to update upstream DNS source to enhance the robustness, in case of ShellCrash failed.

:global ospfInstance "shellsrash"
:global ospfDNS "1.1.4.5" # Your ShellCrash instance IP
:global publicDNS "223.5.5.5" # Change this DNS for your direct internet access

# (true / false), turn debug off after testing or your logs have abundant annoying debug messages.
:local DEBUG false

# marking the final status default value is false (down)
:local isOspfActive false

# Trying to find the neighbor
:local neighborIds [/routing ospf neighbor find instance=$ospfInstance]

# check if neighbor is found
:if ([:len $neighborIds] > 0) do={
 # Note: In this case we will only use one instance.
 # If you have multiple instance, you might need to traversal and find the instance.
 :local neighborState [/routing ospf neighbor get ($neighborIds->0) state]
 
 :if ($DEBUG) do={:log info "[DEBUG] Neighbor found! State: $neighborState"}
 
 # Only consider ospf active when state is "Full". Remember, not "full". This state might vary on different RouterOS version
 :if ($neighborState = "Full") do={
 :set isOspfActive true
 }
} else={
 # if length is 0, this neighbor is not found. check bird configuration/status.
 :if ($DEBUG) do={:log info "[DEBUG] No OSPF neighbor found. Logic assumes DOWN."}
 :set isOspfActive false
}

# change DNS according to the variable isOspfActive 
:local currentDNS [/ip dns get servers]

:if ($isOspfActive) do={
 # OSPF Up
 :if ($DEBUG) do={:log info "[DEBUG] Decision: OSPF is UP."}
 
 :if ($currentDNS != $ospfDNS) do={
 :log warning "OSPF Shellcrash Up: Switching to OSPF DNS ($ospfDNS)."
 /ip dns set servers=$ospfDNS
 /ip dns cache flush
 } else={
 :if ($DEBUG) do={:log info "[DEBUG] DNS is already correct (OSPF)."}
 }
} else={
 # OSPF Down (or missing neighbor)
 :if ($DEBUG) do={:log info "[DEBUG] Decision: OSPF is DOWN/MISSING."}
 
 :if ($currentDNS != $publicDNS) do={
 :log warning "OSPF Shellcrash Down: Switching to Public DNS ($publicDNS)."
 /ip dns set servers=$publicDNS
 /ip dns cache flush
 } else={
 :if ($DEBUG) do={:log info "[DEBUG] DNS is already correct (Public)."}
 }
}

Switch off bird when shellcrash is crashed

We will use a bash script to actively check if shellcrash is alive. If the API is unreachable, it will stop the Bird service to withdraw OSPF routes.

Copy and paste the following block into your PVE Shell (logged in as the clash user, or a user with sudo privileges):

# 1. Create the watchdog script
# Note: We use 'EOF' (quoted) to prevent variable expansion during creation.
sudo tee /home/clash/check_clash.sh > /dev/null << 'EOF'
#!/bin/bash

# Configuration
API_URL="http://127.0.0.1:9999/"
LOG_FILE="/home/clash/clash_watchdog.log"
DEBUG=false
INTERVAL=3

# Init state (0=unknown, 1=normal, 2=crashed)
LAST_STATE=0

while true; do
 # Check ShellCrash API
 HTTP_CODE=$(curl -s -m 1 -o /dev/null -w "%{http_code}" "$API_URL")
 
 if [ "$DEBUG" = "true" ]; then 
 echo "[DEBUG] Dashboard code: $HTTP_CODE"
 fi

 if [ "$HTTP_CODE" == "200" ]; then
 # ShellCrash is ALIVE
 if [ "$LAST_STATE" != "1" ]; then
 DATE=$(date "+%Y-%m-%d %H:%M:%S")
 
 # Check if Bird is running using systemctl's return code (more reliable)
 sudo systemctl is-active --quiet bird
 BIRD_EXIT_CODE=$?
 
 if [ $BIRD_EXIT_CODE -ne 0 ]; then
 if [ "$DEBUG" = "true" ]; then 
 echo "ShellCrash detected. Starting Bird..."
 fi
 echo "[$DATE] ShellCrash recovered/detected. Starting Bird..." >> $LOG_FILE
 sudo systemctl start bird
 fi
 
 LAST_STATE=1
 fi
 else
 # ShellCrash is DEAD
 if [ "$LAST_STATE" != "2" ]; then
 DATE=$(date "+%Y-%m-%d %H:%M:%S")
 if [ "$DEBUG" = "true" ]; then 
 echo "[$DATE] Alert! ShellCrash died (Code: $HTTP_CODE). Stopping Bird!"
 fi
 echo "[$DATE] Alert! ShellCrash died (Code: $HTTP_CODE). Stopping Bird!" >> $LOG_FILE
 
 sudo systemctl stop bird
 LAST_STATE=2
 fi
 fi

 sleep $INTERVAL
done
EOF

# 2. Make it executable
sudo chmod +x /home/clash/check_clash.sh

# 3. Create Systemd Service
sudo tee /etc/systemd/system/clash-watchdog.service > /dev/null << EOF
[Unit]
Description=ShellCrash Watchdog for Bird OSPF
After=network.target

[Service]
User=clash
Group=clash
ExecStart=/home/clash/check_clash.sh
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target
EOF

# 4. Enable and Start
sudo systemctl daemon-reload
sudo systemctl enable --now clash-watchdog.service
sudo systemctl status clash-watchdog.service

Install Easytier on Pve

Tue, 26 Aug 2025 10:30:16 +0800

If your internet provider interferes with the GitHub, you can use the following commands to install EasyTier:

apt update -y sudo unzip openssl 
wget -O /tmp/easytier.sh "https://ghfast.top/https://raw.githubusercontent.com/EasyTier/EasyTier/main/script/install.sh" && sudo bash /tmp/easytier.sh install --gh-proxy https://ghfast.top/
systemctl disable --now easytier@default.service

Check the latest proxy here.

Otherwise just simply use the default GitHub URL.

apt update -y sudo unzip openssl 
wget -O /tmp/easytier.sh "https://raw.githubusercontent.com/EasyTier/EasyTier/main/script/install.sh" && sudo bash /tmp/easytier.sh install
systemctl disable --now easytier@default.service

Should you wish to use easytier dashboard to manage, you can enable the service with the following command:

export USER=<your_username>
echo "[Unit]
Description=EasyTier Service
After=network.target syslog.target
Wants=network.target

[Service]
Type=simple
ExecStart=/usr/sbin/easytier-core -w $USER

[Install]
WantedBy=multi-user.target
" | sudo tee /etc/systemd/system/easytier.service

systemctl enable --now easytier.service

Then you are able to access the EasyTier dashboard at https://easytier.cn/web.

Advanced FnOS Configuration and Usage Tips

Fri, 08 Aug 2025 11:26:29 +0800

This article focuses on optimizing FnOS. For instructions on installing FnOS on PVE or a physical machine, please see the previous article.

Network Optimization

The following optional steps can improve network throughput and privacy.

Execute this script to enable the BBR congestion control algorithm:

cat > /etc/sysctl.d/99-enable-bbr.conf <<EOF
net.core.default_qdisc=fq
net.ipv4.tcp_congestion_control=bbr
EOF

Apply the BBR configuration without rebooting:
```
sysctl -p /etc/sysctl.d/99-enable-bbr.conf
```

Execute this script to enable modern IPv6 privacy address standards (RFC 7217 and RFC 4941):

cat > /etc/sysctl.d/99-enable-private-ipv6.conf <<EOF
net.ipv6.conf.all.use_tempaddr = 2
net.ipv6.conf.default.use_tempaddr = 2
net.ipv6.conf.all.addr_gen_mode=1
net.ipv6.conf.default.addr_gen_mode=1
EOF

Apply the sysctl configuration:
```
sysctl -p /etc/sysctl.d/99-enable-private-ipv6.conf
```
🚨 Warning: Do not use the “EUI-64” option in the FnOS web interface. Doing so will negate these privacy enhancements by exposing the device’s MAC address in its IPv6 address.
Applying Network Changes

To activate the new IPv6 address settings, the network interface must be reset. This can be done with nmcli without a full system reboot.

🚨 Important: Execute these commands from the PVE web console (Xterm.js), as running them over SSH will cause a disconnection due to the IPv6 address change and may not be recoverable.

Step 1: Identify the Connection Name List all active connections to find the name of your primary interface.
```
nmcli connection show
```
The output will list available connections. Note the name of your Ethernet connection, which is often Wired connection 1.

Step 2: Reset the Connection Use the identified name to restart the network interface:
```
nmcli connection down "Wired connection 1" && nmcli connection up "Wired connection 1"
```
The network interface will restart. You can confirm the new IPv6 address configuration using ip a or in the Summary page for the VM in the PVE web console.

Enabling IPv6 Router Advertisements (RA) for Hosted Virtual Machines

To allow VMs inside it to also acquire IPv6 addresses via SLAAC, its RA settings must be changed:

cat > /etc/sysctl.d/99-virtual-machine.conf <<EOF
net.ipv6.conf.all.accept_ra = 2
net.ipv6.conf.default.accept_ra = 2
EOF

sysctl -p /etc/sysctl.d/99-virtual-machine.conf

Enabling Memory and CPU Hot-plug as a PVE/QEMU Guest

Modify /etc/default/grub to enable memory hot-plug:

- GRUB_CMDLINE_LINUX="modprobe.blacklist=pcspkr"
+ GRUB_CMDLINE_LINUX="modprobe.blacklist=pcspkr memhp_default_state=online"

Create a UDEV rule to enable CPU hot-plug:

cat > /lib/udev/rules.d/80-hotplug-cpu.rules <<EOF
SUBSYSTEM=="cpu", ACTION=="add", TEST=="online", ATTR{online}=="0", ATTR{online}="1"
EOF

Shut down the virtual machine.
In the PVE web console, on the Processor page, check “Enable NUMA”.
In the PVE web console, on the Options page, check Memory and CPU in the “Hotplug” options list.

Installing FnOS on Proxmox VE (PVE)

Sun, 03 Aug 2025 10:12:00 +0800

Introduction

This document addresses common issues encountered on the process of installing FnOS in a PVE environment, such as a locked system and an unresponsive PVE web console. By following these steps, you will achieve a fully accessible, secure, and optimized FnOS virtual machine.

If you have already completed the installation and want to see more optimization tips, please see this article.

Pre-Installation

FnOS ISO: Visit https://www.fnnas.com/ and obtain it via “Direct Download”.

It is recommended that the FnOS VM has at least 3GB of RAM and 2 cores.

Follow the official guide to install the system.

Post-Installation Configuration

On the first boot, two primary issues prevent access to the new FnOS instance:

Login Disabled: The system console does not accept any credentials (including root) until the web-based initial setup is complete. This is problematic for remote machines that may only have IPv6 access.
Unresponsive PVE Console: The Xterm.js serial console in the PVE web UI does not work by default.

The following sections provide solutions to these problems.

1. Gaining Initial Root Access

First, we must bypass the standard boot sequence to set a root password.

In the PVE console, restart the VM. At the GRUB boot menu, press e to edit the boot parameters.
Find the line that begins with linux. It will look similar to this:
```
linux /boot/vmlinuz-6.12.18-trim root=UUID=... ro quiet
```
At the end of this line, add single init=/bin/bash. This parameter boots the system directly into a root shell.
Press Ctrl+X or F10 to boot.
In the root shell, execute the passwd command to set a new password for the root user.
Once done, reboot the VM. The boot parameter change is temporary and will not persist.

2. Enabling Permanent Console Access and Functionality

With root access established, permanently modify the GRUB configuration to enable the PVE serial console and other features.

Log in as root with the newly set password.
Edit the GRUB configuration file: nano /etc/default/grub.

Modify the GRUB_CMDLINE_LINUX_DEFAULT line to include serial console and optional nested virtualization IOMMU support.

For Intel CPUs:

- GRUB_CMDLINE_LINUX_DEFAULT="quiet"
+ GRUB_CMDLINE_LINUX_DEFAULT="quiet console=tty0 console=ttyS0,115200 intel_iommu=on iommu=pt"

For AMD CPUs:

- GRUB_CMDLINE_LINUX_DEFAULT="quiet"
+ GRUB_CMDLINE_LINUX_DEFAULT="quiet console=tty0 console=ttyS0,115200 amd_iommu=on iommu=pt"

Note: console=ttyS0,115200 is the key parameter for PVE Xterm.js functionality.

Save the changes and exit the editor.
Apply the new configuration by executing:
```
update-grub
```
Reboot the VM. The PVE web console should now be fully functional.

3. Installing the QEMU Guest Agent

For better integration with PVE, such as displaying network information in the VM summary, we need to install the qemu-guest-agent.

```shell
apt update && apt -y install qemu-guest-agent
systemctl enable --now qemu-guest-agent
```

Conclusion

You have successfully configured an FnOS instance on Proxmox VE. The system is now fully accessible and properly integrated with the PVE host. Further configurations and optimizations will be presented in a new article.