Firmware architecture¶
This page is for developers who want to understand how Ethernet Switch OS works before they change it. It follows a configuration change from the user down to the switch chip, and shows where each part lives in the source.
The idea in one paragraph¶
The switch is a small Linux computer whose network ports happen to be
connected to a switch chip. Linux sees every front port as an ordinary
network interface (lan1, lan2, ...), and the switch is configured the
way any Linux bridge is configured: a VLAN-aware bridge, bridge VLAN
entries, VLAN interfaces with IP addresses. The kernel's switchdev and
DSA frameworks pass that configuration on to the switch chip, which then
forwards frames in hardware. There is no switch ASIC SDK and no
proprietary driver anywhere in the firmware.
Built on clixon¶
Ethernet Switch OS is based massively on clixon, an open-source configuration management framework driven by YANG models. Clixon provides the datastores, YANG validation, transactions, and the RESTCONF server, CLI and SNMP frontends. Ethernet Switch OS adds only the switch-specific part: a backend plugin that turns a committed YANG configuration into those kernel settings. The plugin is written in Rust and lives in clixon-switch-rs.
Many thanks to the clixon developers for this foundation. Without their work this project would not exist.
Overview¶
flowchart TB
browser["Web browser<br/>(web UI, plain JavaScript)"]
script["Scripts, curl"]
ssh["SSH client"]
nms["SNMP manager"]
neighbors["Neighboring devices"]
subgraph user["Userspace"]
lighttpd["lighttpd<br/>HTTPS port 443, login"]
restconf["clixon_restconf<br/>127.0.0.1 port 80"]
swupdate["SWUpdate<br/>127.0.0.1 port 8080"]
cli["clixon_cli<br/>login shell of the admin account"]
snmpd["snmpd (net-snmp)<br/>UDP port 161"]
clixon_snmp["clixon_snmp<br/>AgentX subagent"]
backend["clixon_backend<br/>datastores, YANG validation, transactions"]
plugin["clixon-switch plugin (Rust)<br/>validate, plan, reconcile"]
helpers["mstpd, udhcpc"]
lldpd["lldpd<br/>LLDP"]
end
subgraph kernel["Linux kernel"]
net["Bridge br-lan, bridge VLANs, 802.1Q links,<br/>IPv4 addresses, FDB, per-VLAN STP states"]
switchdev["switchdev / DSA"]
driver["DSA switch driver"]
end
asic["Switch chip"]
browser --> lighttpd
script --> lighttpd
lighttpd --> restconf
lighttpd -- "/update/" --> swupdate
ssh --> cli
nms --> snmpd --> clixon_snmp
snmpd -- "AgentX, LLDP-MIB" --> lldpd
neighbors <-->|LLDP frames| lldpd
restconf -- "UNIX socket" --> backend
cli -- "UNIX socket" --> backend
clixon_snmp -- "UNIX socket" --> backend
backend -- "transaction callbacks" --> plugin
plugin -- "rtnetlink" --> net
plugin -- "starts, configures" --> helpers
plugin -- "starts, lldpcli" --> lldpd
helpers -- "netlink" --> net
net --> switchdev --> driver -- "registers (MMIO, MDIO, SPI, SMI)" --> asic
| Layer | What it is | Source |
|---|---|---|
| Web UI | Static HTML, CSS and JavaScript, no build step. Talks to the switch only through RESTCONF. | meta-ethernet-switch-os recipes-webui/ |
| HTTPS front end | lighttpd: TLS, the password check, the web UI's files, and forwarding to RESTCONF and SWUpdate. See Access and security. | meta-ethernet-switch-os recipes-extended/lighttpd/ |
| RESTCONF | clixon_restconf, clixon's native HTTP/1 server, on 127.0.0.1 only. |
clixon, recipe in meta-ethernet-switch-os recipes-clixon/ |
| CLI | clixon_cli, generated from the YANG models (clixon's autocli), plus password and factory-reset from a small C plugin. It is the login shell of the admin account. |
clixon; CLI spec and plugin in clixon-switch-rs clixon/ |
| SNMP | net-snmp's snmpd and clixon_snmp as its AgentX subagent; lldpd as a second one for LLDP-MIB. Read-only. |
net-snmp, clixon |
| LLDP | lldpd, started and configured by the plugin. |
lldpd, trimmed in meta-ethernet-switch-os recipes-networking/lldpd/ |
| Configuration backend | clixon_backend: holds the datastores, validates against YANG, runs transactions, loads the plugin. |
clixon |
| Backend plugin | clixon-switch, a Rust shared library. Validates each commit and brings the kernel in line with it. Runs the RPCs set-password (which also creates the admin account) and factory-reset. |
clixon-switch-rs |
| Linux networking | Bridge, VLANs, addresses, spanning tree states. Configured over rtnetlink. | Linux kernel |
| Hardware offload | switchdev and DSA, and the switch chip's DSA driver. | Linux kernel, BSP layer |
Hardware support: switchdev and DSA¶
No SDK, no proprietary driver¶
Commercial switch firmware usually drives the switch chip through the chip vendor's SDK: a large, closed or restrictively licensed library that programs the chip from userspace, next to (or instead of) the Linux network stack. Ethernet Switch OS does not do that. It relies only on two Linux kernel frameworks:
- switchdev is the kernel's generic interface for offloading bridge and VLAN configuration to hardware. When the bridge adds a VLAN to a port, learns an address or changes a port's spanning tree state, switchdev hands the change to the driver, which programs the chip.
- DSA (Distributed
Switch Architecture) is the switchdev framework for switch chips that
hang off an Ethernet interface of the CPU. That interface is the
conduit (
eth0); every front port becomes a user port, a network interface of its own (lan1...lanN). Frames between the CPU and the chip carry a small tag that says which port they belong to.
So the firmware never talks to the switch chip directly. The plugin configures a normal Linux bridge, and the kernel driver decides what the hardware does with it. The same userspace binary runs on every board.
Which switches can be supported¶
In principle, every switch chip that has a switchdev or DSA driver in the
Linux kernel can run Ethernet Switch OS. The mainline kernel has DSA
drivers for many chips, for example Marvell (mv88e6xxx), Broadcom
(b53), MediaTek (mt7530), Microchip (ksz, lan9303), NXP (sja1105,
felix), Qualcomm (qca8k) and Realtek (rtl8365mb, rtl8366rb), and
switchdev drivers for larger chips such as Marvell Prestera, Mellanox
Spectrum (mlxsw) and Microchip Sparx5 and LAN966x. OpenWrt maintains more
out-of-tree drivers, like the one for the Realtek RTL83xx family used here.
"In principle", because the driver has to do more than create the ports. What the firmware expects from it:
| Feature | Needed for | Without it |
|---|---|---|
Bridge offload (port_bridge_join/leave) |
Forwarding in hardware | Every frame between two ports goes through the CPU. |
VLAN filtering offload (port_vlan_add/del) |
VLANs, 802.1Q access and trunk ports | VLANs are forwarded in software only, if at all. |
| FDB offload | Address learning and aging in hardware; the forwarding database in SNMP | Flooding or software forwarding. |
Port STP state (port_stp_state_set) |
STP and RSTP | Spanning tree cannot block ports in hardware. |
MST offload (vlan_msti_set, port_mst_state_set) |
MSTP with several instances | Only the CIST is possible. |
The Linux DSA Feature List Explorer shows which DSA driver supports which of these features, and which chips each driver covers.
The Raspberry Pi switch shows why this matters: the mainline rtl8365mb
driver in Linux 6.18 only creates the ports. Without the bridge, VLAN and
FDB offload that the BSP backports from newer kernels, every frame between
two front ports crosses the ENC28J60 SPI link at about 5 Mbit/s.
A new switch therefore needs a BSP layer with a kernel whose driver has
these features, and a device tree that names the front ports lan1,
lan2, ... See Bringing up new hardware.
The current boards¶
| Board | Switch chip | Kernel driver | CPU connection |
|---|---|---|---|
| Zyxel GS1900-8, RTL8382MI test switch | Realtek RTL8380M / RTL8382M (switch and MIPS CPU in one SoC) | rtl83xx DSA driver and rtl838x_eth from OpenWrt, patched into Linux 6.18 by meta-rtl83xx-bsp |
internal |
| Raspberry Pi 4-port managed switch HAT | Realtek RTL8367S | mainline rtl8365mb DSA driver, managed over bit-banged SMI, plus OpenWrt's backports for bridge, VLAN and FDB offload |
ENC28J60 on SPI as conduit |
| QEMU x86-64 switch | none | none: the eight virtio-net ports are plain interfaces, the bridge forwards in software | — |
The QEMU switch is the proof that the plugin does not depend on DSA: it
treats DSA user ports as switch ports by default, and
CLIXON_SWITCH_PORTS="lan1 ... lan8" in /etc/default/clixon-backend
names other interfaces instead. The same mechanism would serve a pure
switchdev driver whose ports are not DSA ports.
Minimum hardware¶
The firmware is built for small switches with at least 16 MB of flash and 128 MB of RAM. Every component is chosen with that in mind: musl and busybox instead of glibc and systemd, a compressed read-only root filesystem, and no services beyond the ones listed on this page.
The numbers below are those of the Zyxel GS1900-8, with Linux 6.18.48.
Flash¶
The 16 MiB SPI-NOR flash is divided into five partitions:
| Partition | Size | Content |
|---|---|---|
u-boot |
256 KiB | The stock Zyxel bootloader. Never written. |
u-boot-env |
64 KiB | Bootloader environment |
u-boot-env2 |
64 KiB | Second bootloader environment (bootpartition) |
data |
2 MiB | JFFS2, the writable layer on top of the root filesystem: the saved configuration, the admin password, SSH host keys, the HTTPS certificate. Kept across firmware updates, erased by a factory reset. |
firmware |
13.6 MiB | The firmware itself, see below. Rewritten by a firmware update. |
| Total | 16 MiB |
The firmware in the firmware partition:
| Part | Size |
|---|---|
| Kernel with device tree (uImage, padded to 64 KiB) | 2.2 MiB |
| Root filesystem (squashfs, xz compressed) | 5.4 MiB |
| Used | 7.6 MiB |
| Free | 6.0 MiB |
A firmware update file (.swu) is the content of the firmware partition,
so it has the same size.
RAM¶
Of the 128 MiB, the kernel keeps about 8 MiB for itself (its code and data,
and the table that describes every page of memory) and reports 119.6 MiB
as usable. Right after boot, free shows:
| Size | What it is | |
|---|---|---|
| Used | 30.9 MiB | Kernel allocations and all processes |
| Buffers and cache | 22.9 MiB | Mostly pages of the root filesystem, already decompressed |
| Available | 64.5 MiB | What new processes can get, including cache that can be dropped |
Normal operation needs only about 31 MiB. The minimum is set by the moments when much more is needed at once:
- Firmware update. There is only one firmware slot, so SWUpdate must
receive the whole update file and check its SHA-256 hash before it
erases the
firmwarepartition. The file is kept in/tmp, which is RAM: 7.6 MiB. SWUpdate, its libraries and its web page are also copied to RAM first, because the update overwrites the squashfs they would otherwise be loaded from. - First install and recovery. The TFTP boot image runs the whole system
from RAM: its root filesystem unpacks to 19.0 MiB, and the factory
update file it then receives is another 9.6 MiB in
/tmp, on top of the same services as in normal operation. - Features and logins. LLDP (
lldpd) runs unless turned off. Spanning tree (mstpd), SNMP (snmpd,clixon_snmp) and the DHCP client run only when configured, and every SSH login to the CLI starts anotherclixon_cli. Every clixon process loads the YANG models into memory, and the OpenConfig models are large. - Cache. The programs run from a xz-compressed squashfs. When memory gets short, the kernel drops their pages from the cache and has to read and decompress them again from flash the next time they are used, which is slow on this CPU.
With 64 MiB, about 56 MiB would be usable. The first install alone needs roughly 19 + 9.6 + 31 = 60 MiB, so it does not fit. In normal operation about 25 MiB would remain for the cache, the optional services, CLI sessions and a firmware update, which is not enough headroom for all of them.
What happens on a configuration change¶
A change is a clixon transaction, for example a RESTCONF request that puts
lan3 into VLAN 20. The CLI's commit and the web UI take the same path.
The numbered steps are explained below the diagram.
%%{init: {"flowchart": {"wrappingWidth": 320}}}%%
flowchart TB
req(["<b>Web UI, curl or CLI</b><br/>PATCH /restconf/data/... or commit"])
subgraph clixon["clixon_restconf and clixon_backend"]
direction TB
edit["Write the change into<br/>the candidate datastore"]
yang["Validate against YANG:<br/>types, ranges, must, leafref"]
edit --> yang
end
subgraph plugin["clixon-switch plugin"]
direction TB
validate["<b>1 Validate</b><br/>JSON to DesiredState, reject<br/>what is not implemented"]
observe["<b>2 Observe</b><br/>kernel to ActualState"]
plan["<b>3 Plan</b><br/>operations, actual to desired"]
apply["<b>4 Apply</b><br/>operations over rtnetlink"]
match{"Kernel<br/>matches?"}
helpers["<b>5 Helpers</b><br/>mstpd, udhcpc, snmpd, lldpd"]
validate --> observe --> plan --> apply --> match
match -- no --> observe
match -- yes --> helpers
end
subgraph kernel["Linux kernel"]
direction TB
bridge["br-lan, bridge VLANs,<br/>VLAN links, addresses"]
asic["switchdev, DSA driver:<br/>program the switch chip"]
bridge --> asic
end
ok(["<b>Commit done</b><br/>HTTP 204 No Content"])
err(["<b>Commit refused</b><br/>kernel reverted, HTTP 400 with the error"])
req --> clixon
yang --> validate
apply -.-> bridge
helpers --> ok
clixon -- invalid --> err
plugin -- any step fails --> err
The steps in the plugin:
- Validate.
switch-modelparses the target configuration, which clixon passes as RFC 7951 JSON, into aDesiredState. It checks what YANG cannot express (a port's VLANs must be declared, only one DHCP client, 802.1D timer rules, ...) and rejects everything the firmware does not implement, such as MTU, IPv6 or subinterfaces. The data models describe much more than the switch can do, and a setting that is stored but has no effect would be a lie. - Observe.
switch-netreads the kernel's current state over rtnetlink into anActualState: links and their kinds, bridge membership, bridge VLAN entries, addresses. - Plan. A pure function computes the list of operations from the actual to the desired state, in an order the kernel accepts and without interrupting traffic: a port's new PVID is added before its old one is removed, a new management address before the old one goes.
- Apply and repeat. The operations are sent over rtnetlink, then the plugin observes again, until the kernel matches. Some operations have side effects that are easier to observe than to predict.
- Helpers. The plugin starts, reconfigures or stops
mstpd(spanning tree),udhcpc(DHCP client),snmpdwithclixon_snmp, andlldpd(LLDP), as the configuration asks for them. They are children ofclixon_backend. A failinglldpdis only logged, because LLDP is on by default and must not keep the startup configuration from being applied.
If the commit fails, clixon calls trans_revert, and the plugin reconciles
the kernel back to the previous configuration the same way.
The plugin is declarative: it does not translate a diff into commands
but always brings the whole kernel state it owns (the bridge br-lan, the
switch ports and the VLAN interfaces on the bridge) in line with the whole
configuration. Anything else in the kernel is left alone. The mapping from
configuration to kernel objects is in the
clixon-switch-rs README.
State data (counters, link state, DHCP lease, spanning tree roles,
firmware version) takes the reverse path: clixon calls the plugin's
statedata callback, which reads the kernel over netlink, asks mstpd with
mstpctl and lldpd with lldpcli (LLDP neighbors), and reads /proc.
Data model¶
The configuration is described by YANG. It uses OpenConfig where OpenConfig
has a model (openconfig-interfaces, openconfig-vlan,
openconfig-if-ip, openconfig-spanning-tree, openconfig-lldp),
ietf-snmp for SNMP, and
the project's own module clixon-switch for what neither covers: the VLAN
database, port-based VLANs, the MSTP CIST, system contact and location, and
state such as the DHCP lease and /system/state. The MIBs served over SNMP
are translated to YANG as well, so clixon_snmp can serve them from the
same backend.
All YANG files are in yang/ of clixon-switch-rs; yang/vendor/ holds
the imported OpenConfig and IETF modules. What the switch accepts is listed
under YANG models.
Datastores and persistence¶
clixon keeps the configuration in XML datastores:
| Datastore | Where | Content |
|---|---|---|
candidate_db |
/var/run/clixon-switch/ (RAM) |
The CLI's working copy until commit. |
running_db |
/var/run/clixon-switch/ (RAM) |
The active configuration. |
startup_db |
link to /var/lib/clixon/clixon-switch/startup_db (flash) |
The saved configuration, loaded at boot. Kept across firmware updates. |
failsafe_db |
/var/run/clixon-switch/ (RAM) |
The factory default. clixon falls back to it when startup_db does not commit. |
A commit changes running only. save in the CLI, "Save configuration" in
the web UI or a RESTCONF copy-config from running to startup makes it
permanent. Only startup_db is on flash, because clixon rewrites the other
datastores on every edit. A factory reset erases it with
everything else on the data partition.
Factory reset¶
A factory reset puts the switch back into the state of a fresh install: the
factory configuration at 192.168.1.1, no admin account (the
first-login setup runs again), new SSH host keys and a
new HTTPS certificate. The firmware stays.
Everything the switch writes lands in the overlay's upper layer on the
data partition, so the reset erases that partition. It cannot be erased
while the overlay on it is in use, so the reset takes two steps, split by a
reboot:
trigger ─► ethernet-switch-os-factory-reset ─► marker /overlay/.factory-reset ─► sync, reboot
boot ─► overlay-init mounts data ─► marker? ─yes─► wipe data ─► build the overlay ─► init
└─no────────────►┘
Every trigger runs the same script, /usr/sbin/ethernet-switch-os-factory-reset
(recipe ethernet-switch-os-auth):
| Trigger | Path |
|---|---|
| Reset button, held 5 s or more (GS1900-8, test switch) | triggerhappy → ethernet-switch-os-reset-key |
CLI factory-reset, after a y/N question |
RPC clixon-switch:factory-reset → backend plugin |
| Web interface: Administration → Factory reset | the same RPC over RESTCONF |
root on the serial console |
the script itself |
The RPC runs the script with --later: it writes the marker and returns,
and the reboot follows two seconds later, so the reply still reaches the
client. A short press of the button only reboots.
How the wipe is done depends on the storage. overlay-init belongs to the
BSP layer:
| Board | Wipe |
|---|---|
| RTL83xx (SPI-NOR, JFFS2) | flash_eraseall -j erases the whole data partition. Nothing of the old configuration, password or keys stays readable in the flash. |
| Raspberry Pi, QEMU (ext4) | The upper and work directories of the overlay are deleted. The blocks are not overwritten: on an SD card they may still be readable. |
If the erase fails, overlay-init continues with a tmpfs overlay: the switch
comes up in the factory state, but keeps nothing across a reboot.
The factory .swu of the RTL83xx boards writes an empty JFFS2 to data,
which is a factory reset as well.
Boot sequence¶
- The bootloader (the stock U-Boot on the RTL83xx boards, U-Boot on the Raspberry Pi, EFI Boot Guard on QEMU) loads the kernel.
- The kernel probes the switch driver, which creates the conduit and the
front ports
lan1...lanN, named by the device tree. - An overlay init script mounts the read-only squashfs root with the
writable
datapartition on top (JFFS2 on the RTL83xx flash, ext4 on the SD card and the QEMU disk). Before that, it wipesdataif a factory reset left its marker there. - busybox
initruns the SysV init scripts:clixon-backendrunsprepare-datastore(createsstartup_dbfrom the factory default on the first boot) and startsclixon_backend. The backend commitsstartup_dbthrough the plugin, which createsbr-lan, adds the ports and puts the management address onvlan1. There is no other network configuration: no/etc/network/interfaces, no NetworkManager.clixon-restconfstartsclixon_restconfon 127.0.0.1, port 80.dropbear(SSH,-w: no root logins) andswupdate(firmware update, 127.0.0.1 port 8080).lighttpdmakes the HTTPS certificate on the first boot and after a factory reset, and listens on port 443 in front of both.
Processes at runtime¶
| Process | Started by | Port / socket | Role |
|---|---|---|---|
clixon_backend |
init script | /var/run/clixon-switch.sock |
Configuration, with the plugin loaded |
clixon_restconf |
init script | TCP 80 on 127.0.0.1 | RESTCONF |
lighttpd |
init script | TCP 443 | HTTPS and the login for the web interface, RESTCONF and SWUpdate |
clixon_cli |
SSH or serial login of the admin account | — | CLI |
dropbear |
init script | TCP 22 | SSH |
swupdate |
init script | TCP 8080 on 127.0.0.1 | Firmware update and its web page |
mstpd |
plugin, while spanning tree is enabled | — | STP, RSTP, MSTP |
udhcpc |
plugin, while a DHCP client is configured | — | DHCP client |
snmpd, clixon_snmp |
plugin, while SNMP is enabled | UDP 161, AgentX socket | SNMPv3 agent |
lldpd (two processes, one as nobody) |
plugin, while LLDP is enabled (factory default) | raw socket on the switch ports, control socket | LLDP, and LLDP-MIB while SNMP is on |
Only ports 22 and 443 are open to the network, plus UDP 161 while SNMP is on. LLDP is not IP: its frames stay on the link they arrive on. The TFTP initramfs, the factory installer, also leaves SWUpdate on port 8080 of every address: it has no password yet, and it only runs after a boot from the bootloader's serial console.
Access and security¶
Accounts¶
| Account | Password | Serial console | SSH | Web, RESTCONF |
|---|---|---|---|---|
root |
none | yes, a shell | refused | no |
| admin, name chosen in the setup, UID 1000 | chosen in the setup | yes, the CLI | yes, the CLI | yes |
A fresh switch has no admin account at all: the first-login setup creates it. Until then SSH has no account to let in, and the web interface shows only the setup form.
root has an empty password, but can only log in where someone has the
hardware in hand. dropbear runs with -w (no root logins), /etc/securetty
lists the console devices only, not the pseudo terminals of SSH sessions,
and su is restricted to the wheel group, which nobody is in. The admin's
login shell is the clixon CLI (ethernet-switch-os-cli, which prints a joke
and execs clixon_cli), so it never gets a shell to try.
The image sets this up in ethernet-switch-os-image-common.inc: the image
feature empty-root-password, but neither allow-empty-password (dropbear's
-B; no account has an empty password that SSH would need to accept) nor
allow-root-login, which would drop -w.
First-login setup¶
The firmware ships with /etc/ethernet-switch-os/setup-required and no admin
account. While the file exists:
- The web interface shows only a form for the username and password of the
admin account. lighttpd answers two RESTCONF requests without a login:
reading
setup-requiredand theset-passwordRPC. A script can do the same with the RPC directly:
curl -k -H 'Content-Type: application/yang-data+json' \
-d '{"clixon-switch:input":{"username":"ops","new-password":"a good password"}}' \
https://192.168.1.1/restconf/operations/clixon-switch:set-password
- There is no SSH login, and on the serial console only
root.rootcan set the switch up from there too:ethernet-switch-os-set-passwordasks for the username and the password.
The setup creates the account with useradd -u 1000 -N -g users -G clicon
(the clicon group opens clixon's backend socket to the CLI; users as the
primary group, so no group of the admin's name is made), sets the password
and deletes the file, which leaves a whiteout in the overlay. A factory reset
erases the overlay: the account is gone and the file is back.
The username:
- a lower case letter or
_, then up to 31 lower case letters, digits,_or-: the part of the POSIX portable set that useradd,login, dropbear and htpasswd (no:) all take; - not the name of an existing user or group (
root,clicon,lighttpd,usersand the rest of/etc/passwdand/etc/group); - fixed until a factory reset.
set-passwordrefusesusernameafter the setup.
The backend plugin (switch_net::account in clixon-switch-rs), the script
and, for the form's hints, the YANG model (pattern) and the web interface
check the same rules.
Until the setup anyone who reaches the switch can create the admin account. Do it before connecting the switch to a network others use.
The admin is UID 1000, not a name. Nothing in the firmware knows the
name the user picked, and nothing needs to: the set-password script, the
backend plugin's current-password check and the migration below look up the
user with UID 1000 in /etc/passwd; lighttpd lets in any user in its
htpasswd file (valid-user), which only ever holds the admin's line; and
the CLI runs as whoever logged in.
Switches updated from older firmware keep cli. Before the username
could be chosen, the image had a fixed admin account cli. After an update,
the data partition still has the old /etc/shadow with cli's password,
and setup-required is deleted there, but the new image's /etc/passwd has
no cli: nobody could log in, and there would be no setup either. The init
script ethernet-switch-os-auth (before dropbear) sees exactly that, a
cli in /etc/shadow and no UID 1000 in /etc/passwd, and adds cli back
as UID 1000 and to clicon. The switch then works as before, with the old
password. A factory reset ends that, and the setup asks for a name.
Passwords¶
/etc/shadow is the only place the password lives. Whoever sets it, the
same script does the work:
web setup form ─┐
CLI "password" ─┼─► RPC clixon-switch:set-password ─► backend plugin ─► ethernet-switch-os-set-password
web "Change ─┘ (setup: checks username; (root) ├─ setup: useradd -u 1000 NAME
password" later: checks ├─ chpasswd -c BCRYPT → /etc/shadow
current-password) ├─ copy of name, hash → /etc/lighttpd/htpasswd
root on serial ─────────────────────────────────────────────────────────►├─ rm setup-required
└─ restart lighttpd
The backend plugin checks the rules before it runs the script: the username
in the setup, afterwards current-password against /etc/shadow with
crypt(), and always the password rules (8 to 128 characters, no control
characters). The RPC answers access-denied (HTTP 403) for a missing or
wrong current password, and invalid-value (HTTP 400) for a username it
does not take. root on the serial console runs
ethernet-switch-os-set-password directly, to regain access without a
factory reset: it changes the password of UID 1000, whatever its name.
The hash is bcrypt ($2b$, cost 8, BCRYPT_MIN_ROUNDS and
BCRYPT_MAX_ROUNDS in /etc/login.defs). Every program that checks a
password uses musl's crypt(): login, su, dropbear, lighttpd and the
backend plugin. musl knows bcrypt and SHA-crypt, and bcrypt is the harder of
the two to attack with GPUs. yescrypt would be stronger still, but musl does
not have it: it needs libxcrypt in place of musl's crypt() for all of these
programs, and it is memory-hard by design (16 MiB per check at its default
cost) on a switch with 128 MiB. shadow writes bcrypt hashes only when built
with it (recipes-extended/shadow in meta-ethernet-switch-os).
Cost 8 is 2^8 rounds, chosen to keep a check well under a second on the
RTL838x. A login pays that once. The web interface sends several requests every few seconds,
so lighttpd keeps a checked login for 10 minutes (auth.cache); the script
restarts lighttpd so that a changed password counts at once.
HTTPS: lighttpd in front¶
clixon_restconf and SWUpdate's web server listen on 127.0.0.1 only, without TLS or a login. lighttpd does both for them, so there is one certificate, one login and one origin:
network ──443/tcp──► lighttpd ─┬─ / static files, /usr/share/ethernet-switch-os/www
(TLS, ├─ /restconf 127.0.0.1:80 clixon_restconf
basic ├─ /.well-known 127.0.0.1:80 clixon_restconf
auth) └─ /update/ 127.0.0.1:8080 SWUpdate (prefix stripped)
──22/tcp───► dropbear (-w)
UART ──────────► login: root or the admin
| URL on port 443 | Login | Goes to | Notes |
|---|---|---|---|
/, the web interface's files |
no | /usr/share/ethernet-switch-os/www |
The files hold no data; the setup form has to load without a password. |
/restconf/... |
yes | 127.0.0.1:80 |
|
POST /restconf/operations/clixon-switch:set-password |
no | 127.0.0.1:80 |
Creates the admin account in the setup; needs current-password once it exists. |
GET /restconf/data/clixon-switch:system/state/setup-required |
no | 127.0.0.1:80 |
The web interface asks it to choose between setup form and normal page. |
/.well-known/... |
no | 127.0.0.1:80 |
RESTCONF root discovery (RFC 8040). |
/update/... |
yes | 127.0.0.1:8080 |
map-urlpath strips /update; WebSocket upgrade for the progress messages; the .swu is streamed through, not stored first. |
The login is HTTP basic auth for the admin account, realm "Ethernet Switch OS",
checked against /etc/lighttpd/htpasswd (mode 0640, group lighttpd).
lighttpd binds port 443 as root, reads the certificate and key, and then runs
as the user lighttpd. It is built without pcre, so its configuration
(recipes-extended/lighttpd/files/lighttpd.conf in meta-ethernet-switch-os)
uses prefix and exact matches only.
SWUpdate's page works below /update/ because it uses relative URLs only
and builds its WebSocket URL from location.pathname, with wss: on
HTTPS.
The certificate is self-signed, P-256, made by lighttpd's init script with
openssl req into /etc/ethernet-switch-os/tls/. P-256 takes seconds on
the RTL838x where RSA-2048 would take a minute. The switch has no real-time
clock, so the validity is fixed (2000 to 9999) instead of counted from the
clock. Browsers warn about it; there is no way to import a certificate yet.
Not covered: user roles. There is one admin account, and anyone logged in may do everything. NACM (RFC 8341) is on the README's TODO list.
Operating system base¶
- Built with the Yocto Project (openembedded-core, branch wrynose),
distro
ethernet-switch-os, based onpoky-tiny: musl libc, busybox as init, shell and device manager (mdev). No systemd, udev or D-Bus. Flash is the constraint: the GS1900-8 has 16 MiB of SPI-NOR for everything. - Linux 6.18 on all current boards.
- Root filesystem: read-only squashfs, with an overlay on the
datapartition for everything that is written (the saved configuration, the admin password, SSH host keys, the HTTPS certificate). - SWUpdate installs
.swuupdate files: in place on the RTL83xx boards (one firmware slot), A/B with rollback on the Raspberry Pi and QEMU. See Firmware update. - Rust for the plugin, cross-compiled by Yocto. On the RTL83xx boards
that is
mips32r2big endian with musl, a tier 3 Rust target. - Version:
DISTRO_VERSIONis a base version set by hand plus the output ofgit describeonmeta-ethernet-switch-os, the layer that holds the distro configuration, every recipe and the images (conf/distro/include/ethernet-switch-os-version.inc). Without tags that is0.0.0-3eae8394; a tagv0.1.0makes it0.1.0, and a commit after it0.1.0-1-g1a2b3c4d, so a release number takes a tag and no change to the metadata. The revision alone isBUILD_IDin/etc/os-release. It reaches the login banner,/etc/os-release, the RESTCONFos-version, the SNMPsysDescrand theversionfield of everysw-description. One revision cannot describe a build made from six repositories, soimage-buildinfowrites the branch and revision of all layers to/etc/buildinfo.
Where the code is¶
| Repository | Content |
|---|---|
| ethernet-switch-os | kas files per board, CI, QEMU scripts, this documentation |
| meta-ethernet-switch-os | The distro: clixon and cligen recipes (with patches), the clixon-switch recipe, init scripts, the web UI, SWUpdate integration, mstpd, net-snmp and lldpd adjustments. Reaches into each BSP through dynamic-layers/. |
| clixon-switch-rs | The backend plugin, the YANG models, the CLI spec, clixon.xml, the factory default |
| meta-rtl83xx-bsp | Realtek RTL83xx: kernel with the OpenWrt drivers, device trees, boot image, flash layout, rtl838x-qemu recipe |
| meta-rpi-managed-switch-bsp | Raspberry Pi switch HAT: kernel configuration and backports, device tree overlay, SD card layout, A/B update |
| meta-qemu-switch-bsp | QEMU x86-64 switch: machine, disk layout, EFI Boot Guard |
| rtl838x-qemu | QEMU models of the RTL838x SoC, for testing the real GS1900-8 image |
The BSP layers are hardware only and boot to a shell without the distro layer. The distro layer does not depend on any BSP. See Layers and kas files.
Inside clixon-switch-rs, the plugin is split so that most of it can be tested on a development machine without clixon or root rights:
| Crate | Content |
|---|---|
switch-model |
JSON → validated DesiredState; state data as XML; snmpd.conf. Pure functions. |
switch-net |
ActualState, the planner, reconcile, the netlink backend, a fake kernel for tests; mstpd, udhcpc, snmpd and lldpd management |
clixon-sys |
Declarations of the libclixon functions in use |
clixon-plugin |
A safe Rust interface to clixon's plugin callbacks. Panics are caught, so a bug fails the transaction instead of crashing clixon_backend. |
clixon-switch-plugin |
The shared library clixon_backend loads; wires the other crates together |
Where to start¶
| You want to ... | Start in |
|---|---|
| Support a new setting | The YANG model if needed (yang/), accept it in switch-model (supported.rs, lib.rs), map it to the kernel in switch-net (plan.rs, netlink.rs). Add unit tests and a pytest integration test. The CLI and RESTCONF get it for free. |
| Change the web UI | recipes-webui/ethernet-switch-os-webui/files/www/ in meta-ethernet-switch-os. It only uses RESTCONF. |
| Add a daemon or package | packagegroup-ethernet-switch-os-base.bb or the image .bbappends in meta-ethernet-switch-os. Mind the flash size. |
| Try the plugin quickly | cargo test on the host, then dev/container.sh in clixon-switch-rs, which runs clixon with the plugin on dummy interfaces. See its README. |
| Test the whole firmware | The QEMU boards, see Testing and CI. |
Bringing up new hardware¶
- Find a Linux kernel with a DSA or switchdev driver for the switch chip that has the offload features listed above. The Linux DSA Feature List Explorer lists the DSA drivers, the chips they cover and the features each one implements. Check OpenWrt too, for drivers not in mainline yet.
- For a chip family the project already knows (Realtek RTL83xx), a new
machine
.confand device tree in the existing BSP and a board file inkas/board/are enough, see Adding a board. Otherwise write a new BSP layer, see Adding a BSP. It has to boot to a shell on its own. - Name the front ports
lan1,lan2, ... in the device tree. The plugin and SNMP derive port numbers from these names. - Add the board to
conf/distro/include/ethernet-switch-os-boards.incin meta-ethernet-switch-os: its front ports for the factory default, and its hardware revision for SWUpdate. - Add
.swurecipes for the board's flash layout underdynamic-layers/<bsp>/in meta-ethernet-switch-os.
No change to the plugin is needed, unless the driver behaves differently from the ones above.