Wednesday, January 10, 2007

[en] Throughput tests WRTSL54GS v1.1 vs WRT54GS v3.0

Throughput tests WRTSL54GS v1.1 vs WRT54GS v3.0

The goal of this test is to measure and compare the NAT'ing/routing/switching behavior of these two WRT54G's [1] running OpenWRT Whiterussian RC6 [5]:

models/nCPUflash/RAMeth switch
WRTSL54GS v1.1 [2]CJK11xxxxxxxBroadcomm 4704rev9 266Mhz8/32MBinCPU
WRT54GS v3.0 [3]CGN5xxxxxxxBroadcomm 5352 200Mhz8/32MBBCM5325

IMPORTANT: nbd ( openwrt developer) pointed a problem with "entropy gathering which fills /dev/random from wifi" in WR rc6 and suggested me trying a newer build (09test); I'll try it an repost the results (specially the wifi ones)
--jjo; Jan 10, 2007.

Testbed, methodology

  • Topology

  •                        --------> iperf test connection------>
    WIFI-NATed [CLIENT]+)) wifi (( +[ WRT ]+=== wan ===+[SERVER] (iptables MASQUERADE'd)
    LAN-NATed [CLIENT]+=== lan ===+[ WRT ]+=== wan ===+[SERVER] (iptables MASQUERADE'd)
    LAN-routed [CLIENT]+=== lan ===+[ WRT ]+=== wan ===+[SERVER] (plain routing)
    LAN-switched [CLIENT]+=== lan ===+[ WRT ]+=== lan ===+[SERVER] (hardware switching)

    "lan": any LAN (1of4) port
    "wan": WAN port
    "wifi": 802.11g link at 54Mbps (aprox 2mts distance between antennas)
    iptables MASQUERADE'd: default whiterussian setup (masquerade both wifi,lan to wan)
    plain routing: accomplished with iptables full "flushing" + server route to client via WRT

  • Hardware

    • client: IBM Thinkpad T42, lan: e1000 (10/100/1000), wifi: ipw2200
    • server: generic PC sempron 2200+, lan: 8139too
    • routers: (stated above)

  • Software
    • client, server: iperf [4] (-c, -s) on linux-2.6, glibc-2.3+.
    • routers: OpenWRT WhiteRussian RC6 [5].


Throughput values in megabits per second taken from iperf's output. Please note that measured rates by iperf correspond to _application_, ie: they don't consider protocol headers, etc.

CPU usage % shown by top for ksoft_IRQd from a ssh logged session, this "top" session was not considered for throughput values (separately taken) to avoid "measuring load influence": 100% CPU usage was aprox= 90% ksoft_IRQd + 8% dropbear + 2% top.

Throughput (CPU usage %)
WRTSL54GS23.5mbit/s (65%)47.5mbit/s (90%+)55.8mbit/s (90%+)93.7mbit/s (0%)
WRT54GS 10.5mbit/s (14%)34.2mbit/s (90%+)40.4mbit/s (90%+)93.8mbit/s (0%)



WIFI-NATed at 23.5 mbits (802.11g @54Mbps) with ~65% CPU usage denotes "reasonable" CPU availability for this 802.11g saturated link.

LAN-NATed at 47.5 mbits shows the CPU bottleneck when routing+MASQUERADING, considering that ip_conntrack table has just a few entries this is a VERY optimistic max.

LAN-routed at 55.8 mbits is pretty poor, I think that ip_conntrack processing is ALSO affecting this scenario (if loaded it gets called even for non-nated streams); it would be veryVERY useful to have ip_conntrack optionally loaded by openwrt (currently static).

LAN-switched at 93.8 mbits is expectable given that hardware only switching is involved.


LAN-NATed and LAN-routed values give about 75% when compared to its newer cousin, 75% is roughly 200/266, respective CPU clocks [MHz] (this confirms CPU saturation).

WIFI-NATed at 10.5 mbits (dbl-confirmed 802.11g @54Mbps) is a bad smelling frog, I double checked openwrt setup... nothing strange found; about 1mo ago I was running RC5 getting ~20mbits+ rates (informal, from rsync/ftp rates), so it maybe something with RC6 wifi in this hardware, dunno(?).

As expected, LAN-switched gave same results.


Both WRT's seem reasonable for 802.11g @54Mbps even when NAT'ing traffic (common usage), obviously depending on traffic patterns, how many clients, etc ... given the constrained RAM (and CPU) for NAT+conntracking ... NOT tested here.

On the contrast, their CPU is not powerful enough to saturate _both_ (LAN+WAN) 100mbps links simultaneously, even when "only" routing (55mbit/s max on WRTSL54GS); this could be enhanced by having ip_conntrack optionally (not)loaded by openwrt.

Out of band disclaimer

Ok ... I know, you agree: blogger sucks for writing technical reports .. sheesshhhh :-P


[4]Iperf bandwidth measurement tool:

This thread at openwrt forums talks about openwrt experiences on Linksys WRTSL54GS model, and was the starting point to encourage loading openwrt on WRTSL54GS v1.1 model.


redondos said...

Great research!

Any news about the tests with the 09test build?

I'm also using RC6 and I'm concerned this might be an issue with my GS v2.

free ps3 said...

Thanks for the nice post!