RPi4 vs RPi5 as 5×9 LMS WiFi monitoring hardware

Introduction

The new RPi5 hardware has been available for some time, and we’ve finally had the time and opportunity to conduct some hands-on tests. While the 2-3x CPU performance gain compared to the RPi4 is noteworthy, it isn’t the primary focus of this test. The RPi4 already has sufficient CPU power to perform multiple simultaneous measurements as a 5×9 Lightweight Monitoring System (LMS) probe. Instead, the focus of this test is to compare the WiFi performance of two Raspberry Pi generations that utilize the same WiFi chip but have different internal architectures.

This test is conducted not under ideal laboratory conditions but within a real home network environment utilizing Mikrotik equipment. Additionally, a third type of WiFi monitoring hardware, the Asus PN51 ultracompact mini PC, is introduced solely for comparison purposes. The aim is to evaluate the performance of the inexpensive Raspberry Pi SBCs against a more expensive x86 device.

The primary focus of the test lies in evaluating WiFi performance, which is assessed through Speedtest (utilizing the Internet connection) and Network Bandwidth (utilizing local server) measurements. Each WiFi measurement hardware device is integrated into the 5×9 LMS system as a Probe. The LMS system takes responsibility for Probe configuration, management, test execution, performance measurement collection, data retention and visualizations. For more details about the 5×9 LMS, click here.

Test setup description

Test network

As mentioned, the test is conducted in a real home network environment based on Mikrotik equipment. The HAP AC serves as the router connected to the Internet, the RB260GSP functions as the L2 gigabit switch and PoE source for three APs, while CAP ACs are utilized as WiFi access points.

• CAP AC AP (RBcAPGi-5acD2nD)
  • RouterOS 7.13.4
  • 2×2 MIMO, 867 Mbit/s
• RB260GSP PoE switch (CSS106-1G-4P-1S)
  • SwOS 2.13
• HAP AC router (RB962UiGS-5HacT2HnT)
  • RouterOS 7.13.4
  • Fiber Internet connection, 500/250Mbps Dl/Ul speed

WiFi hardware devices

As the test name suggests, both RPi4 and RPi5 devices are used. To compare the WiFi performance of these inexpensive single-board computers (SBCs) with more robust hardware, Asus PN51 ultracompact mini PC, featuring an AMD Ryzen processor and Intel AC WiFi card, is also included in the test.

• RPi4, 8GB RAM, official case and 15W PSU,
  • WiFi client role
    • WiFi 5, 1×1 MIMO, 433Mbps (BCM4345/6 – Cypress CYW43455)
  • Ethernet used for OoB management
• RPi5, 8GB RAM, official cooling case and 27W PSU
  • WiFi client role
    • WiFi 5, 1×1 MIMO, 433Mbps (BCM4345/6 – Cypress CYW43455)
  • Ethernet used for OoB management
• Asus PN51, 32GB RAM, Ryzen 5700U
  • WiFi client role
    • WiFi 5, 2×2 MIMO, 867 Mbps (Intel AC 8265)
  • Network Bandwidth server role
    • 1GE LAN connection (Realtek RTL8125 2.5GbE)
  • Ethernet used for OoB management
    • 1GE LAN connection (Realtek RTL8125 2.5GbE)

WiFi hardware device management

To streamline the process, eliminate the need for manual setup of measurement infrastructure and simplify performance results collection and data visualization, all three hardware devices are enrolled as Probes within the 5×9 LMS. Subsequent screenshots related to hardware device management are taken from the LMS Node Manager configuration frontend. The 5×9 LMS oversees test creation, schedule definition, test execution, measurement data collection, and performance metrics visualizations via automatically created Grafana dashboards.

• 5×9 Lightweight Monitoring System
  • LMS instance deployed in 5×9 Lab

WiFi network setup

The test network consists of 3 access points (APs), so Mikrotik CAPsMAN feature is used for centralized AP management. Local forwarding functionality is disabled (AP traffic is not tunneled to the router), and client-to-client forwarding functionality is enabled. CAPsMAN is auto-selecting 2,4 and 5GHz channels from a predefined channel list for each band and channel width test case. The 2.4GHz test is conducted in a relatively crowded environment with a 20MHz channel width. For the 5GHz band, three tests are conducted with 20, 40, and 80MHz channel widths, respectively. While some 5GHz channels permit higher transmission power, tx-power is limited to 23 dBm (200mW) to avoid having a high power RF source in the home.

• 2,4GHz WiFi test configuration
  • Tx power limited to 20dBm (100mW)
  • 20MHz channel width
  • CAPsMAN autoselect from channels 1, 6 and 11
• 5GHz WiFi tests configuration
  • Tx power limited to 23dBM (200mW)
  • 20, 40 and 80MHz channel width
  • 20MHz control channel width
  • CAPsMAN autoselect from frequencies
    • 20MHz test – 5180,5200,5220,5240
    • 40MHz test – 5180,5220,5260,5300
    • 80MHz test – 5180,5260,5500

Tests

All test measurements are performed through the 5×9 LMS system, with tests categorized into two sections: WiFi connection validation and performance assessment.

The connection validation section ensures that all WiFi connection establishment-related timers and errors are collected to confirm a successful connection to the SSID. Additionally, periodic captures of WiFi radio conditions are performed to evaluate the surrounding WiFi environment and assess CAPsMAN channel selection

• WiFi connectivity framework
  • Used on all Probes
  • Connection establishment-related timers and errors
• WiFi radio scan
  • Used on all Probes
  • WiFi radio conditions capture

The performance assessment section measures WiFi performance using the Speedtest and Network Bandwidth measurements. Two different measurements were selected to confirm the correlation of results. The Speedtest is conducted towards a server hosted on the internet, limited by the internet connection speed to 500/250Mbps. The Network Bandwidth test is performed locally towards a NetBW server hosted on the PN51 Probe’s wired interface. All three NetBW clients execute WiFi measurements towards the NetBW server. The design of the 5×9 LMS Probe engine enables the PN51 to serve as both the NetBW server and client simultaneously, facilitating measurements conducted from the local WiFi to the local wired interface

• Speedtest
  • Used on all Probes
  • Up and Down Speedtest
• Network Bandwidth client
  • Used on all Probes
  • Up and Down network bandwidth
• Network Bandwidth server
  • Used only on PN51 Probe
  • Responder for NetBW clients
• Ethernet connectivity framework
  • Used only on PN51 Probe, assures wired connectivity for Network Bandwidth server

 

WiFi tests were conducted across 2.4GHz-20MHz, 5GHz-20MHz, 5GHz-40MHz, and 5GHz-80MHz spectrums. Each test lasted a minimum of 6 hours, ensuring at least 24 measurement samples for each probe. A strict deterministic schedule for Probe runs was implemented to prevent any overlap in measurements.

• RPi 4 measurements schedule: 0,15,30 and 45 min
• RPi 5 measurements schedule: 5,20,35 and 50 min
• PN51 measurements schedule: 10,25,40 and 55 min

 

Each measurement run covers both connection validation and performance assessment phases and includes the following tests

• Monitoring WiFi connection establishment
  • WiFi connection-related timers
  • Link up, DHCP time and AP connection time + related errors if any
• Speedtest measurement
  • Towards Internet server
  • Uses Internet connection (500/250Mbps)
  • Up and down directions
• Network BW
  • Towards Network Bandwidth server (PN51 Probe, wired 1GE LAN connection)
  • Up and down directions
  • TCP, 30s test duration

Periodical WiFi radio conditions snapshot is recorded via WiFi radio scan module for later analysis and troubleshooting purposes (if needed). Upon closer examination, you’ll notice that the WiFi radio scan module (wifiradioscan) is bound to the Ethernet interface. This is an intentional feature, not a bug. By binding it to the Ethernet interface, we can monitor the WiFi radio surroundings with Probes that aren’t directly engaged in WiFi-related measurements.

 

Test results

Summary

As a picture speaks a thousand words, the easiest way to summarize all tests is to compare measurement results normalized to the same scale.

The Raspberry Pi 4 performs comparably to the group for 2.4 and 5GHz with a 20MHz channel width, but it lags behind in 5GHz measurements with 40 and 80MHz channel widths. In all measurements, PN51’s 2×2 MIMO support offers some advantages, although the advantage is not significant compared to RPi5.

Details

2,4GHz – 20MHz test

The WiFi 2.4GHz performance with a 20MHz channel of both RPi 4 and RPi 5 is identical. However, the PN51 demonstrates better upload and download speeds due to its 2×2 MIMO support. A prerequisite to enabling MIMO support for downstream direction on the Intel AC card is to manually disable SMPS (Spatial Multiplex Power Save). Otherwise, the Intel AC card will only utilize one stream in downstream for some reason.

 

5GHz – 20MHz test

The WiFi 5GHz performance with a 20MHz channel is identical between RPi 4 and RPi 5. However, the PN51 demonstrates better download and upload speeds due to its 2×2 MIMO card.

 

5GHz – 40MHz test

In the 5GHz band with a 40MHz channel, the RPi5 exhibits a noticeable improvement over the RPi4. Furthermore, the PN51 demonstrates higher download and upload speeds (2×2 MIMO again).

 

5GHz – 80MHz test

The RPi5 demonstrates significant improvements over the RPi4. Furthermore, the PN51 displays improved download and upload speeds (2×2 MIMO again), although the RPi5 closely trails behind. It’s interesting to note that the upload and download performance of the RPi5 is more stable than that of the PN51.

Conclusion

Is the new RPi5 better than the RPi4 for WiFi monitoring? Absolutely. Despite using the same WiFi chip, it seems that the RPi4 encounters an internal bottleneck, particularly limiting bandwidth on the 5GHz band for 40 and 80MHz scenarios. One potential culprit for this bottleneck is the SDIO interface, which has been upgraded on the RPi5. However, it’s worth noting that there isn’t much available documentation for the RPi5 to confirm this.

Can the RPi4 be used for WiFi monitoring? In our opinion, yes. It’s unlikely that 80MHz channels on the 5GHz band will be extensively used in crowded production WiFi environments, considering the limited number of available channels (only 5 for 80MHz and 2 for 160MHz). This makes the RPi4 suitable for 2.4GHz and 5GHz scenarios using 20 and 40MHz channel widths. Moreover, tests that consume maximum bandwidth periodically by the Probe are not recommended for production WiFi networks. Instead, network bandwidth tests with a capped limit should be utilized, such as periodically verifying that WiFi provides certain and stable throughput. To provide a comprehensive view of WiFi performance, it’s essential to include additional measurements such as DHCP, DNS, HTTP, YouTube (video is great benchmark for wireless networks), and more. Additionally, the RPi4 can provide valuable insights into WiFi service operation, including connection timings, errors, and radio conditions, which immediately detect and pinpoint specific WiFi problems (RF, DHCP, AAA, …) and shorten troubleshooting time.

How does the RPi5 compare to the PN51 with an Intel AC card? While the PN51 boasts certain advantages due to its 2×2 MIMO support, these advantages are not significant. For a complete comparison, it’s worth noting that the PN51 is priced at over 5 times higher than the RPi5, making the RPi5 an even more desirable hardware choice for remote WiFi monitoring probes.

Last Thought

Our 5×9 LMS Probe is completely hardware agnostic. If the performance of the RPi5 SBC does not meet your requirements, we have validated E.E.P.D. SBC hardware based on AMD Ryzen PRO 6000 equipped with an M.2 slot, similar to the PN51 hardware used in this test.

 

This compact SBC can support high-performance M.2 WiFi6 or WiFi7 cards of your choice, enabling monitoring of Gig+ WiFi networks.

Additionally, we heavily utilize SBCs (both RPi and Ryzen based) and x86 servers, for extensive monitoring of networks, services, and applications. However, as it is getting late, we’ll explore this topic further in the following blog posts 🙂

Author: Mario Jurcevic, 5×9 Networks