Speed test and Speed Optimization on Firewalla

Firewalla

March 23, 2023 22:18
Updated

This document is a work in progress.

Testing the speed of your network is both an art and a science. Especially if you have a Gigabit WAN network or faster, here are some tips to get the most meaningful results and to use Firewalla to optimize all your equipment.

If this does not solve your problem, please get in touch with the support team here https://help.firewalla.com/hc/en-us/requests/new (If possible, please let us know some of your test results, it will make the diagnostic process faster)

Overview

Testing and tuning your network can be done in a series of steps:

Test Ethernet connections starting those coming directly from your firewalla and then on any segment that does not seem to perform. For this, use the local network test described below by visiting http://fire.walla:8833/ss/ with an ethernet connection.
Test Wi-Fi using either the Wi-Fi Speed Test in the Firewalla app (described below) or use http://fire.walla:8833/ss/ on any device.
Test From Firewalla to the Internet. In the Firewalla app we have an Internet test that measures only the connection from Firewalla to Internet speed servers (shown in white). Your network is shown in Green. By testing each separately you can quickly identify where any issues are.

Check your Ethernet infrastructure

Make sure you have the right cables. Gigabit networks require CAT 5E, 6, or 7 ethernet cables. Replace any CAT 5 cables if you find them. This is the most common problem.
Make sure cables are in good condition. If you have any doubt about a cable, test or replace it.
Sometimes devices will negotiate at a lower speed.
1. On Firewalla, tap on Settings > About > Port Speed to look at port speed to check the speed of devices connected directly to Firewalla.
2. On other devices, check to see the device settings to make sure they are connecting at the speed you expect and that a slower speed wasn't manually or automatically set. A bad cable or connecting to a slow switch can cause a device to slow the connection automatically. Below is an example for macOS where the speed can be set to automatic or to a fixed speed.
Remember, the speed from one point to another point is only as good as the slowest point in the journey. If your ethernet is going through a slow switch, everything downstream will be slowed as well. Make sure any ethernet switch can handle wire-speed switching for all the ports.
Test via an Ethernet connection to get a baseline. This takes all of the variables that come with Wi-Fi out of the equation. Wi-Fi tests may be comparable in slower networks, but for anything beyond 200 Mbit, you should use Ethernet first.
When possible, use a PC/MAC for the test with at least gigabit ethernet.
Your test device should start as close to the router as possible. We have seen people use ethernet over long coax cables (MoCA)... If you have a MoCA network in your home, test on either side to eliminate the MoCA from the equation. Meaning the ethernet up to the MoCA adapter and then again on the other side. If you find an issue, consult with a networking expert who can advise you on a fix.
If you have daisy-chained switches, make sure the connection between the links is not saturated when you are doing testing. Otherwise, you will be limited by the switch's interconnect speed.

Strategies

The goal is to test each segment of your network. Each step will be described in detail below.

Test something connected over the proper grade of CAT cable; if you have Firewalla Gold Plus you need CAT 6a. CAT 5e is adequate for 1Gbps connections like Purple and Gold. See how the speed looks using http://fire.walla:8833/ss/ This should show a speed between 1Gbps to 2.5Gbps depending on what device you connect to Firewalla.
Test the next segment either ethernet or Wi-Fi as the case may be. Repeat for each physical network segment until you have tested all legs of the network.
If all the LAN tests well, use the Internet speed test in Firewalla to test the speed from Firewalla to the speedtest server (which eliminates the LAN).
Use a public server like speedtest.net to test the overall speed from a device to an external server.

Speedtests

When it comes to internet speed tests, many people look to sites such as https://www.speedtest.net/. The problem with these sites is they give you an incomplete picture. Firewalla can help you tune your network with tools that compliment the public speedtests.

Firewalla Speed tests

Firewalla provides two kinds of speed tests: LAN and WAN. The LAN is in green and the WAN test is in white.

Having separate tests allows you find if bottlenecks are within your LAN, or the WAN. If either the LAN or WAN connections are poor, your experience will suffer. If your LAN is poor, calling your ISP won't help. Just diagnose and fix the issue. If the WAN is poor, chances are you need help from your ISP. Being armed with this level of detail is better than only having something like speedtest.net. That is not to say public test servers aren't useful. Having all three tests is better because it helps confirm both the WAN speed test and the overall picture.

Test your LAN

LAN tests will help you identify issues within your network. If you find no issues on your LAN, that is information that will help when you have to call your ISP.

1. You can check the connection from your Firewalla to any device connected directly over ethernet. See Settings > About > Port Speed.
2. All Firewalla boxes have the option of testing the connection speed from a device to Firewalla. In the Firewalla mobile app, use the Wi-Fi Speed test (Network Performance > Wi-Fi Speed)
  
  If this speedtest is slow and not consistent, you are likely just using the wrong speed test server, please see https://help.firewalla.com/hc/en-us/articles/4413511352083-Network-Performance-and-Quality-Monitoring "Tunning Section"
3. For devices that aren't running the Firewalla app or devices connected with ethernet, use http://fire.walla:8833/ss/ from any web browser. This will test only your local network. This tells you the speed of your LAN. (if you are using Safari and the local test is slow, try Chrome)
  
  Start by testing ethernet where possible because Wi-Fi can vary so much due to interference and signal strength It is a great way to find the best placement for your Wi-Fi. This covers the area shaded in green above.

When all is working well, the LAN speed test results should be close to the maximum expected speed of the connection. So, if you have a 1 GbE connection from your test device to Firewalla (including all cables, switches, or other devices in between) the LAN speed test should be very close to that.

Tracing a performance issue

If you find a problem on your LAN, you can begin narrowing it down by testing at different points. For example, in this network let's say Computer 1 is showing poor performance.

Check that the computer has a 1 GbE connection.
Check the cable "C" between Computer 1 and the switch. Try new ones if you can, or switch a cable with one that you know is working well.
Check that the switch is operating at 1GbE or better. Switches usually indicate speed by the lights on each ethernet port.
Check the cables that make up the connection "A" between Gold and the switch.
Check port 1 on Gold to make sure it shows 1GbE as well.
Move Computer 1 to port 3 of Firewalla (removing the switch) and see if the performance differs. This will test a different port on Firewalla to exclude a problem with Firewalla.
Remove the cable from the switch and connect the computer directly to Firewalla port 1. This confirms that the switch or cables in the old path were somehow an issue.

Tuning WI-Fi

When testing your Wi-Fi, move close to an AP. The speed test should be close to the top speed of your Wi-Fi AP. If it is not near the advertised speed, you can check:

Is your Access Point connected via ethernet or is it a wireless backhaul? Ethernet is always preferable when possible.
Perhaps your AP needs to be adjusted. Try moving it to a different location. Physical obstructions can interfere with performance.
If your AP has adjustments for the radios, consider settings such as radio power.
Check to see if the channels you are using are saturated by nearby APs. Use less crowded channels.
If your device supports 5Ghz, Wi-Fi speed may be better, but the range is usually shorter than with 2.4Ghz.

Wi-Fi tuning is a topic unto itself, so this is just a starting point.

Test your WAN

Once you have tested your LAN and have a baseline for network performance, you can move on to your WAN connection. There are two options to test the WAN.

Use the Internet Speed Test in the Firewalla mobile app.

.
Use the Internet Speed test or the terminal-based test measure from Firewalla to the speed test server. Details listed below in the "Pros" section.

External Speed test Servers

Once you have your WAN test, it isn't a bad idea to compare it to a public speedtest server. But note there can be differences for a lot of reasons.

Not all speed test sites are equal, and not all of them will work consistently around the world. Try a different test provider/target if you can. Location matters.
In general, the speed test given by your service provider is likely the best (most close to you).
Some speed tests will limit the maximum bandwidth getting tested. (for example, fast.com only measures speeds up to 250 Mbits)
We recommend speedtest.net, fast.com and dslreports.com
If possible, use an app instead of the web page for the test. For example, speedtest.net has OSX apps that you can download and use. It is likely more reliable.
If you are not using Firewalla in Router mode, check your router to see if it has a setting for device or traffic prioritization that limits the speed test.
Always check Smart Queue Rules to make sure you applied the right QoS correctly. We've had many cases where the customer applied a QoS rule which slowed down the network in an unexpected or undesired way.

Time

If you are using a shared medium (cable modem, for example), your speed may be impacted by what your neighbors are doing as well or how many speedtests are running at the same time. So testing speed during off-hours may have more accurate results.
If you have slower internet, it is good to test when things are quiet in the house. For example, if someone is loading a 4k youtube video, you are going to see a spike of 40 or more Megabits, and that will compete with your speed test. Similarly, iCloud backups and other large uploads or downloads can have an impact.
Try a couple of times. You may get different results, so try a few times in case you have some outlying results; high or low.

Location

Some speed tests do have the option to pick the location of the target server. The ones close to you are likely to be the fastest. Try a few others if you are not getting the speed you are after. it
Make note of which servers are consistently better and worse so you can select what's best.
Be consistent. Once you dial in a server that works well, testing the same site each time should give you an apples-to-apples comparison vs. jumping to a random server which might make it impossible to know why things have changed.

Firewalla Configuration:

Firewalla monitoring mode will influence test result slightly.
Router mode provides the fastest result, then DHCP mode, then Simple Mode.
See this for reference https://help.firewalla.com/hc/en-us/articles/115004292514-How-does-Firewalla-Intercept-Traffic-

Multi-WAN

If you have multiple WANs and are in a load balancing mode, and your speed test uses the same destination IP address, then the max speed will be whatever the link is used. Firewalla load balancing is based on the destination IP; if the speed test server IP is the same, all the traffic will go to the same circuit. A workaround is to run two different speed tests on two different servers.

Wi-Fi vs. Ethernet

The quality of Wi-Fi is not always reliable when doing speed testing. The final speed depends on many things, may even include how your neighbor is using the same channel.

Here is an example (this is Speed Test via DSL Reports over Ethernet):

Here is the same test using Wifi (Access Point 15ft away). Here even with Smart Queue on, you can see some bufferbloat, and that's likely from the Wifi side. For a better test, please use an Ethernet connection instead.

For Pros

Speed Test Inside Firewalla for WAN (PRO ONLY)

If you know how to access the Firewalla SSH shell you can use this to do a quick speed test. This test may NOT always reach gigabit speed because the software is python.

pi@firewalla:~ (GoldJCMain) $ remote_speed_test 
Retrieving speedtest.net configuration...
Testing from Comcast Cable (73.162.248.251)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Razzolink Inc (San Jose, CA) [7.37 km]: 21.692 ms
Testing download speed................................................................................
Download: 714.04 Mbit/s
Testing upload speed................................................................................................
Upload: 39.63 Mbit/s

If you have gigabit connections, you should be able to use the speedtest cli. The test binary you will have to install them yourself.

For the Gold series:

wget https://install.speedtest.net/app/cli/ookla-speedtest-1.0.0-x86_64-linux.tgz

If you have Purple:

wget https://install.speedtest.net/app/cli/ookla-speedtest-1.2.0-linux-armel.tgz

Uuntar this and run speedtest

tar -zxvf ookla-speedtest*.tgz

pi@firewalla:~/tools (GoldJCMain) $ ./speedtest

   Speedtest by Ookla
     Server: EGI Hosting - Santa Clara, CA (id = 32408)
        ISP: Comcast Cable
    Latency:    10.23 ms   (1.53 ms jitter)
   Download:   934.95 Mbps (data used: 1.1 GB)                               
     Upload:    39.67 Mbps (data used: 29.7 MB)                               
Packet Loss:     0.0%

#If you want to test it on each interface

pi@firewalla:~/tools (GoldJCMain) $ ./speedtest --interface eth0

The above two tests are run in sequence, you can see the speed difference due to different test tools.

Simple Ping Tests

If you hear a choppy voice during zoom calls or feel your speed is slower, try to use "ping" to test your network.

First, ping a well-known and stable IP, like 1.1.1.1 or 8.8.8.8, if you see something like the below, something between your PC/MAC to the internet is dropping the packet

ping 1.1.1.1

In parallel, ping Firewalla's main IP or your router IP.

ping 192.168.2.1

If your ping to your router or Firewalla has dropped packet, and the "time" is erratic and >100ms, then your LAN has a problem, usually, this is due to your WIFI connection or a bad switch.

VPN Speed Test

You can also test the speed of your connection from your VPN phone to your Firewalla by using the Speed test that is normally for LAN Wi-Fi testing.

Comments

22 comments

Sukumar Patel

January 24, 2021 16:55
Which IP do we use in FWG when it's in router mode?
0

Comment actions Permalink
Firewalla

January 24, 2021 18:38
if you are doing local testing, it will be the gateway IP (of your network segment) of the Firewalla Gold.
0

Comment actions Permalink
Chris Hewitt

February 21, 2021 18:03
This doesn’t work with the Internet IP address - right?
0

Comment actions Permalink
mobius strip

March 14, 2021 10:39
I've read that ISP's cheat and grant traffic priority to speed test websites to give their customers the dishonest appearance that the boosted results are accurate measurements that represent the customer's typical internet speeds...does the choice of speedtest providers have this in mind?

I've read good things about self-hosting your own instance of librespeed is one such way of effectively combating this, just to put this out there in case the issue hasn't already been addressed....

https://github.com/librespeed/speedtest
0

Comment actions Permalink
Michael Bierman

September 03, 2021 14:31

Edited
Want to have speedtest cli on Firewalla Gold? This will tell you how fast your internet connection is right on Firewalla (no wifi or Ethernet involved)

Firewalla will remove anything installed after upgrades so you can install a script to reinstall for you after firewalla upgrades and possibly reboots. See https://gist.github.com/mbierman/9ac6a35622ee5a0c631ed6f6ad74b722.

Then you can run speedtest.
```
speedtest --interface=eth0
```
Or
```
speedtest --interface=eth1
```
if you have dual WAN and want to test WAN2
0

Comment actions Permalink
remotebloke

July 21, 2021 19:37
The wget command doesn't work.

pi@firewalla:~ (Firewalla) $ wget https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-x86_64-linux.tgz
--2021-07-21 20:30:37-- https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-x86_64-linux.tgz
Resolving bintray.com (bintray.com)... 108.168.194.93
Connecting to bintray.com (bintray.com)|108.168.194.93|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2021-07-21 20:30:38 ERROR 403: Forbidden.
0

Comment actions Permalink
Michael Bierman

July 21, 2021 20:22
Yeah, speedtest changed something. I'll update my script shortly.
0

Comment actions Permalink
Sukumar Patel

July 22, 2021 02:27
Why don't you just use Speedtest CLI?
0

Comment actions Permalink
remotebloke

July 22, 2021 15:44
speedtest cli fails with:

Traceback (most recent call last):
File "/usr/bin/speedtest", line 11, in <module>
load_entry_point('speedtest-cli==2.0.0', 'console_scripts', 'speedtest')()
File "/usr/lib/python3/dist-packages/speedtest.py", line 1832, in main
shell()
File "/usr/lib/python3/dist-packages/speedtest.py", line 1729, in shell
secure=args.secure
File "/usr/lib/python3/dist-packages/speedtest.py", line 1009, in __init__
self.get_config()
File "/usr/lib/python3/dist-packages/speedtest.py", line 1081, in get_config
map(int, server_config['ignoreids'].split(','))
ValueError: invalid literal for int() with base 10: ''
0

Comment actions Permalink
Michael Bierman

July 22, 2021 16:37

Edited
Interesting. it is working correctly for me. Are you running the latest speedtest version?
0

Comment actions Permalink
remotebloke

July 24, 2021 07:08
Yes I think so, I just installed it.

speedtest-cli is already the newest version (2.0.0-1)
0

Comment actions Permalink

Sukumar Patel

July 24, 2021 07:57

Weird, it's working fine for me.

pi@firewalla:~ (The Patels FWG) $ speedtest -s 19081

   Speedtest by Ookla
     Server: Airtel Broadband - Mumbai (id = 19081)
        ISP: Airtel Broadband
    Latency:     1.98 ms   (0.22 ms jitter)
   Download:   855.39 Mbps (data used: 677.5 MB)
     Upload:   913.85 Mbps (data used: 718.3 MB)
Packet Loss: Not available.
 Result URL: https://www.speedtest.net/result/c/861224b7-7f6a-4e7c-bbe6-68cdb6fece26

I followed the steps mentioned here

Speedtest CLI - Internet connection measurement for developers

## If migrating from prior bintray install instructions please first...
# sudo rm /etc/apt/sources.list.d/speedtest.list
# sudo apt-get update
# sudo apt-get remove speedtest
## Other non-official binaries will conflict with Speedtest CLI
# Example how to remove using apt-get
# sudo apt-get remove speedtest-cli
curl -s https://install.speedtest.net/app/cli/install.deb.sh | sudo bash
sudo apt-get install speedtest

Ben Someone

September 02, 2021 09:56

Edited
Thanks for supplying this, makes me feel quite happy with the quality of monoprices cat6 cables! with a 16 port netgear switch in between the test computer and the firewalla I hit 1006.10 up, 999.33 down 2.49ms ping and 0.14ms jitter with a fairly standard dell win 10 system. I've seen 1000 down briefly on internet transfers so I can safely say, the firewalla gigabit ports work properly!
0

Comment actions Permalink
Michael Bierman

September 02, 2021 18:48
@Ben I don't know if this would interest you, but for fun I wrote a script that can capture speedtest data from both of my Firewalla WAN connections and send the data to a google spreadsheet to make some nice graphs. I'm running it 1/hour at least for now. It can also capture and save to a log file on Firewalla. You can turn off either of these if you don't want one or the other.

Thought I'd share.

https://gist.github.com/.../6a32df2909202c373a6a39063181dc40
1

Comment actions Permalink
Ben Someone

September 03, 2021 01:40
@Michael oh my god that's a ton of beautiful data definitely awesome work there!
0

Comment actions Permalink
Michael Bierman

September 03, 2021 06:49
@Ben you are very kind. Thanks!
-1

Comment actions Permalink