Firewalla Gold Frequently Drops Internet

Comments

16 comments

  • Avatar
    James Willhoite

    No issues here. I'm running 1.973 (19c4b67e) and have not had a single outage. I have a script that runs on a different computer to test my work's internet and will report any outage. It runs every 2 minutes and has not logged that my internet has been out.

    0
    Comment actions Permalink
  • Avatar
    ColoRock

    I’ve found that the outages are just 30 seconds. Typically 20 min apart. Almost clockwork.

    I’ve tried disabling all firewalla features except the most basic firewall rule. My ISP has “reset” my connection on their end. Numerous factory resets including a USB flash reset. I’ve set ethernet port 2 as the WAN port to change the WAN’s MAC address, forcing the ISP to issue a new IP for the device (I have gig fiber, DHCP). Disconnected all devices on my LAN. The issue continues.

    If I have my old router handle DHCP with the ISP and run firewalla in bridge mode, no issue.

    Researching online, the behavior might point to firewalla DHCP “forgetting” the IP address assigned by the ISP and requesting a new one. Are there logs where I could see if this is happening?

    Today I requested a temporary static IP from my ISP hoping to verify this hunch.

    Worth noting that my ISP allows the use of any router.

    Any thoughts or other things to try would be appreciated!

    0
    Comment actions Permalink
  • Avatar
    Firewalla

    I just started a support ticket with you; what you need to tell us is to see if you have any events alarms that shows the ISP is down ... If you do not see that, then we may need logs or even get into the system and check it out. 

    0
    Comment actions Permalink
  • Avatar
    ColoRock

    My ISP updated firmware on their device. Then set their port from auto-negotiate to full duplex 1g as they said this has sometimes fixed similar issues. But, didn’t fix.

    I tried an unmanaged switch between Firewalla and ISP as suggested in similar FWG issue.

    Also didn’t fix.

    I put my old router (no transparent bridge option) between the ISP and firewalla instead. Old router getting IP from ISP via DHCP. Firewalla WAN assigned it’s IP from old router’s subnet. Connection now very stable, but now I’m double NAT, so not ideal…but better.

    Requesting new or different service box from ISP as next step.

    0
    Comment actions Permalink
  • Avatar
    Firewalla

    What is the make and model of your current ISP router? or modem? 

    0
    Comment actions Permalink
  • Avatar
    ColoRock

    Calix GigaPoint 803G. Model 100-04255 10

    0
    Comment actions Permalink
  • Avatar
    Philip

    I had a very similar problem and noted that the firewalla gold was quite hot. I put a small usb computer fan blowing upwards over the gold and it fixed the problem. The first time this happened, I thought maybe I was unlucky and my unit was temperature sensitive. However, I bought another Gold for another office and it did exactly the same thing. 

    0
    Comment actions Permalink
  • Avatar
    Firewalla

    Where did you place the Firewalla? does it have good air circulation? the Gold should not overheat and cause issues unless it gets really really hot. Feel free to contact our support, we can help you 

    0
    Comment actions Permalink
  • Avatar
    ColoRock

    That’s interesting about the heat. I’ve had a 120mm fan running on mine for months just because…no real reason. I still have the issue described in the thread though unless I double NAT.

    0
    Comment actions Permalink
  • Avatar
    Firewalla

    Did you contact support or look at the network events to see why it is dropping? examples here https://help.firewalla.com/hc/en-us/articles/360053534593-How-to-debug-network-connectivity-issues-

     

    0
    Comment actions Permalink
  • Avatar
    AO

    This issue here is what I am facing at the moment. And, I just got my Firewalla Gold yesterday. Super Super regretting it right now given that, I got it outside US and sending it back is pretty not worth it. It drops anytime and anyhow. I tried all modes, the one that allowed me be able to stay the longest before dropping and reconnecting on a video / audio call was the DHCP Mode, which was 20mins.

    I factory reset the device, changed the order of connectivity.

    My connection is as thus:

    - Fibre comes into the house --> Connects to ISP Modem --> Disable DHCP Server on Modem -> Connect Firewalla to ISP Modem -> Express VPN Router Connects to Firewalla which other devices connects to.

    When it's working, everything works fine, but, it's pretty useless when it drops every 20min to 60min

    0
    Comment actions Permalink
  • Avatar
    ColoRock

    That sounds very similar to my issue. While FW tech support hasn’t solved the issue for me, they are very responsive in my experience. Open a ticket with them. Enable remote support (Settings-help and support) to allow them to temporarily look at your logs.

    You might try turning DHCP server back on for your modem, reboot everything. It would get the gateway info/public IP (as it did before) from your ISP, and would just issue a local IP to Firewalla (use router mode). Like my setup, you might end up with a double NAT configuration (double firewall), but I’ve had no performance issues. You just might have to setup some port forwarding rules on your modem if you want to setup VPN on FWG. Annoying, but not hard.

    0
    Comment actions Permalink
  • Avatar
    AO

    @ColoRock well, that works and it has been stable since then. But then, tracing logs to originating device has been pretty impossible 😔😔

    0
    Comment actions Permalink
  • Avatar
    ColoRock

    I’m glad that at least gives you a stable connection while you troubleshoot. Going back to my original post, I still think this is a bug introduced in 1.973 that only affects the Firewalla in rare situations related to the ISP hardware. I’d be willing to flash my FWG with the version before 1.973 to prove it, but FW has said that isn’t an option. Please post updates!

    0
    Comment actions Permalink
  • Avatar
    AO

    So, this is what I did in the last 8hrs, and it has been really stable.

    I use Router mode, but, with custom configuration to connect directly to my Fibre's VLAN with the ID my provider uses for there router, and assigned a port to IPTV as it's required by that setup even though I don't use it.

    So, it looks like this:

     

    And, I disable IPv6, when it was enabled, it was dropping like earlier, every 30min - 60mins. I removed my proxy router, and the ISP's router, Disabled the Mesh's router's DHCP Server, and changed it to AP mode, so that Firewalla assigns the IP. This fixes the double NAT issue.

    And my external proxy, well, I just install the Open VPN profiles on Firewalla.

    Also, one other thing I noticed was that, the Firewalla app was very slow / less responsive, and every action takes 20-30secs to execute. But, with this setup, it's really fast.

    Thanks for the help while trying to sort this out hahaaa.

    0
    Comment actions Permalink
  • Avatar
    ColoRock

    I ended up installing pfSense on another router to see if I couldn't diagnose exactly what was happening since pfSense has great logs/diagnostics. The pfSense router installed with all defaults experienced the same issue I described above.

    I eventually noticed the disconnect with the ISP always occurred when the gateway ARP record had between 200 and 30 seconds remaining (varies), and then the connection with the ISP would immediately come back up when that ARP record expired and a new gateway ARP record was created (at 1200 seconds).

    For a few ARP cycles, I manually deleted the ARP record before it got below 200 (forcing it to send a new ARP request) and I never lost connection with the ISP. So, it appeared my ISP was dropping my connection if it didn't receive an ARP request from me within about 1000 seconds of the last request. But, Firewalla (and pfSense) are set to 1200 as the default interval. I asked my ISP about it, and they said they use "industry standard settings"...so no help there.

    I figured out how to SSH into the Firewalla and edit the user_crontab file following this article and other research on the web: https://help.firewalla.com/hc/en-us/articles/360054056754-Customized-Scripting- (definitely for pros as the article indicates).

    I added this script to the user_crontab file which deletes the gateway ARP record every 10 minutes (forcing a new ARP request to the ISP, similar to me manually deleting):

    */10 * * * * sudo arp -d ###.###.###.###

    If you use this script, replace the #'s with your gateway IP address. While my connection was completely stable with that script, I continued a ticket with Firewalla, sharing what I had learned. They suggested a different script that they put in the user_crontab file for me with remote support enabled (I suggest this route if the above article scares you at all).

    */10 * * * * sudo /usr/bin/nmap -sn $(ip r show default | awk '{print $3}')

    I don't 100% understand their script, but it also works. It sends an ARP request every 10 minutes using nmap to the gateway IP.

    0
    Comment actions Permalink

Please sign in to leave a comment.