Firewalla Gold - Random drops of LAN, low load, no ISP issues
Today my Firewalla Gold has been having huge issues with probably more than 40-50 drops of traffic.
I initially thought it was a problem with my ISP, but checking with them the modem (Motorola MB8600 single connection, not LAGed) is showing full quality, with no dropped packages.
Yes, I have power cycled both the modem and my Firewalla several times, and the problem persists.
I started a timestamped (5 sec) ping on both my local desktop (wired connection to Firewalla LAN) and the Firewalla (SSH + Screen + Ping, so it persists, even if my desktop connection drops).
The result is that the Firewalla WAN sees no drop, while my desktop has a drop of ~20 sec (this time, but drops have random lengths).
Last firmware update was to v1.975 (aca400e3) on Jan. 17, 2023 02:01, so it's not caused by an overnight firmware update. And yes, everything has been working perfectly until today.
As soon as I got online today at ~07:00am, I have had issues.
At this point, I'm not sure how to further debug my Firewalla problem. Any suggestions or logs I should be looking into? And again, I have not done any configuration nor network topology changes since the firmware update.
-
From a PC, ping your firewalla gateway (or fire.walla) and see if you have dropped there. This will test out the LAN side. When you ping to WAN test, find something that's low latency from your site, like your DNS server, and ping that. (or you can check out https://help.firewalla.com/hc/en-us/articles/4413511352083-Network-Performance-and-Quality-Monitoring
-
LAN ICMP should be default ON ... if not tap on settings->advanced->configurration->Block ICMP
You should also follow this and do a few tests https://help.firewalla.com/hc/en-us/articles/360056875493-Speed-test-and-Speed-Optimization-on-Firewalla
-
That worked, thanks.
Doing a bit more digging now, here's what I've found so far:
pi@firewalla:~ (StormFirewall) $ zcat /var/log/syslog.2.gz | grep lost
pi@firewalla:~ (StormFirewall) $ grep -a lost /var/log/syslog.1
Feb 8 07:35:24 firewalla ifplugd(eth0)[24494]: Link beat lost.
Feb 8 13:17:48 firewalla ifplugd(eth0)[2887]: Link beat lost.
Feb 8 13:17:49 firewalla ifplugd(eth2)[2927]: Link beat lost.
Feb 8 13:17:50 firewalla ifplugd(eth3)[2947]: Link beat lost.
Feb 8 07:38:03 firewalla ifplugd(eth0)[2887]: Link beat lost.
Feb 8 07:38:17 firewalla ifplugd(eth0)[2887]: Link beat lost.
Feb 8 07:59:33 firewalla ifplugd(eth0)[2887]: Link beat lost.
Feb 8 07:59:46 firewalla ifplugd(eth0)[2887]: Link beat lost.
Feb 8 08:39:25 firewalla ifplugd(eth2)[2927]: Link beat lost.
Feb 8 08:41:02 firewalla ifplugd(eth3)[2947]: Link beat lost.
Feb 8 14:17:48 firewalla ifplugd(eth0)[2808]: Link beat lost.
Feb 8 14:17:49 firewalla ifplugd(eth2)[2846]: Link beat lost.
pi@firewalla:~ (StormFirewall) $ grep -a lost /var/log/syslog
Feb 8 12:33:07 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 12:53:43 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 13:11:14 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 13:14:44 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 13:50:00 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 14:28:38 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 15:21:33 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 15:50:30 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 15:51:04 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 16:11:56 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 16:12:00 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 16:13:51 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 16:14:44 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 16:15:44 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 16:17:51 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 16:18:03 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 16:18:13 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 18:00:56 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 18:11:30 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 18:13:31 firewalla ifplugd(eth2)[2846]: Link beat lost.
pi@firewalla:~ (StormFirewall) $So, at 7:35 am I started having dropped LAN issues. However, the issues started before then.
Looking at the syslog in detail, the trouble started at 05:56:42:
Feb 8 05:53:20 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 05:53:20 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 05:53:35 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 05:53:35 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 05:53:54 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 05:53:54 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 05:54:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:100244,proxy-1:96704,worker.wg0:130420,worker.br0:313572,worker.tun_fwvpn:130580,
Feb 8 05:54:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:157328, api:121880, main:239340,redis:177916,b7:28796,b6:28796,proc:244, thread:430 main_thread: 23 mon_threads: 23 mainfile:83 monfile:27 sys:2304#0110#011385343 temp:43
Feb 8 05:54:20 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 05:54:20 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 05:54:53 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 05:54:53 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 05:55:02 firewalla pi: Firewalla checkpoint every 5 mins
Feb 8 05:55:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:100244,proxy-1:96704,worker.wg0:130420,worker.br0:313572,worker.tun_fwvpn:130580,
Feb 8 05:55:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:157108, api:121832, main:239380,redis:177916,b7:28624,b6:28624,proc:238, thread:430 main_thread: 23 mon_threads: 23 mainfile:83 monfile:27 sys:2304#0110#011385343 temp:43
Feb 8 05:55:20 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 05:55:20 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 05:55:25 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 05:55:25 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 05:55:53 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 05:55:54 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 05:56:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:100244,proxy-1:96704,worker.wg0:130420,worker.br0:313572,worker.tun_fwvpn:130580,
Feb 8 05:56:03 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:157216, api:121908, main:252436,redis:177916,b7:28624,b6:28624,proc:246, thread:439 main_thread: 23 mon_threads: 23 mainfile:118 monfile:27 sys:2336#0110#011385343 temp:44
Feb 8 05:56:23 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 05:56:23 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 05:56:30 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 05:56:30 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 05:56:42 firewalla kernel: [4985397.072129] net_ratelimit: 30576 callbacks suppressed
Feb 8 05:56:42 firewalla kernel: [4985397.072161] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:42 firewalla kernel: [4985397.074547] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:42 firewalla kernel: [4985397.077076] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:42 firewalla kernel: [4985397.078196] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:42 firewalla kernel: [4985397.079230] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:42 firewalla kernel: [4985397.080573] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:42 firewalla kernel: [4985397.087212] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:42 firewalla kernel: [4985397.088083] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:42 firewalla kernel: [4985397.089168] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:42 firewalla kernel: [4985397.095925] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:51 firewalla kernel: [4985405.969394] net_ratelimit: 982 callbacks suppressed
Feb 8 05:56:51 firewalla kernel: [4985405.969399] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:51 firewalla kernel: [4985405.971744] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:51 firewalla kernel: [4985405.974250] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:51 firewalla kernel: [4985405.976491] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:51 firewalla kernel: [4985405.977471] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:51 firewalla kernel: [4985405.978543] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:51 firewalla kernel: [4985405.979676] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:51 firewalla kernel: [4985405.980798] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:51 firewalla kernel: [4985405.982213] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:52 firewalla kernel: [4985406.991621] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:56:56 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 05:56:56 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 05:57:00 firewalla kernel: [4985414.927859] net_ratelimit: 256 callbacks suppressed
Feb 8 05:57:00 firewalla kernel: [4985414.927863] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:57:00 firewalla kernel: [4985414.929453] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:57:00 firewalla kernel: [4985414.931232] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:57:00 firewalla kernel: [4985414.941609] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:57:00 firewalla kernel: [4985414.942690] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:57:00 firewalla kernel: [4985414.943781] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:57:00 firewalla kernel: [4985414.944845] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:57:00 firewalla kernel: [4985414.945968] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:57:00 firewalla kernel: [4985414.947064] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:57:00 firewalla kernel: [4985414.948145] br0: received packet on eth3 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 05:57:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:100244,proxy-1:96704,worker.wg0:130420,worker.br0:313572,worker.tun_fwvpn:130580,
Feb 8 05:57:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:157540, api:121816, main:240272,redis:177916,b7:28624,b6:28624,proc:238, thread:430 main_thread: 23 mon_threads: 23 mainfile:83 monfile:27 sys:2304#0110#011385343 temp:4320:6d:31:01:35:4c is the MAC address of my Firewalla Port 1.
Port 1 goes to a switch, which in turn goes to three main wired connection points in my house, where they also end up in a switch each and then devices (2x wifi mesh points).
Port 2 goes directly to my office, a switch, then printer, NAS, main computer and a third wifi mesh point.
Port 3 is unused.
Port 4 is WAN.
So, at 05:56:42 all hell breaks loose, and something is spewing out spurious packets, not sure what from. No one had woken up yet, so there was no activity in the house, no new devices turned on.
At 06:30 am I turned on my laptop (Port 2) and noticed networking issues. Intermittent access.
The same messages continue until 07:03:49, when the same notification starts on eth2, but with the same MAC address as Port 1. There's no other connection between the two networks than the Firewalla. The wifi mesh end-points (Eero) are all wired, so they do not wifi mesh, only using the wired back-haul.
At 07:35 am I rebooted the Firewalla.
While services are starting back up, you can see that the issue continues:
Feb 8 07:38:56 firewalla kernel: [ 153.800649] Ebtables v2.0 registered
Feb 8 07:38:56 firewalla systemd[1]: firekick.service: Control process exited, code=exited status=1
Feb 8 07:38:56 firewalla systemd[1]: firekick.service: Failed with result 'exit-code'.
Feb 8 07:38:56 firewalla systemd[1]: firekick.service: Service hold-off time over, scheduling restart.
Feb 8 07:38:56 firewalla systemd[1]: firekick.service: Scheduled restart job, restart counter is at 2.
Feb 8 07:38:56 firewalla systemd[1]: Stopped Firewalla UI.
Feb 8 07:38:56 firewalla systemd[1]: Started Firewalla UI.
Feb 8 07:38:58 firewalla kernel: [ 155.836260] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:38:58 firewalla kernel: [ 155.838318] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:38:58 firewalla kernel: [ 155.839495] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:38:58 firewalla kernel: [ 155.840594] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:38:58 firewalla kernel: [ 155.841873] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:38:58 firewalla kernel: [ 155.842860] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:38:58 firewalla kernel: [ 155.844140] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:38:58 firewalla kernel: [ 155.845243] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:38:58 firewalla kernel: [ 155.846448] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:38:58 firewalla kernel: [ 155.847584] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:38:59 firewalla systemd[1]: Stopping FireRouter DNS...
Feb 8 07:38:59 firewalla systemd[1]: Stopped FireRouter DNS.
Feb 8 07:38:59 firewalla systemd[1]: Started FireRouter DNS.
Feb 8 07:38:59 firewalla pi: FIREWALLA.ACLAUDIT Starting
Feb 8 07:38:59 firewalla pi: FIREWALLA.ACLALARM Starting
Feb 8 07:38:59 firewalla bash[16261]: 27746
Feb 8 07:39:00 firewalla systemd[1]: Stopping Regular background program processing daemon...
Feb 8 07:39:00 firewalla systemd[1]: Stopped Regular background program processing daemon.
Feb 8 07:39:00 firewalla systemd[1]: Started Regular background program processing daemon.
Feb 8 07:39:00 firewalla systemd[1]: Stopping Regular background program processing daemon...
Feb 8 07:39:00 firewalla systemd[1]: Stopped Regular background program processing daemon.
Feb 8 07:39:00 firewalla systemd[1]: Started Regular background program processing daemon.
Feb 8 07:39:00 firewalla systemd[1]: Stopping System Logging Service...
Feb 8 07:39:00 firewalla rsyslogd: [origin software="rsyslogd" swVersion="8.32.0" x-pid="6743" x-info="http://www.rsyslog.com"] exiting on signal 15.
Feb 8 07:39:00 firewalla systemd[1]: Stopped System Logging Service.
Feb 8 07:39:00 firewalla systemd[1]: Starting System Logging Service...
Feb 8 07:39:00 firewalla rsyslogd: imuxsock: Acquired UNIX socket '/run/systemd/journal/syslog' (fd 3) from systemd. [v8.32.0]
Feb 8 07:39:00 firewalla systemd[1]: Started System Logging Service.
Feb 8 07:39:00 firewalla rsyslogd: rsyslogd's groupid changed to 106
Feb 8 07:39:00 firewalla rsyslogd: rsyslogd's userid changed to 102
Feb 8 07:39:00 firewalla rsyslogd: [origin software="rsyslogd" swVersion="8.32.0" x-pid="16817" x-info="http://www.rsyslog.com"] start
Feb 8 07:39:00 firewalla pi: FIREWALLA.ACLALARM Finished Starting
Feb 8 07:39:00 firewalla systemd[1]: Stopping System Logging Service...
Feb 8 07:39:00 firewalla rsyslogd: [origin software="rsyslogd" swVersion="8.32.0" x-pid="16817" x-info="http://www.rsyslog.com"] exiting on signal 15.
Feb 8 07:39:00 firewalla systemd[1]: Stopped System Logging Service.
Feb 8 07:39:00 firewalla systemd[1]: Starting System Logging Service...
Feb 8 07:39:00 firewalla systemd[1]: Started System Logging Service.
Feb 8 07:39:00 firewalla rsyslogd: imuxsock: Acquired UNIX socket '/run/systemd/journal/syslog' (fd 3) from systemd. [v8.32.0]
Feb 8 07:39:00 firewalla rsyslogd: rsyslogd's groupid changed to 106
Feb 8 07:39:00 firewalla rsyslogd: rsyslogd's userid changed to 102
Feb 8 07:39:00 firewalla rsyslogd: [origin software="rsyslogd" swVersion="8.32.0" x-pid="16940" x-info="http://www.rsyslog.com"] start
Feb 8 07:39:01 firewalla pi: FIREWALLA.ACLAUDIT Finished Starting
Feb 8 07:39:03 firewalla kernel: [ 160.838199] net_ratelimit: 3495 callbacks suppressed
Feb 8 07:39:03 firewalla kernel: [ 160.838203] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:03 firewalla kernel: [ 160.841984] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:03 firewalla kernel: [ 160.842996] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:03 firewalla kernel: [ 160.844201] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:03 firewalla kernel: [ 160.846356] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:03 firewalla kernel: [ 160.847769] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:03 firewalla kernel: [ 160.848714] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:03 firewalla kernel: [ 160.850266] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:03 firewalla kernel: [ 160.851567] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:03 firewalla kernel: [ 160.860680] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:04 firewalla systemd[1]: Stopping Regular background program processing daemon...
Feb 8 07:39:04 firewalla systemd[1]: Stopped Regular background program processing daemon.
Feb 8 07:39:04 firewalla systemd[1]: Started Regular background program processing daemon.
Feb 8 07:39:05 firewalla systemd[1]: Stopping System Logging Service...
Feb 8 07:39:05 firewalla rsyslogd: [origin software="rsyslogd" swVersion="8.32.0" x-pid="16940" x-info="http://www.rsyslog.com"] exiting on signal 15.
Feb 8 07:39:05 firewalla systemd[1]: Stopped System Logging Service.
Feb 8 07:39:05 firewalla systemd[1]: Starting System Logging Service...
Feb 8 07:39:05 firewalla rsyslogd: imuxsock: Acquired UNIX socket '/run/systemd/journal/syslog' (fd 3) from systemd. [v8.32.0]
Feb 8 07:39:05 firewalla rsyslogd: rsyslogd's groupid changed to 106
Feb 8 07:39:05 firewalla systemd[1]: Started System Logging Service.
Feb 8 07:39:05 firewalla rsyslogd: rsyslogd's userid changed to 102
Feb 8 07:39:05 firewalla rsyslogd: [origin software="rsyslogd" swVersion="8.32.0" x-pid="18136" x-info="http://www.rsyslog.com"] start
Feb 8 07:39:06 firewalla systemd[1]: firekick.service: Control process exited, code=exited status=1
Feb 8 07:39:06 firewalla systemd[1]: firekick.service: Failed with result 'exit-code'.
Feb 8 07:39:06 firewalla systemd[1]: firekick.service: Service hold-off time over, scheduling restart.
Feb 8 07:39:06 firewalla systemd[1]: firekick.service: Scheduled restart job, restart counter is at 3.
Feb 8 07:39:06 firewalla systemd[1]: Stopped Firewalla UI.
Feb 8 07:39:06 firewalla systemd[1]: Started Firewalla UI.
Feb 8 07:39:08 firewalla kernel: [ 165.842295] net_ratelimit: 9205 callbacks suppressed
Feb 8 07:39:08 firewalla kernel: [ 165.842300] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:08 firewalla kernel: [ 165.842315] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:08 firewalla kernel: [ 165.842333] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:08 firewalla kernel: [ 165.842355] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:08 firewalla kernel: [ 165.843481] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:08 firewalla kernel: [ 165.843495] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:08 firewalla kernel: [ 165.843509] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:08 firewalla kernel: [ 165.843538] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:08 firewalla kernel: [ 165.844623] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:08 firewalla kernel: [ 165.844639] br0: received packet on eth2 with own address as source address (addr:20:6d:31:01:35:4c, vlan:0)
Feb 8 07:39:13 firewalla kernel: [ 170.847671] net_ratelimit: 10002 callbacks suppressedI do a few more reboots of both the Firewalla and the Modem, and eventually I unplug everything on Port 1, leaving only the office connected.
The spurious "received packet on [ethX] with own address as source address" messages stop, but now I start getting the momentary LAN drops:
Feb 8 12:31:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101404,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:31:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:98888, api:97800, main:177028,redis:147168,b7:3064,b6:3064,proc:244, thread:424 main_thread: 23 mon_threads: 23 mainfile:79 monfile:27 sys:2464#0110#011385342 temp:42
Feb 8 12:32:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101404,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:32:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:100920, api:97792, main:176064,redis:147172,b7:3064,b6:3064,proc:247, thread:424 main_thread: 23 mon_threads: 23 mainfile:82 monfile:27 sys:2496#0110#011385342 temp:42
Feb 8 12:33:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101404,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:33:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:101940, api:97712, main:177500,redis:147184,b7:3064,b6:3064,proc:244, thread:424 main_thread: 23 mon_threads: 23 mainfile:79 monfile:27 sys:2464#0110#011385342 temp:42
Feb 8 12:33:06 firewalla kernel: [13856.062398] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 12:33:06 firewalla kernel: [13856.063480] br0: port 2(eth2) entered disabled state
Feb 8 12:33:07 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 12:33:09 firewalla ntpd[7746]: Deleting interface #6 br0, 10.9.8.7#123, interface stats: received=0, sent=0, dropped=0, active_time=13766 secs
Feb 8 12:33:10 firewalla kernel: [13859.218799] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 12:33:10 firewalla kernel: [13859.326531] br0: port 2(eth2) entered blocking state
Feb 8 12:33:10 firewalla kernel: [13859.326537] br0: port 2(eth2) entered listening state
Feb 8 12:33:10 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 12:33:25 firewalla kernel: [13874.402206] br0: port 2(eth2) entered learning state
Feb 8 12:33:40 firewalla kernel: [13889.502222] br0: port 2(eth2) entered forwarding state
Feb 8 12:33:40 firewalla kernel: [13889.502227] br0: topology change detected, propagating
Feb 8 12:33:41 firewalla ntpd[7746]: Listen normally on 10 br0 10.9.8.7:123
Feb 8 12:33:41 firewalla ntpd[7746]: new interface(s) found: waking up resolver
Feb 8 12:34:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101404,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:34:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:101996, api:97988, main:186844,redis:147192,b7:3064,b6:3064,proc:248, thread:425 main_thread: 23 mon_threads: 23 mainfile:82 monfile:27 sys:2464#0110#011385342 temp:43
Feb 8 12:35:01 firewalla pi: Firewalla checkpoint every 5 mins
Feb 8 12:35:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101404,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:35:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:101848, api:97940, main:174068,redis:147192,b7:3064,b6:3064,proc:245, thread:425 main_thread: 23 mon_threads: 23 mainfile:80 monfile:27 sys:2496#0110#011385342 temp:42
Feb 8 12:36:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101404,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:36:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:102548, api:97996, main:176040,redis:147192,b7:3064,b6:3064,proc:244, thread:424 main_thread: 23 mon_threads: 23 mainfile:80 monfile:27 sys:2496#0110#011385342 temp:43
Feb 8 12:37:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101404,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:37:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:103632, api:97896, main:175420,redis:147192,b7:3064,b6:3064,proc:244, thread:424 main_thread: 23 mon_threads: 23 mainfile:80 monfile:27 sys:2464#0110#011385342 temp:43again
Feb 8 12:33:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101404,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:33:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:101940, api:97712, main:177500,redis:147184,b7:3064,b6:3064,proc:244, thread:424 main_thread: 23 mon_threads: 23 mainfile:79 monfile:27 sys:2464#0110#011385342 temp:42
Feb 8 12:33:06 firewalla kernel: [13856.062398] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 12:33:06 firewalla kernel: [13856.063480] br0: port 2(eth2) entered disabled state
Feb 8 12:33:07 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 12:33:09 firewalla ntpd[7746]: Deleting interface #6 br0, 10.9.8.7#123, interface stats: received=0, sent=0, dropped=0, active_time=13766 secs
Feb 8 12:33:10 firewalla kernel: [13859.218799] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 12:33:10 firewalla kernel: [13859.326531] br0: port 2(eth2) entered blocking state
Feb 8 12:33:10 firewalla kernel: [13859.326537] br0: port 2(eth2) entered listening state
Feb 8 12:33:10 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 12:33:25 firewalla kernel: [13874.402206] br0: port 2(eth2) entered learning state
Feb 8 12:33:40 firewalla kernel: [13889.502222] br0: port 2(eth2) entered forwarding state
Feb 8 12:33:40 firewalla kernel: [13889.502227] br0: topology change detected, propagating
Feb 8 12:33:41 firewalla ntpd[7746]: Listen normally on 10 br0 10.9.8.7:123
Feb 8 12:33:41 firewalla ntpd[7746]: new interface(s) found: waking up resolver
Feb 8 12:34:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101404,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:34:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:101996, api:97988, main:186844,redis:147192,b7:3064,b6:3064,proc:248, thread:425 main_thread: 23 mon_threads: 23 mainfile:82 monfile:27 sys:2464#0110#011385342 temp:43and again
Feb 8 12:53:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101408,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:53:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:103056, api:98020, main:177572,redis:147212,b7:3064,b6:3064,proc:243, thread:423 main_thread: 23 mon_threads: 23 mainfile:80 monfile:27 sys:2464#0110#011385342 temp:43
Feb 8 12:53:43 firewalla kernel: [15092.247201] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 12:53:43 firewalla kernel: [15092.248513] br0: port 2(eth2) entered disabled state
Feb 8 12:53:43 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 12:53:45 firewalla ntpd[7746]: Deleting interface #10 br0, 10.9.8.7#123, interface stats: received=0, sent=0, dropped=0, active_time=1204 secs
Feb 8 12:53:46 firewalla kernel: [15095.363606] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 12:53:46 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 12:53:46 firewalla kernel: [15095.471384] br0: port 2(eth2) entered blocking state
Feb 8 12:53:46 firewalla kernel: [15095.471391] br0: port 2(eth2) entered listening state
Feb 8 12:54:01 firewalla kernel: [15110.623028] br0: port 2(eth2) entered learning state
Feb 8 12:54:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101408,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:54:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:103312, api:98000, main:176184,redis:147264,b7:3064,b6:3064,proc:250, thread:424 main_thread: 23 mon_threads: 23 mainfile:80 monfile:27 sys:2496#0110#011385342 temp:43
Feb 8 12:54:16 firewalla kernel: [15125.727043] br0: port 2(eth2) entered forwarding state
Feb 8 12:54:16 firewalla kernel: [15125.727047] br0: topology change detected, propagating
Feb 8 12:54:17 firewalla ntpd[7746]: Listen normally on 11 br0 10.9.8.7:123
Feb 8 12:54:17 firewalla ntpd[7746]: new interface(s) found: waking up resolver
Feb 8 12:54:30 firewalla systemd[1]: Reloading FireRouter DNS.
Feb 8 12:54:30 firewalla systemd[1]: Reloaded FireRouter DNS.
Feb 8 12:55:01 firewalla pi: Firewalla checkpoint every 5 mins
Feb 8 12:55:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101408,proxy-1:96516,worker.wg0:130288,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 12:55:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:102756, api:98080, main:178996,redis:147264,b7:3064,b6:3064,proc:244, thread:424 main_thread: 23 mon_threads: 23 mainfile:80 monfile:27 sys:2464#0110#011385342 temp:43and again..
Feb 8 13:11:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101412,proxy-1:96516,worker.wg0:130336,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 13:11:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:104992, api:98148, main:180500,redis:147284,b7:1228,b6:1228,proc:238, thread:418 main_thread: 23 mon_threads: 23 mainfile:77 monfile:27 sys:2400#0110#011385342 temp:43
Feb 8 13:11:14 firewalla kernel: [16143.931391] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 13:11:14 firewalla kernel: [16143.932343] br0: port 2(eth2) entered disabled state
Feb 8 13:11:14 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 13:11:17 firewalla ntpd[7746]: Deleting interface #11 br0, 10.9.8.7#123, interface stats: received=0, sent=0, dropped=0, active_time=1020 secs
Feb 8 13:11:17 firewalla kernel: [16147.011824] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 13:11:17 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 13:11:18 firewalla kernel: [16147.119535] br0: port 2(eth2) entered blocking state
Feb 8 13:11:18 firewalla kernel: [16147.119540] br0: port 2(eth2) entered listening state
Feb 8 13:11:33 firewalla kernel: [16162.271249] br0: port 2(eth2) entered learning state
Feb 8 13:11:48 firewalla kernel: [16177.375200] br0: port 2(eth2) entered forwarding state
Feb 8 13:11:48 firewalla kernel: [16177.375205] br0: topology change detected, propagating
Feb 8 13:11:49 firewalla ntpd[7746]: Listen normally on 12 br0 10.9.8.7:123
Feb 8 13:11:49 firewalla ntpd[7746]: new interface(s) found: waking up resolver
Feb 8 13:12:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101412,proxy-1:96516,worker.wg0:130336,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 13:12:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:101912, api:98064, main:190500,redis:147300,b7:3064,b6:3064,proc:245, thread:419 main_thread: 23 mon_threads: 23 mainfile:77 monfile:27 sys:2432#0110#011385342 temp:43
Feb 8 13:13:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101412,proxy-1:96516,worker.wg0:130336,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 13:13:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:104440, api:97932, main:186636,redis:147300,b7:3064,b6:3064,proc:239, thread:419 main_thread: 23 mon_threads: 23 mainfile:77 monfile:27 sys:2368#0110#011385342 temp:43
Feb 8 13:13:34 firewalla systemd[1]: Started Session 2268 of user pi.
Feb 8 13:14:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101412,proxy-1:96516,worker.wg0:130336,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 13:14:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:105084, api:103340, main:186320,redis:147300,b7:3064,b6:3064,proc:249, thread:427 main_thread: 23 mon_threads: 23 mainfile:77 monfile:27 sys:2496#0110#011385342 temp:42
Feb 8 13:14:43 firewalla kernel: [16352.675346] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 13:14:43 firewalla kernel: [16352.675910] br0: port 2(eth2) entered disabled state
Feb 8 13:14:44 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 13:14:45 firewalla ntpd[7746]: Deleting interface #12 br0, 10.9.8.7#123, interface stats: received=0, sent=0, dropped=0, active_time=176 secs
Feb 8 13:14:46 firewalla kernel: [16355.807901] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 13:14:46 firewalla kernel: [16356.023531] br0: port 2(eth2) entered blocking state
Feb 8 13:14:46 firewalla kernel: [16356.023539] br0: port 2(eth2) entered listening state
Feb 8 13:14:47 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 13:15:01 firewalla pi: Firewalla checkpoint every 5 mins
Feb 8 13:15:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101420,proxy-1:96516,worker.wg0:130336,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 13:15:02 firewalla kernel: [16371.167125] br0: port 2(eth2) entered learning state
Feb 8 13:15:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:133444, api:98464, main:183064,redis:147472,b7:3064,b6:3064,proc:244, thread:425 main_thread: 23 mon_threads: 23 mainfile:77 monfile:27 sys:2464#0110#011385342 temp:43
Feb 8 13:15:17 firewalla kernel: [16386.271250] br0: port 2(eth2) entered forwarding state
Feb 8 13:15:17 firewalla kernel: [16386.271255] br0: topology change detected, propagating
Feb 8 13:15:18 firewalla ntpd[7746]: Listen normally on 13 br0 10.9.8.7:123
Feb 8 13:15:18 firewalla ntpd[7746]: new interface(s) found: waking up resolver
Feb 8 13:16:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:101420,proxy-1:96516,worker.wg0:130336,worker.br0:246684,worker.tun_fwvpn:130596,
Feb 8 13:16:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:101000, api:98492, main:185284,redis:147472,b7:3064,b6:3064,proc:255, thread:430 main_thread: 23 mon_threads: 23 mainfile:87 monfile:27 sys:2560#0110#011385342 temp:43eventually, in the evening, I plug in Port 1 again and I only see 6 of the spurious messages, then it stops. Weird.
Then suddenly I get a completely flustered Firewalla, like it's having a serious coughing fit:
Feb 8 18:57:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:99952,proxy-1:96472,worker.tun_fwvpn:130256,worker.wg0:129876,worker.br0:199396,
Feb 8 18:57:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:106584, api:104236, main:202928,redis:155232,b7:3064,b6:3064,proc:245, thread:423 main_thread: 23 mon_threads: 23 mainfile:82 monfile:27 sys:2496#0110#011385342 temp:41
Feb 8 18:57:52 firewalla kernel: [36941.771034] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 18:57:52 firewalla kernel: [36941.772182] br0: port 2(eth2) entered disabled state
Feb 8 18:57:52 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 18:57:54 firewalla ntpd[7746]: Deleting interface #25 br0, 10.9.8.7#123, interface stats: received=0, sent=0, dropped=0, active_time=2628 secs
Feb 8 18:57:55 firewalla kernel: [36944.919407] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 18:57:55 firewalla kernel: [36945.027208] br0: port 2(eth2) entered blocking state
Feb 8 18:57:55 firewalla kernel: [36945.027216] br0: port 2(eth2) entered listening state
Feb 8 18:57:56 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 18:58:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:99952,proxy-1:96472,worker.tun_fwvpn:130256,worker.wg0:129884,worker.br0:199396,
Feb 8 18:58:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:109124, api:104388, main:203356,redis:155232,b7:3064,b6:3064,proc:252, thread:424 main_thread: 23 mon_threads: 23 mainfile:82 monfile:27 sys:2560#0110#011385342 temp:40
Feb 8 18:58:11 firewalla kernel: [36960.234836] br0: port 2(eth2) entered learning state
Feb 8 18:58:26 firewalla kernel: [36975.338850] br0: port 2(eth2) entered forwarding state
Feb 8 18:58:26 firewalla kernel: [36975.338855] br0: topology change detected, propagating
Feb 8 18:58:27 firewalla ntpd[7746]: Listen normally on 26 br0 10.9.8.7:123
Feb 8 18:58:27 firewalla ntpd[7746]: new interface(s) found: waking up resolver
Feb 8 18:59:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:100084,proxy-1:96472,worker.tun_fwvpn:130256,worker.wg0:129884,worker.br0:199396,
Feb 8 18:59:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:107028, api:104256, main:204516,redis:155232,b7:3064,b6:3064,proc:247, thread:426 main_thread: 23 mon_threads: 23 mainfile:82 monfile:27 sys:2528#0110#011385342 temp:41
Feb 8 19:00:01 firewalla pi: Firewalla checkpoint every 5 mins
Feb 8 19:00:01 firewalla pi: FIREWALLA: REDIS DATA CLEAN
Feb 8 19:00:02 firewalla pi: No reboot needed
Feb 8 19:00:02 firewalla pi: FIREWALLA:APPLY_PROFILE:START
Feb 8 19:00:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:100200,proxy-1:96472,worker.tun_fwvpn:130256,worker.wg0:129884,worker.br0:199396,
Feb 8 19:00:02 firewalla pi: FIREWALLA:APPLY_PROFILE:DONE
Feb 8 19:00:03 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:135892, api:104472, main:203984,redis:155240,b7:3064,b6:3064,proc:251, thread:430 main_thread: 23 mon_threads: 23 mainfile:82 monfile:27 sys:2496#0110#011385342 temp:40
Feb 8 19:01:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:100200,proxy-1:96472,worker.tun_fwvpn:130256,worker.wg0:129884,worker.br0:199396,
Feb 8 19:01:01 firewalla pi: FIREWALLA:UPDATE_ASSETS:START
Feb 8 19:01:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:105580, api:104308, main:195772,redis:155240,b7:3064,b6:3064,proc:256, thread:441 main_thread: 23 mon_threads: 23 mainfile:82 monfile:27 sys:2592#0110#011385342 temp:41
Feb 8 19:01:08 firewalla pi: FIREWALLA:PATCH_SYSTEM:START
Feb 8 19:01:09 firewalla pi: FIREWALLA:PATCH_SYSTEM:DONE
Feb 8 19:01:09 firewalla pi: FIREWALLA:UPDATE_ASSETS:DONE
Feb 8 19:01:20 firewalla kernel: [37150.075239] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 19:01:20 firewalla kernel: [37150.075836] br0: port 2(eth2) entered disabled state
Feb 8 19:01:21 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 19:01:23 firewalla ntpd[7746]: Deleting interface #26 br0, 10.9.8.7#123, interface stats: received=0, sent=0, dropped=0, active_time=176 secs
Feb 8 19:01:24 firewalla kernel: [37153.379635] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 19:01:24 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 19:01:24 firewalla kernel: [37153.487480] br0: port 2(eth2) entered blocking state
Feb 8 19:01:24 firewalla kernel: [37153.487491] br0: port 2(eth2) entered listening state
Feb 8 19:01:24 firewalla kernel: [37154.059246] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 19:01:25 firewalla kernel: [37154.507267] br0: port 2(eth2) entered disabled state
Feb 8 19:01:25 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 19:01:28 firewalla kernel: [37157.191655] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 19:01:30 firewalla kernel: [37159.367158] igb 0000:03:00.0: exceed max 2 second
Feb 8 19:01:30 firewalla kernel: [37159.372255] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 19:01:30 firewalla ifplugd(eth2)[2846]: Executing '/etc/ifplugd/ifplugd.action eth2 down'.
Feb 8 19:01:30 firewalla ifplugd(eth2)[2846]: client: 1
Feb 8 19:01:30 firewalla ifplugd(eth2)[2846]: Program executed successfully.
Feb 8 19:01:31 firewalla kernel: [37160.767661] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 19:01:33 firewalla kernel: [37162.931193] igb 0000:03:00.0: exceed max 2 second
Feb 8 19:01:33 firewalla kernel: [37162.936328] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 19:01:35 firewalla kernel: [37164.315659] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 19:01:35 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 19:01:36 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 19:01:37 firewalla kernel: [37166.491158] igb 0000:03:00.0: exceed max 2 second
Feb 8 19:01:37 firewalla kernel: [37166.496236] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 19:01:38 firewalla kernel: [37167.951685] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 19:01:38 firewalla kernel: [37168.059459] br0: port 2(eth2) entered blocking state
Feb 8 19:01:38 firewalla kernel: [37168.059466] br0: port 2(eth2) entered listening state
Feb 8 19:01:39 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 19:01:44 firewalla kernel: [37173.191247] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 19:01:44 firewalla kernel: [37173.191438] br0: port 2(eth2) entered disabled state
Feb 8 19:01:44 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 19:01:47 firewalla kernel: [37176.439668] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 19:01:47 firewalla kernel: [37176.547597] br0: port 2(eth2) entered blocking state
Feb 8 19:01:47 firewalla kernel: [37176.547604] br0: port 2(eth2) entered listening state
Feb 8 19:01:47 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 19:01:51 firewalla kernel: [37180.763276] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 19:01:51 firewalla kernel: [37180.763520] br0: port 2(eth2) entered disabled state
Feb 8 19:01:52 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 19:01:54 firewalla kernel: [37183.951679] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 19:01:55 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 19:01:56 firewalla ifplugd(eth2)[2846]: Link beat lost.
Feb 8 19:01:57 firewalla kernel: [37186.123200] igb 0000:03:00.0: exceed max 2 second
Feb 8 19:01:57 firewalla kernel: [37186.128291] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Down
Feb 8 19:01:58 firewalla kernel: [37187.499677] igb 0000:03:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Feb 8 19:01:58 firewalla kernel: [37187.607492] br0: port 2(eth2) entered blocking state
Feb 8 19:01:58 firewalla kernel: [37187.607498] br0: port 2(eth2) entered listening state
Feb 8 19:01:59 firewalla ifplugd(eth2)[2846]: Link beat detected.
Feb 8 19:02:01 firewalla pi: FIREWALLA:(debug): FIREWALLA: Bro Memory Stats manager:100200,proxy-1:96472,worker.tun_fwvpn:130256,worker.wg0:129884,worker.br0:199396,
Feb 8 19:02:02 firewalla pi: FIREWALLA:(debug): FIREWALLA: Memory mon:106360, api:104244, main:195516,redis:155240,b7:3064,b6:3064,proc:254, thread:427 main_thread: 23 mon_threads: 23 mainfile:82 monfile:27 sys:2528#0110#011385342 temp:41which continues for at least twice as long, followed by an hour of "peace"...
I'm stumped. What's going on??
Please sign in to leave a comment.
Comments
6 comments