NextDNS CLI on Firewalla revisited (working DHCP host resolution in NextDNS logs)
I ended up rolling my own solution after running into issues with another post here regarding setting up NextDNS CLI on a Firewalla -- I was having problems where setup-router=true wasn't working for me, causing dnsmasq and NextDNS CLI to fight over port 53.
Below is what is working for me on my Firewalla Purple (box v1.974), including device identification in NextDNS logs:
Update:
Nice [un]install scripts provided by Michael Bierman here: https://github.com/mbierman/Firewalla-NextDNS-CLI-install
#!/bin/bash
# install & configure NextDNS CLI on startup of Firewalla
# file goes in: /home/pi/.firewalla/config/post_main.d/
# DNS over HTTPS must be disabled in Firewalla app
# install NextDNS CLI
sudo wget -qO /usr/share/keyrings/nextdns.gpg https://repo.nextdns.io/nextdns.gpg
echo "deb [signed-by=/usr/share/keyrings/nextdns.gpg] https://repo.nextdns.io/deb stable main" | sudo tee /etc/apt/sources.list.d/nextdns.list
unalias apt
sudo apt update
sudo apt install nextdns
# replace -config 123456 with your own NextDNS config ID
# replace 10.10.12.1 with your Firewalla local lan IP
# enable NextDNS caching: https://github.com/nextdns/nextdns/wiki/Cache-Configuration
# set discovery-dns to IP of Firewalla local DNS
# set NextDNS CLI to listen on local network IP (instead of 127.0.0.1 -- allows DHCP host resolution in NextDNS logs)
# define listen port instead of relying on -setup-router
sudo nextdns install -config 123456 -report-client-info -cache-size=10MB -max-ttl=5s -discovery-dns 10.10.12.1 -listen 10.10.12.1:5555
# alternate command to implement conditional configuration: https://github.com/nextdns/nextdns/wiki/Conditional-Configuration
# replace 192.168.122.0/24=abcdef with your own additional network and NextDNS config ID
# sudo nextdns install -config 192.168.122.0/24=abcdef -config 123456 -report-client-info -cache-size=10MB -max-ttl=5s -discovery-dns 10.10.12.1 -listen 10.10.12.1:5555
# Add dnsmasq integration to enable client reporting in NextDNS logs: https://github.com/nextdns/nextdns/wiki/DNSMasq-Integration
cat > /home/pi/.firewalla/config/dnsmasq/mynextdns.conf << EOF
server=10.10.12.1#5555
add-mac
add-subnet=32,128
EOF
# restart Firewalla DNS service
sudo systemctl restart firerouter_dns.service
NextDNS CLI seems to hijack all DNS requests, hence the alternate config option (I created a new NextDNS config to use with devices on one of my VLANs). The DNS servers I specified in the VLAN network config were being ignored (devices show the DNS servers specified by the VLAN config, but NextDNS was still logging/blocking DNS requests to these devices).
Note that NextDNS seems to truncate device names after the first period. You can replace spaces with hyphens or remove them altogether in the Firewalla app for a more informational device name in the NextDNS logs (Firewalla converts spaces in devices names to periods).
There's more I'm probably forgetting to note, but I'm running out of time and have to go prep some food for tomorrow's 4th family celebration. I will update at a later time if something comes to mind, or if there's questions.
-
My firewalla is servicing a small home office environment, so local DNS resolution wasn't/isn't all that important to me. I can tell you that local DNS resolution (forward and reverse) is working fine (for me anyway) and I don't have anything special configured (i.e., Unbound). Clients are using the firewalla as the DNS server entry.
Here's a ping and DNS lookup to a local wireless printer:
➜ ~ ping -c 3 canon.lan
PING canon.lan (10.10.12.135) 56(84) bytes of data.
64 bytes from canon.lan (10.10.12.135): icmp_seq=1 ttl=63 time=9.18 ms
64 bytes from canon.lan (10.10.12.135): icmp_seq=2 ttl=63 time=8.85 ms
64 bytes from canon.lan (10.10.12.135): icmp_seq=3 ttl=63 time=8.11 ms
--- canon.lan ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 8.113/8.714/9.183/0.446 ms➜ ~ host canon.lan
canon.lan has address 10.10.12.135
➜ ~ host 10.10.12.135
135.12.10.10.in-addr.arpa domain name pointer canon.lan.The only oddity I came across is that the "firewalla.lan" forward lookup entry pulls an IP from my first VLAN network (192.168.122.1) instead of the primary network (10.10.12.1). I'm not sure if this is related to the NextDNS setup. Doesn't bother me enough to dig into it further.
➜ ~ host firewalla.lan
firewalla.lan has address 192.168.122.1
➜ ~ host 10.10.12.1
Host 1.12.10.10.in-addr.arpa. not found: 3(NXDOMAIN) -
@Brian nice job!
I fixed a couple of minor issues I found with your script and expanded it so that it is even easier to deploy and uninstall. If anyone is interested, https://github.com/mbierman/Firewalla-NextDNS-CLI-install -
@Michael
Thanks for the extended scripts! I have added a link in the first post.
You mentioned that you fixed some minor issues, but I don't seem to be able to detect what changes were made to the install/config commands (at least, through a simple diff). If you could indicate what errors I may have made, I'll be happy to update the original post with the appropriate corrections.
-
@Brian,
I remember one important one was to unalias apt. Firewalla will halt install if you don't.
I made it so it is just a copy/paste install... or nearly. I haven't asked people to input their ID and IP at install so they don't have to edit the file...maybe I will add later. But if they don't set those, they are warned and it exits. Also, I used variables so they only have to set those in one place.
And of course, the uninstall so it can be removed as easily as it is added. -
@Michael
Thanks for the clarification, I don't recall having to do the unalias step (which doesn't mean much -- most times I don't remember what I had for breakfast...).
I like the added conditionals and variables. Seems like a slick setup you created. Adding user input for ID and IP would be icing. However, you'd also expect someone to be able to simply fill in a couple of variables in their favorite text editor if they're even thinking of getting this intimate with their Firewalla.
Again, thanks for taking things as far as you did!
-
@Brian,
That was my thought too. Also, I have to think about how to ask for missing info elegantly, because they could be installing from curl, or running on their Firewalla... several "entry" points to consider. The asking isn't hard, but handling those corner cases will take a little thought. Meanwhile, this is pretty usable.
-
I've updated the script(s) in several ways:
1. Split the install script and a new script to test if nextdns is running. I use a cronjob to check every 5 minutes and alert me if anything is wrong.
2. Improved the uninstall script to remove all traces of the install.
Feedback is welcome. So far, this seems to be working nicely. Still more to play with, when I get time.
-
I'm seeing something odd. The conditional configs based on MAC address seem to be getting ignored. Does order in the config file matter? In other words, if the default profile is listed first in the config line, then the conditional ones. This seemed work when I had the cli on other devices.
-
Anyone experiencing odd behavior with this configuration where it seems to stop resolving queries? After running for a couple weeks with NextDNS cli installed on the Purple, I was having all sorts of slowdowns and outages, and they all seem to be DNS related. Wasn't seeing this when the Cli was on a Synology and a Linux box.
-
@Richard so I have played with it a lot in the last couple of weeks. The order does seem to matter. Here's my example. You need to fill in your own IPs of course.
73 sudo nextdns install \
74 -config ${IP}/24=${id} \
75 -config $OpenVPN/24=$VPNID \
76 -config $WireGuard/24=$VPNID \
77 -config -report-client-info -cache-size=10MB -max-ttl=5s -discovery-dns ${IP} -listen ${IP}:5555All the networks and mac address come before the line 77.
I have a script that runs every 5 minutes to see if nextdns is up. several times a day nextdns reports it is not running and my script restarts it. It has been solid since this.github.com/mbierman/Firewalla-NextDNS-CLI-install
-
Thanks.
Just an FYI - but when I run the script as per the documentation, I get some errors. When I login to the Purple, I'm logged in as user "pi". If I attempt to follow your instructions per github,
curl -s -L -C- https://raw.githubusercontent.com/mbierman/Firewalla-NextDNS-CLI-install/main/install_nextdnscli.sh | cat <(cat <(bash))
I'll get an error:
!/bin/bash - no such file or directory
So I just manually download the script, place it in the correct directory, and sudo chmod +x and then run it with sudo ./install_nextdnscli.sh
I then see this:
pi@Firewalla:~/.firewalla/config/post_main.d (Firewalla) $ sudo ./install_nextdnscli.sh
./install_nextdnscli.sh: 1: !/bin/bash: not found
./install_nextdnscli.sh: 18: [[: not found
./install_nextdnscli.sh: 21: [[: not foundI've ignored these and it seems to install, but wanted to let you know.
-
I am also looking into moving NextDNS CLI from my Syno NAS to my new FW Purple. On the NAS it has been running rock solid for months, assigned through dhcp. Reading the comment above unfortunately indicate it's not a straightforward as I hoped.
So my concerns are:
- Is there any possible explanation why it would just stop (unfortunately my Linux knowledge is way too limited to feel comfortable just trying it)? IMO DNS availability is just too important to rely on a restart-script on the Purple if it runs fine on my NAS.
- Could/will a Purple FW upgade mess up Nextdns? Or maybe even toggling a DNS setting from the FW app?Maybe others who have tried it can chime in?
-
@Michael,
That's strange. I've been running NextDNS for around two months now w/o issue* using my original method noted above (it works and I'm to busy (lazy?) to change it out for your solution).
*The only times I've run into DNS problems seem to coincide with NextDNS service issues. However, that's hard to confirm since NextDNS doesn't run a public status page/archive. The two times I've run into issues and had time to try and diagnose, I also had problems accessing the NextDNS admin portal. Therefore, I concluded that there was a larger problem at hand which I had no control of. Fortunately, the issues didn't seem to last more than a couple minutes and were resolved automatically (including access to the NextDNS portal), without further troubleshooting from me.
I'll be installing another Firewalla in my FL home in a month or so. I'll give your script a go then and report back if I run into any problems.
-
@Michael,
I just looked over your GitHub issue thread. I vaguely remember reading somewhere that restarting the main firewalla service (IIRC) will also cause all the scripts in the post_main.d/ folder to be executed again.
Judging by the logfile excerpt you posted over there, it seems as though it's a timed restart (every 10 minutes?). Is there something else telling your firewalla to do a soft restart, thereby re-invoking the NextDNS startup script?
-
@Brian, Thanks.
By default I'm not sure you would see any issue. My script runs
sudo nextdns status
That is the only way I know something is wrong. I haven't been able to find any time when DNS isn't working (at least not that I can prove or find another explanation for).
You are correct that when there's a Firewalla restart, all script in ~/.firewalla/config/post_main.d/*.sh automatically run.
I don't know why that would be happening so often during the day. I'm going to do two things:
- Remove the nextdns script in there temporarily so it doesn't run.
- Create a script that will log every time Firewalla restarts to see if there is any correlation.
Update: I forgot I already have a script that logs every FW reboot. There is no correlation to the times nextdns fails and reboots. I have removed the script through to see what happens.
Also, if i repeatedly run
sudo nextdns status
I can get it to fail about 25% of the time. I suspect some issue with it.
Update 2: I wildly over estimated from a small sample size. I ran status 100 times with a 3 second delay between. 7/100 failed. Still, the number should be zero.
Please sign in to leave a comment.
Comments
70 comments