netdata docker on FWGold
Pinned
Hi there!
New user of FWG, and loving it so far. I immediately started tinkering with docker to run a few things that my old pfSense box had, and the one I really miss is its netdata dashboard. I've got the netdata docker running, but can't seem to access it unless I set it to use host mode networking, but then there's no network data in the dashboard, only CPU. Is that a limitation of FWG, or am I missing a docker config or what?
Thanks!
-
I tried it out and here it is
/home/pi/.firewalla/run/docker/netdata/docker-compose.yaml
version: '3.4'
services:
netdata:
image: netdata/netdata
container_name: netdata
hostname: ThePatelsFWG
ports:
- 19999:19999
restart: unless-stopped
cap_add:
- SYS_PTRACE
security_opt:
- apparmor:unconfined
volumes:
- ./netdataconfig/netdata:/etc/netdata:ro
- netdatalib:/var/lib/netdata
- netdatacache:/var/cache/netdata
- /etc/passwd:/host/etc/passwd:ro
- /etc/group:/host/etc/group:ro
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /etc/os-release:/host/etc/os-release:ro
network_mode: host
volumes:
netdatalib:
netdatacache:The commands to try the docker image and add the token to the cloud instance (The command is fetched from the portal)
sudo docker-compose up
sudo docker exec -it netdata netdata-claim.sh -token=xxxx -rooms=yyyy -url=https://app.netdata.cloudOnce verified that its working, persist it as below
Stop the image by hitting ctrl+c
Run the below command
sudo docker-compose up --no-start
sudo systemctl start docker-compose@netdataVerify that its working with
sudo systemctl status docker-compose@netdata
sudo docker psPersist it by creating the below file /home/pi/.firewalla/config/post_main.d/02StartNextData.sh
sudo systemctl start docker
sudo systemctl start docker-compose@netdatasudo systemctl start docker-compose@netdata
-
I generally followed the steps for setting up the Homebridge docker here: https://help.firewalla.com/hc/en-us/articles/360053184374-Guide-Install-HomeBridge-on-Firewalla-Gold-Beta- but used "netdata" instead of "homebridge" for the directory names. The section on setting up mdns isn't relevant, so I skipped that.
Here's my docker-compose.yaml
version: '3'
services:
netdata:
image: netdata/netdata
container_name: netdata
hostname: fire.walla # set to fqdn of host
network_mode: host
ports:
- 19999:19999
restart: unless-stopped
cap_add:
- SYS_PTRACE
security_opt:
- apparmor:unconfined
volumes:
- netdataconfig:/etc/netdata
- netdatalib:/var/lib/netdata
- netdatacache:/var/cache/netdata
- /etc/passwd:/host/etc/passwd:ro
- /etc/group:/host/etc/group:ro
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /etc/os-release:/host/etc/os-release:ro
volumes:
netdataconfig:
netdatalib:
netdatacache: -
There's a further step to claim the node on netdata.cloud by copy and pasting in a docker exec command with a token from your netdata.cloud account. Mine looked like this:
docker exec -it netdata netdata-claim.sh -token=<LONG TOKEN STRING GOES HERE> -rooms=<ROOM ID # GOES HERE> -url=https://app.netdata.cloud
-
I just installed it on the metal as Linux Device
Works great.
More info at: https://app.netdata.cloud
-
Sharing my experience with netdata on a Firewalla Gold (RevA). I was encountering a number of issues in recent months:
- My FWG docker partition (/var/lib/docker) was getting full with the default settings for retention in netdata
- Docker wasn't restarting the container after a reboot;
- Newer versions of netdata have been consuming more and more CPU cycles on the FWG;
- Syslog (journalctl) files were not appearing in the netdata GUI
** Disk Space **
I started by added an 250GB M.2 2242 SSD (https://a.co/d/hoPeROm), and reconfiguring docker to use that as the root for all docker files. Edit the file /lib/systemd/system/docker.service, and update the --data-root as appropriate. For me:
ExecStart=/usr/bin/dockerd --data-root /m2-sata/docker -H fd:// --containerd=/run/containerd/containerd.sock
Note, you can only add the storage on a RevA FWG, danger danger of course. Making the changes persistent in /etc/fstab is also dangerous, but possible.
[I also upgraded to 8GB ram, which gives a bit more breathing room to the host. Other posts in this forum explain how to do it.]
** Auto-start on Boot **
Put your docker-compose.yaml into ~pi/.firewalla/run/docker/netdata (create the directory). Note that I had issues with persisting changes to my netdata configuration on the Firewalla (old version of docker?), and so I changed from using docker volumes to local filesystem storage and bind mounts. My config:
services:
netdata:
image: netdata/netdata:stable
container_name: netdata
pid: host
network_mode: host
restart: unless-stopped
cap_add:
- SYS_PTRACE
- SYS_ADMIN
security_opt:
- apparmor:unconfined
volumes:
# - netdataconfig:/etc/netdata
- /home/pi/.firewalla/run/docker/netdata/netdata-config:/etc/netdata
# - netdatalib:/var/lib/netdata
- /home/pi/.firewalla/run/docker/netdata/netdata-lib:/var/lib/netdata
# - netdatacache:/var/cache/netdata
- /home/pi/.firewalla/run/docker/netdata/netdata-cache:/var/cache/netdata
- /etc/passwd:/host/etc/passwd:ro
- /etc/group:/host/etc/group:ro
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /etc/os-release:/host/etc/os-release:ro
- /etc/timezone:/host/etc/timezone:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- /run/log/journal:/run/log/journal:ro
environment:
- NETDATA_CLAIM_TOKEN=[REPLACE - TOKEN]
- NETDATA_CLAIM_URL=[REPLACE - URL]
- NETDATA_CLAIM_ROOMS=[REPLACE - ROOMS]
- TIMEZONE="[REPLACE - TIMEZONE]"
# Can be used to limit CPU usage, this is a 75% of a single core. Seems to impact performance
# resources:
# limits:
# cpus: '0.75'
#volumes:
# netdataconfig:
# netdatalib:
#netdatacache:Create a file called 'start_docker.sh' and put it in ~pi/.firewalla/config/post_main.d:
#!/bin/bash
sudo systemctl start docker
sudo systemctl start docker-compose@netdata** CPU Usage **
I changed the sampling interval to 5 seconds vs. the default of 1 second -- my FWG was spending too much time gathering metrics!
sudo docker exec -it netdata bash
cd /etc/netdata
./edit netdata.conf # refer to netdata docs if your file is not populated
# edit the entry for "update every"
[db]
# cleanup orphan hosts after = 1h
update every = 5
exit
sudo docker restart netdataI found that while this helped, the disk IO even to the SSD was still impacting the FWG's performance, so I switched netdata from "db = dbengine" to "db = ram" and implemented a netdata parent on another server. Refer to netdata docs. This really reduced the CPU load on my FWG.
** Syslog / journalctl missing **
The netdata agent looks for /var/log/journal or /run/log/journal -- the latter is what the FWG uses, but it's not mounted into the container by default. The docker-compose.yaml above includes the read-only mapping.
- /run/log/journal:/run/log/journal:ro
Please sign in to leave a comment.
Comments
8 comments