Multiple VPN policies between two sites?
I have two Golds, one in a remote location using Starlink and one in the US on 1Gbps fiber. I currently have a VPN set up between these sites and it works fine for site-to-site traffic, but I need to have a different policy for specific hosts at the remote site. Namely, I have devices like TVs that need to "look" like they come from the US, but I don't want to tunnel everything through the US if I don't "have" to. The devices that "must" tunnel are in a specific VLAN, so they come in to the remote Gold unit on VLAN2 when everything else comes in VLAN1.
What I'd like to do is have the following:
- Remote VLAN2 to Internet: force through US, kill if US tunnel is down
- Remote VLAN1 to Internet: allow through direct access
- Allow the two sides to communicate with each other over the VPN tunnel, no matter what VLAN they are in on either side
Is this possible? I have another site where I have a similar setup and what I've done there is force VLAN2 traffic only through the tunnel, and set the tunnel as VPN-based internet with a kill switch, but in that configuration I can't access VLAN1 devices at the remote site from the US site.
-
Hi Tyer,
This is possible. On the remote box, create a route like this:
- Matching: can be:
Internet (all trafic)
Target list (specified domains)
IP /IP ranges
Regions - ON: Pick a device, Group, or a Network
- Interface: pick the VPN
- Route Preference: Static if the traffic MUST use the Interface or Preferred where it will failover to your default Interface.
https://help.firewalla.com/hc/en-us/articles/360061592433-Firewalla-Policy-Content-Based-Routing
- Matching: can be:
-
Does not work. Correctly set up device to be in a group, set a route for the group to be on VPN for the egress of Internet traffic, and instead of looking like it comes from the other side of the VPN, it looks like it comes from the original ISP.
This completely breaks streaming. Any alternatives, since this solution clearly does not work at all?
-
This solution does work!
Why not try with a single device first and ensure you see the remote IP, use a site like whatsmyip.com to verify and troubleshoot from there.
Sometimes taking the device off the network for a few seconds, or a reboot, helps so that it picks up the new route. -
To be perfectly and crystal clear:
- VPN site to site established and operational between "Site A" and "Site B" and between "Site A" and "Site C" - let's say that "Site A" looks like it is coming from IP address X, and "Site B" looks like it is coming from IP address Y, and "Site C" looks like it is coming from IP address Z.
- If I attempt to connect between sites using private IP addressing, everything works 100%. No issues. So VPN is clearly working. B can talk to A (and vice versa), and C can talk to A (and vice versa).
- In the past, I would create a site-to-site VPN between A and C but only specify C's "VPN" network as a valid source of traffic at the VPN layer (not routing), a VLAN that I specified for clients that would always look like they were coming from A. In that configuration, it would work. It does not, today.
- I was advised, per this thread, to create a normal site-to-site VPN between A and B, and A and C, to allow for private communication between the three sites, but that if a specific group, host, or network wanted to be "forced" on the VPN, to do a route.
- I have placed a set of 10 hosts in Site B, based on Firewalla host/group configuration/mappings, to be in a group called "TVs." I have also created a rule saying to route any traffic destined for the Internet from group "TVs" to go into the VPN to Site A.
- One of the hosts in this group is an Apple TV. I installed a "what's my IP?" application, and it states it looks like it's coming from Miami via Starlink (Site B's ISP), instead of Raleigh via AT&T (Site A's ISP). Similarly, flows from that host look like their egress interface is Starlink (though, to be fair, that would also be the egress interface for VPN traffic). This is not correct. The Apple TV should look like it is coming from Site A, and Fubo should see Raleigh-based TV, not Miami-based TV. It does not.
- I have also stripped all VPN and routing off entirely, and only assigned my own cell phone to that routing/VPN policy, and traffic still looks like it's from Site B's ISP. So clearly, traffic is not being forced into the tunnel when it goes to the Internet.
I hope this clarifies my situation. If there's something I am doing that is wrong, please let me know. However, my end goal is the following:
- Site A should be able to access, via Wireguard VPN, sites B and C using private IP addressing.
- Under normal operation, all site hosts should "look" like they are, from the Internet's perspective, to be coming from the locally preferred egress method. Meaning, Site B traffic to whatismyip should look like it's coming from IP address Y.
- If a network group, or a network interface, of a remote site - B, C, or any other - is selected to route via their respective tunnel interface to A, in a Static manner, then those hosts should look like they are coming from IP address X. If the VPN is down, they should lose internet access entirely, while other hosts at B and C not designated for this tunnel forwarding would continue to work fine to the Internet through IP address Y and Z, respectively.
-
Okay, so if I select the devices I want to VPN as the only hosts in the VPN Client profile, and I also select for Internet traffic to be VPN instead of Direct, it works. So, it seems that the "VPN/Direct" Internet button on the VPN Client is what ultimately dictates this behavior.
But this now creates two issues:
1) This completely destroys my site-to-site VPN (verified).
2) Even if I selected, additionally, hosts to be part of the VPN (so that I can manage them remotely), they would have their internet traffic, forcibly, routed to Site A. It's the reverse of my original problem - I don't want EVERY host that "can" VPN across the site-to-site link to also put their Internet traffic through the VPN. This introduces unacceptable latency.
-
Latest update - I attempted the reverse (i.e. set a group of hosts to be in the VPN Client allow list, set the VPN Client Internet action to "VPN" instead of "Direct," and then set one of those hosts to have a route stating that Internet traffic should go right out the Internet interface locally, instead of through the VPN), and it does not work.
So, basically, site-to-site VPN works, and Remote-looking-like-it-comes-from-the-Hub works, but only if you want EVERY host that CAN traverse the tunnel to ALWAYS use the hub for internet, because VPN Client Allow lists ultimately dictate what CAN go into the tunnel, and once they are subscribed to the tunnel, they MUST follow the Internet policy of the VPN Client profile. Kinda makes the routing area of Firewalla useless, frankly, but this is the behavior I'm seeing.
Alternative solutions heartily welcomed!
-
Okay, so here is the solution for this problem/setup:
- If a host is listed in VPN Client, the behavior for any traffic not destined for another host local to the site is to either go directly to the Internet, or via the tunnel, depending on what the VPN Client profile is set to (Direct or VPN, respectively).
- If a host is listed in VPN Client, routing policies make absolutely no impact on traffic flows whatsoever. The VPN Client directive trumps everything else.
- If a host is not listed in VPN Client, it will follow routing rules. If none are specified, all non-local destinations will be sent directly to the internet.
- If a host not listed in VPN Client needs to participate in site-to-site VPN, even though there is an explicit rule created by Firewalla for this traffic under "Rules," you are still required to create a route for traffic to the hub subnet(s) via the tunnel interface.
So basically I can now operate correctly by specifying hosts that need to look like they're coming from the hub by including them in the VPN Client profile, making Internet access on that profile be set to "VPN." This handles hosts that need to appear like they originate from the hub location. Then, I created a route for site-to-site VPN by specifying the local subnet at the remote site, with destination of hub subnet, to have a next-hop of the VPN tunnel (even though those hosts are not selected for the VPN Client profile; this still works, evidently).
This is not intuitive and I have requested Firewalla team to create some kind of document/helper tutorial on the correct order of operations in their software. Being a long time Cisco person, I would have thought that routing policies override anything else, but that is not true in Firewalla... and that's okay, I just have to accommodate the alternative logic used. ;)
Please sign in to leave a comment.
Comments
10 comments