Multiple VPN policies between two sites?

Comments

10 comments

  • Avatar
    Michael Bierman

    Hi Tyer, 

    This is possible. On the remote box, create a route like this: 

    -1
    Comment actions Permalink
  • Avatar
    Tyler Pomerhn

    Excellent advice! I just configured it and will test it out next time I’m on-site. Thanks for the thorough and quick reply!

    0
    Comment actions Permalink
  • Avatar
    Tyler Pomerhn

    Does not work. Correctly set up device to be in a group, set a route for the group to be on VPN for the egress of Internet traffic, and instead of looking like it comes from the other side of the VPN, it looks like it comes from the original ISP.

    This completely breaks streaming. Any alternatives, since this solution clearly does not work at all?

    0
    Comment actions Permalink
  • Avatar
    Andy

    This solution does work!
    Why not try with a single device first and ensure you see the remote IP, use a site like whatsmyip.com to verify and troubleshoot from there.
    Sometimes taking the device off the network for a few seconds, or a reboot, helps so that it picks up the new route.

    0
    Comment actions Permalink
  • Avatar
    Tyler Pomerhn

    Tried this. No help. Does not work. Sorry about my snarky reply earlier, I'm kinda upset about this seemingly basic functionality being broken. ;) 

    0
    Comment actions Permalink
  • Avatar
    Tyler Pomerhn

    To be perfectly and crystal clear:

    • VPN site to site established and operational between "Site A" and "Site B" and between "Site A" and "Site C" - let's say that "Site A" looks like it is coming from IP address X, and "Site B" looks like it is coming from IP address Y, and "Site C" looks like it is coming from IP address Z.
    • If I attempt to connect between sites using private IP addressing, everything works 100%. No issues. So VPN is clearly working. B can talk to A (and vice versa), and C can talk to A (and vice versa).
    • In the past, I would create a site-to-site VPN between A and C but only specify C's "VPN" network as a valid source of traffic at the VPN layer (not routing), a VLAN that I specified for clients that would always look like they were coming from A. In that configuration, it would work. It does not, today.
    • I was advised, per this thread, to create a normal site-to-site VPN between A and B, and A and C, to allow for private communication between the three sites, but that if a specific group, host, or network wanted to be "forced" on the VPN, to do a route.
    • I have placed a set of 10 hosts in Site B, based on Firewalla host/group configuration/mappings, to be in a group called "TVs." I have also created a rule saying to route any traffic destined for the Internet from group "TVs" to go into the VPN to Site A.
    • One of the hosts in this group is an Apple TV. I installed a "what's my IP?" application, and it states it looks like it's coming from Miami via Starlink (Site B's ISP), instead of Raleigh via AT&T (Site A's ISP). Similarly, flows from that host look like their egress interface is Starlink (though, to be fair, that would also be the egress interface for VPN traffic). This is not correct. The Apple TV should look like it is coming from Site A, and Fubo should see Raleigh-based TV, not Miami-based TV. It does not.
    • I have also stripped all VPN and routing off entirely, and only assigned my own cell phone to that routing/VPN policy, and traffic still looks like it's from Site B's ISP. So clearly, traffic is not being forced into the tunnel when it goes to the Internet.

    I hope this clarifies my situation. If there's something I am doing that is wrong, please let me know. However, my end goal is the following:

    • Site A should be able to access, via Wireguard VPN, sites B and C using private IP addressing.
    • Under normal operation, all site hosts should "look" like they are, from the Internet's perspective, to be coming from the locally preferred egress method. Meaning, Site B traffic to whatismyip should look like it's coming from IP address Y.
    • If a network group, or a network interface, of a remote site - B, C, or any other - is selected to route via their respective tunnel interface to A, in a Static manner, then those hosts should look like they are coming from IP address X. If the VPN is down, they should lose internet access entirely, while other hosts at B and C not designated for this tunnel forwarding would continue to work fine to the Internet through IP address Y and Z, respectively.
    0
    Comment actions Permalink
  • Avatar
    Tyler Pomerhn

    Further troubleshooting - turned off, completely, all VPN client hosts - despite this completely breaking site to site VPN - and only routed traffic for internet for TVs from client to hub site. Still does not work.

    0
    Comment actions Permalink
  • Avatar
    Tyler Pomerhn

    Okay, so if I select the devices I want to VPN as the only hosts in the VPN Client profile, and I also select for Internet traffic to be VPN instead of Direct, it works. So, it seems that the "VPN/Direct" Internet button on the VPN Client is what ultimately dictates this behavior.

    But this now creates two issues:

    1) This completely destroys my site-to-site VPN (verified).

    2) Even if I selected, additionally, hosts to be part of the VPN (so that I can manage them remotely), they would have their internet traffic, forcibly, routed to Site A. It's the reverse of my original problem - I don't want EVERY host that "can" VPN across the site-to-site link to also put their Internet traffic through the VPN. This introduces unacceptable latency.

    0
    Comment actions Permalink
  • Avatar
    Tyler Pomerhn

    Latest update - I attempted the reverse (i.e. set a group of hosts to be in the VPN Client allow list, set the VPN Client Internet action to "VPN" instead of "Direct," and then set one of those hosts to have a route stating that Internet traffic should go right out the Internet interface locally, instead of through the VPN), and it does not work.

    So, basically, site-to-site VPN works, and Remote-looking-like-it-comes-from-the-Hub works, but only if you want EVERY host that CAN traverse the tunnel to ALWAYS use the hub for internet, because VPN Client Allow lists ultimately dictate what CAN go into the tunnel, and once they are subscribed to the tunnel, they MUST follow the Internet policy of the VPN Client profile. Kinda makes the routing area of Firewalla useless, frankly, but this is the behavior I'm seeing.

    Alternative solutions heartily welcomed!

    0
    Comment actions Permalink
  • Avatar
    Tyler Pomerhn

    Okay, so here is the solution for this problem/setup:

    • If a host is listed in VPN Client, the behavior for any traffic not destined for another host local to the site is to either go directly to the Internet, or via the tunnel, depending on what the VPN Client profile is set to (Direct or VPN, respectively).
    • If a host is listed in VPN Client, routing policies make absolutely no impact on traffic flows whatsoever. The VPN Client directive trumps everything else.
    • If a host is not listed in VPN Client, it will follow routing rules. If none are specified, all non-local destinations will be sent directly to the internet.
    • If a host not listed in VPN Client needs to participate in site-to-site VPN, even though there is an explicit rule created by Firewalla for this traffic under "Rules," you are still required to create a route for traffic to the hub subnet(s) via the tunnel interface.

    So basically I can now operate correctly by specifying hosts that need to look like they're coming from the hub by including them in the VPN Client profile, making Internet access on that profile be set to "VPN." This handles hosts that need to appear like they originate from the hub location. Then, I created a route for site-to-site VPN by specifying the local subnet at the remote site, with destination of hub subnet, to have a next-hop of the VPN tunnel (even though those hosts are not selected for the VPN Client profile; this still works, evidently).

    This is not intuitive and I have requested Firewalla team to create some kind of document/helper tutorial on the correct order of operations in their software. Being a long time Cisco person, I would have thought that routing policies override anything else, but that is not true in Firewalla... and that's okay, I just have to accommodate the alternative logic used. ;) 

    0
    Comment actions Permalink

Please sign in to leave a comment.