it's still possible to setup a VPN to a remote
host and interface with
the IPIP mesh there. It's not idea, but it's a solution. As long as all
the nearby sites have a route over VPN, it can work. By tuning the VPN
parameters, latency can be kept down.
That is what we have operating right now. Our 44.137.0.0/16 network
is now BGP announced from a datacenter where we have a MikroTik CCR1009
router that has many users connected via different kinds of VPN
(L2TP/IPsec, GRE, GRE/IPsec, GRE6 (GRE over IPv6)) and the users run
BGP to announce their subnets and receive our subnet.
They can (and do) inter-connect using radio links and setup BGP peering
over there as well, so the routing between users can be over radio and
towards internet over the VPN. We have some BGP communities to steer
the routing, e.g. to prefer a multihop radio path over a path via VPN
which is always two hops at most.
This router is on the IPIP mesh as well, but when that mesh were to be
taken down we only would need to setup some similar VPNs and BGP
peerings, of course one to UCSD where a router would be for the space not
announced separately (44.0.0.0/9 and 44.128.0.0/10), and anyone in the
world would have the possibility to setup a VPN server for some subnet
(be it BGP announced or not, that does not matter) and serve their
local users from it. Users in a region not covered by such a service
could connect to UCSD and have similar performance to what it is now.
Those VPN servers do not need to peer with ALL others, but they can peer
with a couple of them and we still have redundancy.
In addition to this, we also offer OpenVPN VPNs without BGP, for users
that are endpoints only. A single fixed address is routed to the VPN
client when it is connected.
This all is really easy for the users to setup, and when the L2TP/IPsec
VPN is used it works over all internet connections even when they have
dynamic address, are behind CGNAT, have agressive firewalls, or whatever.
Should it prove necessary, an SSL based VPN like SSTP could easily be
added (it uses TCP port 443 so it always works, but of course VPNs using
TCP transport are to be avoided when possible)
BGP in the MikroTik routers is configured in a few mouseclicks or a
cut-and-paste of a small text fragment. Set the local AS and ID, add
a peer with remote address and AS, set 2 or 3 options, and it works.
For best operation, use a small routing filter that can be cut-and-paste.
Rob