11 Aug
2021
11 Aug
'21
5:16 p.m.
On 8/11/21 11:35 AM, Toussaint OTTAVI via 44Net wrote:
No, it would make no difference whatsoever!
The backbone system would internally use BGP to distribute the subnet routes for all
client systems connected to it, much like the IPIP mesh now uses RIP to advertise that
info. The difference would be that it would be dynamic, as determined by connection of
endpoints, not a static portal route list.
The "holes" in those routes default to the route to internet, and the 44.190
ranges would fall into that category.
There is no need at all for a sysadmin to do anything special for that.
Furthermore, with "every sysadmin" you suggest that this would be a task for
anyone in the network, while in reality only the backbone sysadmins would be tasked with
that, as the remainder of admins either have a static route for 44/9 and 44.128/10 or they
have configured BGP to automatically exchange routes with the backbone routers (and their
local radio network).
Yes, as I said above:
automatically created routes for each tunneled peer, and a default route 0.0.0.0/0 for
everything else (including what is now 44.190).
For a long time now, everything has worked
perfectly for those users who announce on internet BGP and do ALSO register on the IPIP
mesh.
They know how to route to other IPIP mesh members via a tunnel and route the remainder to
internet. They are reachable for everyone that has sane routing.
Of course, it works. But every sysadmin has to know exactly how to route every subnet.
F/ex, not everybody knows 44.190 routing policy. Grouping into "Internet" and
"Intranet" categories, each category in a parent subnet, would hugely simplify
things. But managing a database on the portal about what
subnet is Intranet and what subnet is Internet would probably do the job, too. Some
scripts could update routing policies from the portal at POP level, which will be managed
by skilled sysadmins. Not really "KISS", but it would work.
No database
other than the BGP route table would be required.
There is a 0.0.0.0/0 route in the route table pointing to the local internet connection,
and several routes to participants in the backbone (similar to what we now have on IPIP)
which are more specific and are taken when available.
We have this setup here for a long time and I can assure you we do nothing special for
44.190.
Our route table looks like this:
# ip route
default via 213.222.29.193 dev eth0 onlink
unreachable 10.0.0.0/8
44.0.0.0/9 via 213.222.29.193 dev eth0 src 44.137.0.1
44.0.0.1 via 169.228.34.84 dev tunl0 proto ampr-ripd metric 4 onlink
44.2.0.2 via 98.20.210.138 dev tunl0 proto ampr-ripd metric 4 onlink
.....
44.185.112.0/24 via 87.252.188.119 dev tunl0 proto ampr-ripd metric 4 onlink
44.188.1.1 via 70.80.196.6 dev tunl0 proto ampr-ripd metric 4 onlink
44.188.192.222 via 109.227.66.46 dev tunl0 proto ampr-ripd metric 4 onlink
unreachable 169.254.0.0/16
unreachable 172.16.0.0/12
unreachable 192.168.0.0/16
213.222.29.192/28 dev eth0 proto kernel scope link src 213.222.29.194
#
The additional features of backbone routers I can think of are:
- a reverse-path filter in the backbone routers that drops traffic from internet coming
from addresses that are supposed to be tunnel-routed
- a source-address filter in the backbone routers that drops traffic from addresses not
assigned to anyone
These are protections agains address spoofing from internet
- a feature for users of the network to disallow all traffic from/to any address outside
net-44, selectable in the account admin page for backbone users, for users who want (or
can) have nothing to do with internet.
With such a "smart backbone" there is no need whatsoever to arrange networks in
different categories based on their address.
Such settings could be changed at anytime without requiring a renumber.
That is a design issue. It can be easily solved
especially when the new backbone network is present that can do BGP annouces itself, so it
is completely transparent to the users if each subnet is on the tunneled network, or on a
44-net subnet only announced on internet. It will always be routed correctly without the
operator needing to set separate routes.
Separate routes will still be needed at POP level. Endpoints will just route all to the nearest POP, so
that things remain simple. At least for remote endpoints connected to a POP via a tunnel;
but for endpoints connected to several neighbors via meshed radio links, things may not be
as simple...
--
This makes me think our previous idea of a 3-level topology (backbone, POP, Access) may
not be enough. Maybe, we need an intermediate :
- Backbone : world-wide infrastructure managed by ARDC, and providing network background
- Point of Presence (POP) : local / regional platform, connected to the backbone, and
providing connectivity to end-users
- (new) Router : Intermediate mesh router, connected to several neighbors via radio
links, and contributing to the global routing in relation with nearest POPs
- Access : Small low-cost router (Mikrotik/OpenWRT etc...) or software (VPN client, RPi)
offering 44net IPs to endpoints by connecting to one or several POPs with PnP technologies
Of course the whole network can have several layers. In the early discussion there was
already mention of separating the routing backbone from the client access PoP, and that
could make sense. We have the above setup here, as our routers (MikroTik) cannot
reasonably do OpenVPN which is a technology we want to offer. And we also are on IPIP
mesh. So we have different routers for both and they exchange routes via BGP.
And of course we have users connected via tunnel that are at the same time routers in the
radio network, provide local user access via Radio, etc.
The beauty of an IP network is that you can connect that all together in a semi-structured
way without having to do it the same way everywhere.
In our network, we could continue to service users the way we do now, and if there is an
ARDC PoP in the region as well users can opt to directly connect to there, and of course
we would connect there announcing our route(s).
Then everyone can connect the way they want to, with or without internet routing as they
wish.
Rob