Now that we are all going to have to dive into our router
configurations, wouldn't it be a
good time to make some changes that are long overdue?
Like getting rid of the IPIP mesh and replace it with something more
modern and supported
by off-the-shelf routers, works behind NAT, etc?
I would say setup some routers with VPN of different types around the
world, have everyone
connect to there using a suitable VPN protocol, run BGP on it to
announce the gateway subnets.
A $50 MikroTik can do those jobs, for those that still want to run a
JNOS system on MS-DOS
they can put one in front of their box and still use it. People are
already using it for IPIP mesh,
a change in topology would be only a config change for them. And other
routers mentioned
here can do it too, without having to get external programs installed on
them.
Those that want direct connection without a centralized system in the
path can simply setup
a VPN connection between them and configure the BGP peers, it will
automatically work.
There is no need to use only a single protocol in such a network, only
the peers have to agree,
so you can select from anything like L2TP/IPsec, OpenVPN, Wireguard,
just plain GRE or even IPIP,
etc etc. Just at this time I am trying to move my colocated machine
that runs as an IPIP mesh
member and I face that stupid "protocol 4 is not passed by the firewall"
problem again. Arghh!!
Also we could get that IPv6 idea going. Remember it has been discussed
many times and the
only things we still need is some agreement on how to register and
distribute the "list of AMPRnet
prefixes in IPv6 space". Again that could be done using BGP, no need to
setup yet another
registration portal with downloadable files.
Note that Daniel EA4GPZ put some ideas around IPv6 on his site:
https://destevez.net/ipv6-for-amateur-radio/
Rob
> I saw that. Messages are readable on Android, but appears blank in Thunderbird. When we answer, the answer remains blank (but the text is readable in the message source)
> Will check...
Mail isn't what it used to be. Programs like Thunderbird auto-decide whether to send the mail in text
or HTML, and somewhere along there are linebreaks inserted in the messages that I do not insert myself
and that do not appear in my Sent folder, which fouls up the layout. I see that with some other
people's messages as well, so the fault may be in the mailing list software.
The funny thing is that sometimes when my messages, that appear broken when posted, get quoted by
other people they suddenly are OK.
It is now completely unclear to me whether I am supposed to breakup paragraphs by line or to type
everything on a single line. Both ways things are getting fouled up.
Rob
W.r.t. the issue of /64 or smaller addresses, my plan is as follows:
- we have a /16 network within AMPRnet (44.137.0.0/16)
- it is very easy to get a /48 network in IPv6 (every home customer at
my ISP gets one of them, I have one as well. it would be easy to get
a /48 assigned to our gateway where 44.137.0.0/16 is BGP routed now
- I want to make a 1:1 mapping of every assigned address in our IPv4
subnet to a /16 network within that /48 space.
E.g. when we get 1234:5678:abcd::/48 from our ISP, I would map
44.137.0.1 to 1234:5678:abcd:0001::/64, up to 44.137.255.255 mapped
to 1234:5678:abcd:ffff::/64
- This means every single address allocated in IPv4 gets a /64 subnet
in IPv6, any /28 in IPv4 (common subnet size for hams) gets a /60
subnet, etc.
This way, standard practices for using IPv6 addresss can be used, and
the proposal of Daniel EA4GPZ can be followed to encode the callsign and
service in the lower 64 address bits as desired (for convenience only).
We would "just" have to add IPv6 addresses to all routers in our network,
but it can be done in an almost automated way because the IPv6 addresses
and subnets can be calculated from the IPv4 addresses already configured
using a simple tool.
Of course it still requires quite some work, and causes double maintenance
on things like firewalls and routing filters, so I have not done it yet.
But it sure would be possible.
Rob
> > - I want to make a 1:1 mapping of every assigned address in our IPv4
> > subnet to a /16 network within that /48 space.
> > E.g. when we get 1234:5678:abcd::/48 from our ISP, I would map
> > 44.137.0.1 to 1234:5678:abcd:0001::/64, up to 44.137.255.255 mapped
> > to 1234:5678:abcd:ffff::/64
> I'm a bit lost at what you're suggesting, and the practical way it would
> be implemented. At first glance, this looks like a recipe for
> suboptimal routing to the outside world.
The above is an idea how to add IPv6 addresses to the 44.137.0.0/16
network that
is BGP connected in a datacenter and where users are connected via VPN
and radio
paths. The added IPv6 addresses would be from the datacenter space and
would be
routed along the same paths as the 44.137 traffic. Of course that is
suboptimal, but
it works for users that do not have IPv6 themselves.
> I have 2 AMPRnet IPv4 allocations - one BGP connected, one IPIP
> connected. Both sites have IPv6 available. Here, where the IPIP mesh
> terminates has a /56. I currently have a /64 on the VPS, but as there's
> only one host, it wouldn't be difficult to carve that into AMPRnet and
> non AMPRnet parts, because IPs are assigned by hand and are all on the
> same interface.
In my opinion it is not a good idea to deploy a system based on
splitting a /64 net
in smaller subnets. You could do that locally, but you would not want
to give other
local users an IPv6 network that is smaller than a /64. This is going
to cause all kinds
of difficulties. That is why my intention is to give every user at
least a /64.
Rob
> > Setting up BGP on a MikroTik is much much much easier than getting
> > IPIP mesh to run on anything!
> I see an opportunity here to learn BGP. I'm for this idea.
I posted an example configuration above. Of course this is not the same
as "learn BGP", but
I think many users of the current IPIP mesh also did not "learn IPIP"
and "learn RIP" but only
copied an example configuration and fiddled with it until it worked.
Of course when you want to setup a regional VPN server you need some
configuration for that
as well, but when such a system would be deployed of course examples for
that can be
given as well.
BGP for such a small closed network is not that complicated. Basically
every system maintains
a TCP connection (port 179) with all its peers, and it sends the
networks that it can route.
It receives the same information from the other side, and fills the
route table. The active
route is selected on a couple of criteria, where the least number of
hops is usually preferred.
It is possible to send tags along each route (called bgp-communities)
that can be used to
prefer certain routes over others, e.g. to prefer routes over radio when
both a radio and a
VPN path exist.
There are some issues with BGP, e.g. the total lack of security in the
protocol. Anyone can claim
that they have a subnet and all the others will happily route all
traffic for that subnet to them.
The routing filters are an attempt to work around the most severe
problems, but as can be seen
on the internet (which also uses BGP) it is difficult to make it
completely failsafe.
Also, in our world it is a bit of a nuisance that there is no way to
incorporate some form of
dynamically determined link quality in the routing decision. Links are
either up or down. But for
this proposed use (replacement of the IPIP mesh) that is not a problem,
it mainly affects the
use of BGP on the radio links in our network.
Rob
> First thought would be that BGP is too difficult for 90% of the HAM operators.
Setting up BGP on a MikroTik is much much much easier than getting IPIP mesh to run on anything!
Rob
Amateur Radio Digital Communications [ARDC] is a United States
charitable 501(c)(3) nonprofit public benefit corporation that has
long owned and managed the Internet address space known as the
AMPRNet.
Nearly 40 years ago, early in the evolution of the Internet, this
address allocation was acquired to be used for the mutual benefit
of Amateur Radio and digital communications technology.
Amateur Radio operators ("hams") use the global radio spectrum set
aside for them by international treaty in non-commercial ways to
improve engineering, research, experimentation, training, education,
and emergency communications. Having the AMPRNet available over
the past four decades has facilitated integration of the Internet
with radio-based technologies long used by hams. This long term
interaction has been key to development of now ubiquitous wireless
technology such as WiFi and the ability to browse the Internet or
to stream various media to your mobile phone.
Over those past decades, a portion of the AMPRNet IPv4 address space
has rarely been used, and recent utilization surveys show that it
is not likely to ever be needed by hams.
Initially free, IPv4 Internet addresses are now highly valuable,
and there is an international marketplace in which to sell them.
ARDC has sold some of its unused and unneeded address space, but
retains a more than ample supply of IPv4 addresses for current and
future use by the many Amateur Radio operators worldwide. The sale
amounts to some millions of dollars, which will be used in the
furtherance of ARDC's continuing public benefit purpose.
Before the sale, the AMPRNet consisted of the addresses 44.0.0.0
through 44.255.255.255 (in Internet notation, 44.0.0.0/8). Post-sale,
it consists of addresses 44.0.0.0 through 44.191.255.255 (44.0.0.0/9
plus 44.128.0.0/10). The uppermost 1/4 of the former AMPRNet address
space (44.192.0.0/10) has been withdrawn from ham radio use and
sold to another owner, however over 12 million IPv4 addresses remain
for amateur radio use.
ARDC will use the proceeds of this address sale to further its
mission to support, promote, and enhance Amateur Radio, digital
communications, and broader communication science and technology
by funding grants and scholarships for scientific research,
experimentation, education, open access, and innovation in information
and communications technology, with an emphasis on benefiting the
international Amateur Radio service.
For further information, please see <https://www.ampr.org/amprnet/>.
Best wishes and 73,
The ARDC Board of Directors
> The IPIP mesh may be non-standard, but it is distributed, without any single point of failure. To get between two points, the two gateways have to have IP connectivity to each other. That's it. The two end-points can troubleshoot directly.
> But every proposal I've seen on this list involves adding at least two other ham points of failure. For example, I would presumably connect to some other ham's BGP node and the other end of the connection would do the same. Why?
Mainly because it makes it an "outgoing" connection for most people.
That means it is not affected by NAT, CGNAT, dynamic addresses, inflexible router firewalls, etc.
Getting IPIP to work bidirectionally often requires to setup the internal system as "DMZ" (which in home router speak means that it will receive all incoming traffic not matched by the NAT engine).
This can only be done for a single system, and even then it often does not work correctly (e.g. it receives only TCP/UDP and not other protocols).
You should not under-estimate the difficulty that IPIP is causing for more and more people.
As an example, I am currently trying to migrate my physical colocation server to a VM. The service offered by the hosting company uses Apache Cloudstack.
While the VM itself works fine and can load the ipip module, it turns out that Cloudstack cannot allow IPIP ingress in its firewall settings.
So I will have to look for another hosting option...
By putting VPN servers in datacenters and having the users connect to them, you avoid those problems.
Also, using the topology I propose, there is no requirement that the entire network runs a single tunneling protocol or a single topology.
Only the peers need to support the same protocol. And it can be a symmetric protocol like IPIP or GRE when you like that better.
Also, there is no requirement that there only be a single connection! You can setup crosslinks to wherever you like.
There will rarely be traffic to all IPIP mesh peers. There are more than 500 gateways in the list, and when I look in our stats I see we have had traffic to only 23 of them in the past day.
Most active gateways can probably point out at most 5 peers to which they have regular traffic. You can setup crosslinks to those.
And this system is not dependent on a central RIP sender! Every node is active in distributing its own subnets and receiving the others.
There is no need for a portal that registers the subnets, they only need to be configured in the gateway routers.
A portal could still be useful as a management system for VPN accounts. Especially at the major hubs, the users need to have a VPN account that assigns them a point-to-point address, and the BGP on that router needs to be configured with the AS number on that address. This could be done using some form of portal and a program that configures the router using its API.
That is not mandatory. On our gateway the addition of new users still is done manually. I once have added a large number of skeleton BGP peers and VPN accounts, and when a new user requests a connection I find an available one, enter the username (callsign), a random password, and enable it. In the BGP peer I enter the AS number and enbale it. Done.
There always will be some dependencies, but they do not need to be single-point and with some money available they do not need to be only relying on volunteers and unclear situations where some company hosts a service only because a worker there has arranged it.
I think it has big advantages over the system we have now.
Rob
Well, I think there are enough examples of how to do it (also already
doing it within AMPRnet),
lots of people who can contribute knowledge and will volunteer to help,
the only thing we lack
here in this group is someone (with the appropriate authority) who says
"Lets' do that!" and
gets everyone moving.
That is why these discussions usually go on for a couple of days here,
then die out, and
nothing ever changes. We are still running the same system as 25 years
ago, with only
the addition of RIP auto route distribution.
This gets worse over time, as those who would like to change things see
this and leave, and
in the end we only have a number of old hats left who don't change anything.
That does not improve our reputation... sometimes newcomers ask "why do
we use that
method when it is so difficult to implement and it can all be done so
much easier", and the
only response can be "don't ask...".
Rob
Because of the current changes, people want to change firewall entries
that check an address to be "within AMPRnet".
This of course used to be simple, like "-s 44.0.0.0/8" as a matcher in
Linux iptables.
I already read remarks like "well I just need to split those rules in
two, one for 44.0.0.0/9 and another for 44.128.0.0/10".
This is not really required. And it is not so easy when the match was
in a "not" context, like: ! -s 44.0.0.0/8.
Also it makes the firewall rule list longer and maybe less intuitive.
Maybe not everyone knows how ipset can be used to match more than one
address in the same rule.
The ipset program creates "address sets" that can be lists of addresses,
networks, portnumbers etc that can then be referenced in a match.
For example, create an address set that contains the new AMPRnet ranges:
ipset create AMPRnet hash:net
ipset add AMPRnet 44.0.0.0/8
ipset add AMPRnet 44.128.0.0/10
ipset add AMPRnet 44.224.0.0/15
(the last one of course has to go, but I kept it there to give our
German friends some time to renumber)
Now instead of "-s 44.0.0.0/8" you can use: -m set --match-set AMPRnet src
Instead of "-d 44.0.0.0/8" it would be: -m set --match-set AMPRnet dst
And instead of "! -s 44.0.0.0/8" you still can use the form: -m set !
--match-set AMPRnet (src or dst), so
there is no need to change the structure of the rules to work around the
problem of not having a "not" operator.
Of course you need to arrange for the above commands to be run before
the iptables rules are loaded.
No problem for me as I have always done that in an own shellscript not
using distributor's solutions that use iptables-save or similar.
(I prefer to have the possibility to have comments, use variables
holding certain addresses and networks, etc)
In MikroTik RouterOS the same feature is available as "Address list" in
the firewall.
You can create an address list and load those subnet values:
/ip firewall address-list
add address=44.0.0.0/9 list=amprnet
add address=44.128.0.0/10 list=amprnet
and then use that in filter rules by using Src.Address List instead of
Src.Address for the match.
Rob
