Subject: Re: [44net] Strange Broadcasts... From: Tom Hayward esarfl@gmail.com Date: 06/11/2015 08:45 PM
To: AMPRNet working group 44net@hamradio.ucsd.edu
Forget about this; it's just syntax. Who cares if we have to have hundreds of P2P interfaces instead of a single P2MP? The script creates them, so the burden of both methods is equivalent.
Well, I think it is much better (at least, cleaner) to have a single IPIP interface (P2MP) and have a route table (separate or merged with the main table) updated by a routing protocol.
What you have made will work, but requires frequent downloading of a file and patching of the configuration to process changes. I did that before on Linux, but after having installed ampr-ripd I would not want to go back to that method...
About routes: in Linux it is possible to have routes bound to an interface that has a broadcast nature (e.g. ethernet) and with a gateway that is outside the subnet served by the system itself.
So you can partition a subnet and still have only a single gateway. This can be useful on ham networks where everyone in a local area has a Point-to-Point link to a node, and everyone routes to others via the node, but the node does not have an address in each subnet.
RouterOS cannot handle this, but Linux can. E.g.:
node has subnet 44.137.40.0/23 and has address 44.137.41.254 user has subnet 44.137.41.96/28 and has address 44.137.41.110 on eth0 then: ip route add 44.0.0.0/8 dev eth0 gw 44.137.41.254 onlink
To do this in RouterOS (and in many other routers) you need to set the subnet mask to /23 on the eth0, but that means it will try to contact the others directly, not routing via the node. This will work when the local area is switching/bridging.
Otherwise you will need to have a node address in everyone's subnet.
The same situation occurs when setting up the P2MP IPIP tunnel interface: you assign it the subnet of your local system, so all the routes are outside the interface's subnet and need the "dev" and "onlink" parameters. RouterOS does not support them. But Linux does.
Rob
RouterOS cannot handle this, but Linux can. E.g.:
node has subnet 44.137.40.0/23 and has address 44.137.41.254 user has subnet 44.137.41.96/28 and has address 44.137.41.110 on eth0 then: ip route add 44.0.0.0/8 dev eth0 gw 44.137.41.254 onlink
To do this in RouterOS (and in many other routers) you need to set the subnet mask to /23 on the eth0, but that means it will try to contact the others directly, not routing via the node. This will work when the local area is switching/bridging.
Otherwise you will need to have a node address in everyone's subnet.
The same situation occurs when setting up the P2MP IPIP tunnel interface: you assign it the subnet of your local system, so all the routes are outside the interface's subnet and need the "dev" and "onlink" parameters. RouterOS does not support them. But Linux does.
You might not have found the solution yet. But RouterOS DOES support PtP addressing on PtMP interfaces.
/ip address add 44.137.41.110/32 network=44.137.41.254 interface=ether1 /ip route add dst-address=44.0.0.0/8 gateway=44.137.41.254
73 de Marc, LX1DUC
Marc, that is a completely different case. The problem is the fact that first of all, IPIP tunnels can not be point to multipoint (you get an error if the remote tunnel endpoint is not specified). Second, on Mikrotik, the route can not be bound to a specific interface lik in 'ip route' since it does not have the 'dev' parameter. Destination is either an IP or an interface. One can not have both. And third, if the gateway is not resolvable by a previous specific route (like in your example), then the system will issue an ARP, get no response and mark the route inactive. On linux systems, this is solved by adding the parameter "onlink" to the route, which is not possible on RouterOS.
And of course, there is the big issue, if all those could be solved, does the IPIP driver consider the route an endpoint directive (like on regular Linux) or just an ordinary route, but this is of no importance right now - it probably does since it is a modern kernel.
Marius, YO2LOJ
-----Original Message----- From: 44net-bounces+marius=yo2loj.ro@hamradio.ucsd.edu [mailto:44net-bounces+marius=yo2loj.ro@hamradio.ucsd.edu] On Behalf Of Marc, LX1DUC Sent: Friday, June 12, 2015 11:28 To: AMPRNet working group Subject: Re: [44net] Strange Broadcasts...
(Please trim inclusions from previous messages) _______________________________________________
RouterOS cannot handle this, but Linux can. E.g.:
node has subnet 44.137.40.0/23 and has address 44.137.41.254 user has subnet 44.137.41.96/28 and has address 44.137.41.110 on eth0 then: ip route add 44.0.0.0/8 dev eth0 gw 44.137.41.254 onlink
To do this in RouterOS (and in many other routers) you need to set the subnet mask to /23 on the eth0, but that means it will try to contact the others directly, not routing via the node. This will work when the local area is switching/bridging.
Otherwise you will need to have a node address in everyone's subnet.
The same situation occurs when setting up the P2MP IPIP tunnel interface: you assign it the subnet of your local system, so all the routes are outside the interface's subnet and need the "dev" and "onlink" parameters. RouterOS does not support them. But Linux does.
You might not have found the solution yet. But RouterOS DOES support PtP addressing on PtMP interfaces.
/ip address add 44.137.41.110/32 network=44.137.41.254 interface=ether1 /ip route add dst-address=44.0.0.0/8 gateway=44.137.41.254
73 de Marc, LX1DUC
_________________________________________ 44Net mailing list 44Net@hamradio.ucsd.edu http://hamradio.ucsd.edu/mailman/listinfo/44net
On 2015-06-12 16:38, Marius Petrescu wrote:
(Please trim inclusions from previous messages) _______________________________________________ Marc, that is a completely different case. The problem is the fact that first of all, IPIP tunnels can not be point to multipoint (you get an error if the remote tunnel endpoint is not specified). Second, on Mikrotik, the route can not be bound to a specific interface lik in 'ip route' since it does not have the 'dev' parameter. Destination is either an IP or an interface. One can not have both. And third, if the gateway is not resolvable by a previous specific route (like in your example), then the system will issue an ARP, get no response and mark the route inactive. On linux systems, this is solved by adding the parameter "onlink" to the route, which is not possible on RouterOS.
And of course, there is the big issue, if all those could be solved, does the IPIP driver consider the route an endpoint directive (like on regular Linux) or just an ordinary route, but this is of no importance right now - it probably does since it is a modern kernel.
OK. Seems like I misunderstood.
Anyway, I'm using the python amprapi script from https://github.com/kd7lxl/python-amprapi, OK it is not RIP but AFAIK the "Master" RIP daemon is also provisioned via scheduled file transfers, so I just skip the detour via the RIP daemon and inject the routes directly from the output of the APMR Portal API. The world isn't perfect and neither is the implementation of the IPIP mesh on every single platform (wether it's RouterOS or Linux), so anybody should just use his preferred method. I had spare Routerboards lying around so why shouldn't I use them?
I do think that regardless of the OS it is much more important that anybody using 44net addresses shall support the IPIP mesh, regardless of any other routing procedures (e.g. direct BGP announcement, etc) in use. I have the "luxury" to be able to do direct BGP announcements, so I can reach BGP-only 44networks without NATing my 44net to my commercial IP address, others don't have this luxury but would eventually like to reach those 44etworks without the requirement for NAT.
just my 2 cents :-)
Have a nice weekend.
vy 73 de Marc, LX1DUC
On 6/12/15 11:06 AM, Marc, LX1DUC wrote:
I do think that regardless of the OS it is much more important that anybody using 44net addresses shall support the IPIP mesh
Why?
I announce subnets via BGP, that should be enough. I maintain the single end point for the 44 network (UCSD gw) is a bad idea, and it's not my fault it has broken routing for more specific networks.
Excelent question, Bryan.
IMHO the fact that not everybody can BGP announce their subnets could be a reason. In that case you can not differentiate between legitimate ampr traffic to your network and public traffic, since the access to your subnet has to be done via public IPs. But as long as you don't offer any network services which are specific for to the non-BGP announced ham community, there is not needed to support the mesh network. Your network is in this case just another IP range on the internet.
73s de Marius, YO2LOJ
-----Original Message----- From: 44net-bounces+marius=yo2loj.ro@hamradio.ucsd.edu [mailto:44net-bounces+marius=yo2loj.ro@hamradio.ucsd.edu] On Behalf Of Bryan Fields Sent: Friday, June 12, 2015 18:22 To: AMPRNet working group Subject: Re: [44net] Strange Broadcasts... ______________________________________________ On 6/12/15 11:06 AM, Marc, LX1DUC wrote:
I do think that regardless of the OS it is much more important that anybody using 44net addresses shall support the IPIP mesh
Why?
I announce subnets via BGP, that should be enough. I maintain the single end point for the 44 network (UCSD gw) is a bad idea, and it's not my fault it has broken routing for more specific networks.
-- Bryan Fields
On 2015-06-12 17:22, Bryan Fields wrote:
(Please trim inclusions from previous messages) _______________________________________________ On 6/12/15 11:06 AM, Marc, LX1DUC wrote:
I do think that regardless of the OS it is much more important that anybody using 44net addresses shall support the IPIP mesh
Why?
I announce subnets via BGP, that should be enough. I maintain the single end point for the 44 network (UCSD gw) is a bad idea, and it's not my fault it has broken routing for more specific networks.
There are several reasons, I'll just provide 3+1, but there are probably many many more:
- not everybody can do BGP - accessing your network will require NAT on the remote end (unless the YLs/OMs ISP allows her/him to originate IP packets with 44net addresses), NAT breaks end-to-end communications - you won't be able to differentiate between commercial access to your 44net and 44net traffic NATed to commercial IP - (I'm not sure about this) in case the YL/OM can send unNATed traffic and the YL/OM didn't configure a revDNS for his 44net IP, you won't be able to send a response back to her/his 44net as the UCSD gw will block traffic from the WAN interface to 44net addresses without revDNS.
I advocate for an united 44net where each participant shall be able to reach any other participant from a routing perspective (there could still be firewalls). I don't think that HamRadio (and I include 44net) is about islands that cannot communicate with each other. But those are just my thoughts. Your opinion my be different.
vy 73 de Marc
On 6/12/15 12:20 PM, Marc, LX1DUC wrote:
I announce subnets via BGP, that should be enough. I maintain the
single end point for the 44 network (UCSD gw) is a bad idea, and it's not my fault it has broken routing for more specific networks.
There are several reasons, I'll just provide 3+1, but there are probably many many more:
- not everybody can do BGP
Granted. But saying the only other solution is via a single world wide gateway is down right silly in 2015. I'm actually providing redundant connections to my users via BGP, matter of fact we just turned up another subnet for another AMPRnet /24 this week via BGP.
- accessing your network will require NAT on the remote end (unless the
YLs/OMs ISP allows her/him to originate IP packets with 44net addresses), NAT breaks end-to-end communications
I don't understand what part NAT plays in this.
- you won't be able to differentiate between commercial access to your
44net and 44net traffic NATed to commercial IP
Again, I don't understand where NAT fits in this discussion/model. I don't differentiate between a 44/8 sourced IP and another. It's all internet traffic, there is no inherent security/authentication of 44/8 addresses.
I advocate for an united 44net where each participant shall be able to reach any other participant from a routing perspective (there could still be firewalls).
I agree, however the configuration of a single gateway announcing 44/8 without the ability to reach more specific networks is _broken_ routing. Let me say this again:
_The UCSD Gateway has BROKEN ROUTING affecting the REACHABILITY of IPIP users._
If BGP users announce a subnet that 99.99999% of the internet can see, but IPIP users behind the UCSD gateway can't reach it, it's not BGP users that have broken routing, it's the silly UCSD gateway.
I've advocated fixing the broken routing at UCSD, but the has been no positive movement in 3+ years to fix this.
I'm not going to touch this bone-headed idea of having a single gateway world wide for the IPIP traffic, other than to say it's bad and broken more than it's working.
I don't think that HamRadio (and I include 44net) is about islands that cannot communicate with each other. But those are just my thoughts. Your opinion my be different.
Well that's the thing, as the UCSD gateway is implemented now, it enforces islands of routing; the IPIP users are basically their own VPN on top of the internet with special addresses. It's a quasi-private GRX like network.
The way to fix this so it works for the legacy IPIP users and standards compliant BGP users of the AMPRNET space is to fix the routing at the gateway. This is simple conceptually and in practice, but as ARDC is not a members organization there is little that can be done other than bitch about it on the list.
Hello Bryan,
Am 13. Jun 2015, um 03:40:28 schrieb Bryan Fields:
- not everybody can do BGP
Granted. But saying the only other solution is via a single world wide gateway is down right silly in 2015.
Oh, it has been subject of considerable debate for at least 20 years. The general problem is, if you have multiple entries into the one AMPR network, under different management, how do you prevent split-brain situations ?
I'm actually providing redundant connections to my users via BGP, matter of fact we just turned up another subnet for another AMPRnet /24 this week via BGP.
Technically, no, if you are not part of the IPIP cloud, you do not. You form your own island, hopefully related to amateur radio digital comms, using 44 IP space, and connect that, possibly redundantly, to the internet.
- accessing your network will require NAT on the remote end (unless the
YLs/OMs ISP allows her/him to originate IP packets with 44net addresses), NAT breaks end-to-end communications
I don't understand what part NAT plays in this.
Assume I run a 44.x machine at home, and I am not connected via BGP, without IPIP I can reach you only from my commercial IP, hence I have to implement NAT. breaking 44<->44 transparency.
Hence we would have no transparent 44<->44 connectivity anymore.
- you won't be able to differentiate between commercial access to your
44net and 44net traffic NATed to commercial IP
Again, I don't understand where NAT fits in this discussion/model. I don't differentiate between a 44/8 sourced IP and another. It's all internet traffic, there is no inherent security/authentication of 44/8 addresses.
That is only you. Others do. While it is not a particularly strong authentication, people do assume that traffic originating from 44.x is originating from or on behalf of a licensed ham, in particular if it is TCP traffic with successful handshake.
If this were "all internet traffic" anyway, then you shouldnt be using 44. address space.
I agree, however the configuration of a single gateway announcing 44/8 without the ability to reach more specific networks is _broken_ routing. Let me say this again:
Actually, no. The conceptual breakage starts when, in the global BGP, more specific routes are announced from a different AS. 44/8 is assigned to ARDC/Brian, and is announced via the AS of UCSD. That ought to be an authoritative statement.
With modern traffic engineering like anycast etc. it is useful to slightly bend the rules. In practice, such non-standard BGP configurations can still be made to work, if the parties within the affected (AMPR-) net cooperate and are willing to apply some further "non-standard" configuration.
If BGP users announce a subnet that 99.99999% of the internet can see, but IPIP users behind the UCSD gateway can't reach it, it's not BGP users that have broken routing, it's the silly UCSD gateway.
You are confusing Internet BGP users, and 44. users. When you are concerned about AMPR, you should be concerned about 44. first, and here your statistics will look way worse.
The situation can be fixed via two ways:
1)you set up an IPIP endpoint via which UCSD and all 44.x can reach your network. You, being a BGP based ISP, do consider this a non-standard solution.
2)UCSD's outer BGP gateway (which is not the box that Brian controls) currently routes 44/8 to Brian's gateway, likely via a high preference static route. That router would have to be taught either all the currently valid 44/x routes, or the exceptions. That involves, at a minimum, lots of effort for the UCSD network admins; and certainly they do not consider this a standard solution.
Well that's the thing, as the UCSD gateway is implemented now, it enforces islands of routing; the IPIP users are basically their own VPN on top of the internet with special addresses. It's a quasi-private GRX like network.
Exactly. It is NOT the internet. By design and on purpose. We are free to experiment and explore in there, without the fear of causing worldwide internet BGP instability, for example. If we wanted to play with the real internet, we could do that (you, having BGP access, being proof). Then we wouldnt need 44/ IPs.
The way to fix this so it works for the legacy IPIP users and standards compliant BGP users of the AMPRNET space is to fix the routing at the gateway. This is simple conceptually and in practice, but as ARDC is not a members organization there is little that can be done other than bitch about it on the list.
No. Thats only the one of the two ways you prefer.
Since UCSD is donating the connectivity for all the rest of the 44/ users, and option 2 involves a signifcant burden to them, and ARDC, being the authority over 44/8 apparently agree, how about you implementing #1 ?
Mario
Mario,
My take from your description of what things should be like:
It is NOT the internet. By design and on purpose. We are free to experiment and explore in there
Is that you are thinking of private address space. Use 10/8, or 192.168/16, or one of the others. That space is specifically designated to not be publicly routable. If your ideal is a private VPN mesh, that’s the space to do it in. the 44/8 space *IS* part of the internet. It *IS* BGP announced, and there are smaller subsections that are announced separately.
Brian’s point is entirely accurate. If the routing issues at UCSD got fixed, everyone could talk to everyone else just fine, including the rest of the internet. It was previously mentioned the benefits of the “authentication” coming from a 44/8 address provides:
That is only you. Others do. While it is not a particularly strong authentication, people do assume that traffic originating from 44.x is originating from or on behalf of a licensed ham, in particular if it is TCP traffic with successful handshake.
If the routing were fixed, it would still work that way. Routing for non BGP announced subnets would go through UCSD through the tunnels, and would show up with the 44/8 source address like you see from your current tunnel partners. So, your statement here, would be incorrect:
Assume I run a 44.x machine at home, and I am not connected via BGP, without IPIP I can reach you only from my commercial IP, hence I have to implement NAT. breaking 44<->44 transparency.
My understanding of the organization here is that 44/8 space is for the amateur radio community to use and experiment with. There’s nothing that says it can’t be directly announced, or that we have to use it for Part 97 traffic, or whatever else. We’re amateur operators, and each of us uses the space in different ways. Some of us happen to have some technical resources available that let us direct announce our subnets and take UCSD out of a SPOF for internet connectivity.
Actually, no. The conceptual breakage starts when, in the global BGP, more specific routes are announced from a different AS. 44/8 is assigned to ARDC/Brian, and is announced via the AS of UCSD. That ought to be an authoritative statement.
With modern traffic engineering like anycast etc. it is useful to slightly bend the rules. In practice, such non-standard BGP configurations can still be made to work, if the parties within the affected (AMPR-) net cooperate and are willing to apply some further "non-standard" configuration.
This is not non-standard. This is exactly how routing is supposed to behave, more specific routes win over less specific routes. Those of us who BGP announce our subnets have more specific routes, and traffic comes to us. The issue comes when routers have statically configured general routes (like at UCSD), and ignore the *ENTIRELY VALID* more specific routes. This is, again, what breaks connectivity between IPIP users, and the BGP users. If the UCSD gateway had proper routing, they could talk fine, with proper source and dest addresses as I talked about above.
Since UCSD is donating the connectivity for all the rest of the 44/ users, and option 2 involves a signifcant burden to them, and ARDC, being the authority over 44/8 apparently agree, how about you implementing #1 ?
I’m aware of several folks who have offered to provide redundant announces, and the HamWAN project I’m part of is working on a peering policy so that anyone who wants to BGP announce can do so through us. I appreciate that UCSD is donating bandwidth to the project, but the routing is *factually* causing issues. I’d like to see it fixed, and then we can all go about our merry way without having these ridiculous threads about what’s “right”. If you want to do IPIP, do that, if you want to do BGP, do that too. Routing should work in either case, and right now it’s not.
Nigel
On Jun 13, 2015, at 05:19, Mario Lorenz ml@vdazone.org wrote:
(Please trim inclusions from previous messages) _______________________________________________ Hello Bryan,
Am 13. Jun 2015, um 03:40:28 schrieb Bryan Fields:
- not everybody can do BGP
Granted. But saying the only other solution is via a single world wide gateway is down right silly in 2015.
Oh, it has been subject of considerable debate for at least 20 years. The general problem is, if you have multiple entries into the one AMPR network, under different management, how do you prevent split-brain situations ?
I'm actually providing redundant connections to my users via BGP, matter of fact we just turned up another subnet for another AMPRnet /24 this week via BGP.
Technically, no, if you are not part of the IPIP cloud, you do not. You form your own island, hopefully related to amateur radio digital comms, using 44 IP space, and connect that, possibly redundantly, to the internet.
- accessing your network will require NAT on the remote end (unless the
YLs/OMs ISP allows her/him to originate IP packets with 44net addresses), NAT breaks end-to-end communications
I don't understand what part NAT plays in this.
Assume I run a 44.x machine at home, and I am not connected via BGP, without IPIP I can reach you only from my commercial IP, hence I have to implement NAT. breaking 44<->44 transparency.
Hence we would have no transparent 44<->44 connectivity anymore.
- you won't be able to differentiate between commercial access to your
44net and 44net traffic NATed to commercial IP
Again, I don't understand where NAT fits in this discussion/model. I don't differentiate between a 44/8 sourced IP and another. It's all internet traffic, there is no inherent security/authentication of 44/8 addresses.
That is only you. Others do. While it is not a particularly strong authentication, people do assume that traffic originating from 44.x is originating from or on behalf of a licensed ham, in particular if it is TCP traffic with successful handshake.
If this were "all internet traffic" anyway, then you shouldnt be using 44. address space.
I agree, however the configuration of a single gateway announcing 44/8 without the ability to reach more specific networks is _broken_ routing. Let me say this again:
Actually, no. The conceptual breakage starts when, in the global BGP, more specific routes are announced from a different AS. 44/8 is assigned to ARDC/Brian, and is announced via the AS of UCSD. That ought to be an authoritative statement.
With modern traffic engineering like anycast etc. it is useful to slightly bend the rules. In practice, such non-standard BGP configurations can still be made to work, if the parties within the affected (AMPR-) net cooperate and are willing to apply some further "non-standard" configuration.
If BGP users announce a subnet that 99.99999% of the internet can see, but IPIP users behind the UCSD gateway can't reach it, it's not BGP users that have broken routing, it's the silly UCSD gateway.
You are confusing Internet BGP users, and 44. users. When you are concerned about AMPR, you should be concerned about 44. first, and here your statistics will look way worse.
The situation can be fixed via two ways:
1)you set up an IPIP endpoint via which UCSD and all 44.x can reach your network. You, being a BGP based ISP, do consider this a non-standard solution.
2)UCSD's outer BGP gateway (which is not the box that Brian controls) currently routes 44/8 to Brian's gateway, likely via a high preference static route. That router would have to be taught either all the currently valid 44/x routes, or the exceptions. That involves, at a minimum, lots of effort for the UCSD network admins; and certainly they do not consider this a standard solution.
Well that's the thing, as the UCSD gateway is implemented now, it enforces islands of routing; the IPIP users are basically their own VPN on top of the internet with special addresses. It's a quasi-private GRX like network.
Exactly. It is NOT the internet. By design and on purpose. We are free to experiment and explore in there, without the fear of causing worldwide internet BGP instability, for example. If we wanted to play with the real internet, we could do that (you, having BGP access, being proof). Then we wouldnt need 44/ IPs.
The way to fix this so it works for the legacy IPIP users and standards compliant BGP users of the AMPRNET space is to fix the routing at the gateway. This is simple conceptually and in practice, but as ARDC is not a members organization there is little that can be done other than bitch about it on the list.
No. Thats only the one of the two ways you prefer.
Since UCSD is donating the connectivity for all the rest of the 44/ users, and option 2 involves a signifcant burden to them, and ARDC, being the authority over 44/8 apparently agree, how about you implementing #1 ?
Mario
-- Mario Lorenz Internet: ml@vdazone.org Ham Radio: DL5MLO@DB0ERF.#THR.DEU.EU _________________________________________ 44Net mailing list 44Net@hamradio.ucsd.edu http://hamradio.ucsd.edu/mailman/listinfo/44net
On Fri, Jun 12, 2015 at 05:06:21PM +0200, Marc, LX1DUC wrote:
... AFAIK the "Master" RIP daemon is also provisioned via scheduled file transfers ...
Actually, the RIP daemon directly accesses the portal's database so changes to the gateways are seen within a few minutes of them being updated on the portal. - Brian
-
Brian Kantor -
Bryan Fields -
Marc, LX1DUC -
Mario Lorenz -
Marius Petrescu -
Nigel Vander Houwen -
Rob Janssen