29 Jan
2014
29 Jan
'14
3:04 p.m.
On 1/29/14 2:05 PM, Steve Wright wrote:
This mesh crap really needs to be binned, or at the
very least not try and
do anything important over it, such as route an entire /16. If you want to
connect a /24 with it to make a neat local play toy then go for it, but
using it as an enterprise routing tool is absurd at the very least, and at
it's WORST, it's very likely to just completely stop anyone from trying to
build anything new over it because it's connectivity and throughput sucks.
This.
So this is how I'd see it work, I need to write up a proposal for it.
You have regional BGP routers that route subnets to the internet. These could
then tunnel the subnets to end users via GRE. End users could route via an
IGP over this tunnel to the regional speaker(s). Multiple tunnels would give
redundancy.
The regional speakers would have a tunnel between them.
In the event of an outage the other BGP speakers would route subnets.
Multiple links from end users to other BGP speakers (or non-speakers that are
aggravation routers) would provide redundancy to the end users.
Of course nothing prevents having a direct BGP speaker with an RF link to end
users, most data centers will not have roof rights however.
We could setup redistribution that would pull announcements from BGP if end
nodes went down.
Each BGP speaker could announce the subnets it knows about and a /8 providing
we have a mesh of the backbone bgp speakers.
Thoughts?
--
Bryan Fields
727-409-1194 - Voice
727-214-2508 - Fax
http://bryanfields.net
29 Jan
29 Jan
3:20 p.m.
New subject: Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
That is basically what I have in mind. Though I would allow the regionals
to determine the VPN protocols to reach the subnets (more than one,
including possibly IPIP).
Then we should have plug and play configurations for subnets. For example,
I can provision a MikroTik with a VPN to a datacenter and deliver it to
subnet admin for $50-100, who places on their LAN with connectivity to an
ISP. Going out from there to RF or a service is the responsibility of the
local subnet.
------------------------------
John D. Hays
K7VE
PO Box 1223, Edmonds, WA 98020-1223
<http://k7ve.org/blog> <http://twitter.com/#!/john_hays>
<http://www.facebook.com/john.d.hays>
On Wed, Jan 29, 2014 at 12:04 PM, Bryan Fields <Bryan(a)bryanfields.net>wrote;wrote:
(Please trim inclusions from previous messages)
_______________________________________________
On 1/29/14 2:05 PM, Steve Wright wrote:
This mesh crap really needs to be binned, or at
the very least not try
and
do anything important over it, such as route an
entire /16. If you want
to
connect a /24 with it to make a neat local play
toy then go for it, but
using it as an enterprise routing tool is absurd at the very least, and
at
it's WORST, it's very likely to just
completely stop anyone from trying
to
build anything new over it because it's
connectivity and throughput
sucks.
This.
So this is how I'd see it work, I need to write up a proposal for it.
You have regional BGP routers that route subnets to the internet. These
could
then tunnel the subnets to end users via GRE. End users could route via an
IGP over this tunnel to the regional speaker(s). Multiple tunnels would
give
redundancy.
The regional speakers would have a tunnel between them.
In the event of an outage the other BGP speakers would route subnets.
Multiple links from end users to other BGP speakers (or non-speakers that
are
aggravation routers) would provide redundancy to the end users.
Of course nothing prevents having a direct BGP speaker with an RF link to
end
users, most data centers will not have roof rights however.
We could setup redistribution that would pull announcements from BGP if end
nodes went down.
Each BGP speaker could announce the subnets it knows about and a /8
providing
we have a mesh of the backbone bgp speakers.
Thoughts?
--
Bryan Fields
727-409-1194 - Voice
727-214-2508 - Fax
http://bryanfields.net
_________________________________________
44Net mailing list
44Net(a)hamradio.ucsd.edu
http://hamradio.ucsd.edu/mailman/listinfo/44net
6:15 p.m.
New subject: Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
John would like to talk with you as I would be interested to know if mikrotik can be used
instead of the rip44d.
Thanks
Sent from my iPhone
> On Jan 29, 2014, at 3:20 PM, K7VE - John <k7ve(a)k7ve.org> wrote:
>
> (Please trim inclusions from previous messages)
> _______________________________________________
> That is basically what I have in mind. Though I would allow the regionals
> to determine the VPN protocols to reach the subnets (more than one,
> including possibly IPIP).
>
> Then we should have plug and play configurations for subnets. For example,
> I can provision a MikroTik with a VPN to a datacenter and deliver it to
> subnet admin for $50-100, who places on their LAN with connectivity to an
> ISP. Going out from there to RF or a service is the responsibility of the
> local subnet.
>
>
> ------------------------------
> John D. Hays
> K7VE
> PO Box 1223, Edmonds, WA 98020-1223
> <http://k7ve.org/blog> <http://twitter.com/#!/john_hays>
> <http://www.facebook.com/john.d.hays>
>
>
> On Wed, Jan 29, 2014 at 12:04 PM, Bryan Fields <Bryan(a)bryanfields.net>wrote;wrote:
>
>> (Please trim inclusions from previous messages)
>> _______________________________________________
>>> On 1/29/14 2:05 PM, Steve Wright wrote:
>>> This mesh crap really needs to be binned, or at the very least not try
>> and
>>> do anything important over it, such as route an entire /16. If you want
>> to
>>> connect a /24 with it to make a neat local play toy then go for it, but
>>> using it as an enterprise routing tool is absurd at the very least, and
>> at
>>> it's WORST, it's very likely to just completely stop anyone from
trying
>> to
>>> build anything new over it because it's connectivity and throughput
>> sucks.
>>
>> This.
>>
>> So this is how I'd see it work, I need to write up a proposal for it.
>>
>> You have regional BGP routers that route subnets to the internet. These
>> could
>> then tunnel the subnets to end users via GRE. End users could route via an
>> IGP over this tunnel to the regional speaker(s). Multiple tunnels would
>> give
>> redundancy.
>>
>> The regional speakers would have a tunnel between them.
>>
>> In the event of an outage the other BGP speakers would route subnets.
>> Multiple links from end users to other BGP speakers (or non-speakers that
>> are
>> aggravation routers) would provide redundancy to the end users.
>>
>> Of course nothing prevents having a direct BGP speaker with an RF link to
>> end
>> users, most data centers will not have roof rights however.
>>
>> We could setup redistribution that would pull announcements from BGP if end
>> nodes went down.
>>
>> Each BGP speaker could announce the subnets it knows about and a /8
>> providing
>> we have a mesh of the backbone bgp speakers.
>>
>> Thoughts?
>>
>> --
>> Bryan Fields
>>
>> 727-409-1194 - Voice
>> 727-214-2508 - Fax
>> http://bryanfields.net
>> _________________________________________
>> 44Net mailing list
>> 44Net(a)hamradio.ucsd.edu
>> http://hamradio.ucsd.edu/mailman/listinfo/44net
>
7:25 p.m.
New subject: Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
On Wed, Jan 29, 2014 at 3:15 PM, Chad <cstar707(a)gmail.com> wrote:
John would like to talk with you as I would be
interested to know if mikrotik can be used instead of the rip44d.
Yes, I wrote a script to create the IPIP tunnels and routes on ROS,
just as rip44d does for Linux. We (HamWAN) are using this script to
add the AMPR IPIP mesh to our BGP-announced subnet. This is necessary
because of the router upstream of amprgw that sends 44/8 back to
amprgw.
https://github.com/HamWAN/hamwan_scripts/tree/master/amprupdate
Unfortunately, this doesn't run on the Mikrotik device itself* (I
could not find a way to parse the encap file with ROS scripting
language). It will need to be run on another device that has Python
and can SSH into your Mikrotik router. Run it via cron on a regular
interval to stay up-to-date with the encap file. You'll need to
replace the addresses on lines 27 and 28 with your subnet and Internet
address[es].
Tom KD7LXL
* It's theoretically possible to run this script, or even rip44d, in a
Linux VM on your Mikrotik router, therefore not requiring another
machine.
8:32 p.m.
New subject: Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
Thanks Tom will look into this thanks for sharing.
will try to set this up this weekend.
Chad Starling
4164772423
011 883 5100 0990 0046 INUM
VA3CWS
On Wed, Jan 29, 2014 at 7:25 PM, Tom Hayward <esarfl(a)gmail.com> wrote:
(Please trim inclusions from previous messages)
_______________________________________________
On Wed, Jan 29, 2014 at 3:15 PM, Chad <cstar707(a)gmail.com> wrote:
John would like to talk with you as I would be
interested to know if
mikrotik can be used instead of the rip44d.
Yes, I wrote a script to create the IPIP tunnels and routes on ROS,
just as rip44d does for Linux. We (HamWAN) are using this script to
add the AMPR IPIP mesh to our BGP-announced subnet. This is necessary
because of the router upstream of amprgw that sends 44/8 back to
amprgw.
https://github.com/HamWAN/hamwan_scripts/tree/master/amprupdate
Unfortunately, this doesn't run on the Mikrotik device itself* (I
could not find a way to parse the encap file with ROS scripting
language). It will need to be run on another device that has Python
and can SSH into your Mikrotik router. Run it via cron on a regular
interval to stay up-to-date with the encap file. You'll need to
replace the addresses on lines 27 and 28 with your subnet and Internet
address[es].
Tom KD7LXL
* It's theoretically possible to run this script, or even rip44d, in a
Linux VM on your Mikrotik router, therefore not requiring another
machine.
_________________________________________
44Net mailing list
44Net(a)hamradio.ucsd.edu
http://hamradio.ucsd.edu/mailman/listinfo/44net
3:33 p.m.
New subject: Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
Hi Bryan,
You have regional BGP routers that route subnets to
the internet. These
could
then tunnel the subnets to end users via GRE. End
users could route via
an
IGP over this tunnel to the regional speaker(s).
Multiple tunnels would
give
redundancy.
This is exactly what BGP enabled subnets are intended for and the one's
already set up work as described.
What you do in that subnet is every subnet's own busines.
But why GRE? it has a bigger overhead compared to IPIP...
The regional speakers would have a tunnel between
them.
Also, everyone can link their directly routed subnets as they wish. The
current arhitecture doesn't prohibit such links.
In the event of an outage the other BGP speakers would
route subnets.
Multiple links from end users to other BGP speakers (or non-speakers that
are
aggravation routers) would provide redundancy to the
end users.
Again this is a subnet internal isuue what you du with your end users.
Of course nothing prevents having a direct BGP speaker
with an RF link to
end
users, most data centers will not have roof rights
however.
The current setups allow the same thing. No one disallows this.
We could setup redistribution that would pull
announcements from BGP if
end
nodes went down.
If you have redundant BGP enabled routers in the BGP announced subnet this
will happen.
Each BGP speaker could announce the subnets it knows
about and a /8
providing
we have a mesh of the backbone bgp speakers.
This is how BGP works... The ampr gateway is the /8 announcer
So nothing new in these idees. It is exactly how the BGP enabled subnets
work right now.
As I stated, the rest of the ampr network is just another BGP announced
subnet, this time the /8 covering the address space not announced by other
BGP enabled subnets.
So I really don't understand the issue. This is exactly what we have right
now. What you do in your own subnet behind the registered gw or behind your
BGP router is your personal stuff and does not affect the rest of the ampr
network. If you want extra GRE tunnels, set them up, if you want redundant
links via tunnels with other announcers, do it. Everything is compliant with
our current setup.
And if you don't want IPIP use BGP routed acces and you will not need it
anymore. For acces to hosts outside your network, you will be routed via
amprgw, and everything will work.
But please don't try to enforce your internal network householding on
others, since not everyone can afford a BGP subnet. In my case a BGP enabled
acces is considered of professional use and is about my monthly income,
which of course I am not able to support.
73 de Marius, YO2LOJ
30 Jan
30 Jan
12:47 a.m.
New subject: Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
On Wed, 29 Jan 2014, Marius Petrescu wrote:
And if you don't want IPIP use BGP routed acces
and you will not need it
anymore. For acces to hosts outside your network, you will be routed via
amprgw, and everything will work.
Sorry, did I accidentally miss an email or two, has something changed
around amprgw that would make the above happen?
Last time I checked, amprgw could not route out any unencapsulated packets
that have a destination address within 44/8. These would be packets from
the IPIP-connected gateways going to a BGP-only site (most IPIP sites can
not send unencapsulated outgoing packets with 44/8 source addresses due to
spoofing filtering at ISPs). The reason was that UCSD's internal network
routes all 44/8 destined packets to amprgw, so amprgw can not send packets
to 44/8 BGP sites at all.
As I understand it, currently all BGP sites must have an IPIP gateway too
to enable connectivity with all the rest of the non-BGP sites.
Sorry for the noise if things have changed since.
But please don't try to enforce your internal
network householding on
others, since not everyone can afford a BGP subnet. In my case a BGP enabled
acces is considered of professional use and is about my monthly income,
which of course I am not able to support.
Agreeing with all the other things you wrote!
We have a BGP setup over here now, and we are locally routing the subnets
forward with either IPIP or OpenVPN (could do GRE if needed), and we could
set up an IGP on top of those just as well.
Further, if there would be a local gateway, with a subnet within
44.139/16, that has a Gateway entry in the portal, it'd automatically get
direct IPIP routing via our BGP gateway (instead of going via UCSD)
because we have the IPIP endpoint present as well (and it gets the RIP
updates).
- Hessu, OH7LZB
1:41 a.m.
New subject: Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
Tnx Hessu for the correction.
What I meant (and what seems logical to me) whas that if you try to reach an
ampr host in the mesh network (let's call a host here M) from a bgp routed
subnet, than you have tis scenario:
- On the router of the upstram provider there shpuld be 2 routes. One to the
BGP'd subnet (let's assume it called A 44.1.1.0/24) and one to 44.0.0.0/8
which is amprgw which is the gw for any 44 traffic.
- If you try a connection from A to M, packets from A will go out "in the
wild" and will be routed to amprgw, encapsulated, and sent to M, like any
internet to ampr access. The responses will flow back the correct route
because of conntrack.
- Any two bgp enabled subntes will talk to each other like any subnet ion
the internet.
- Outgoing connections from M will not work since amprgw does not allow
outgoing connects to 44 addresses. IMHO this rule should be refined to be
"drop outgoing connections to hosts present in the encap file" which will
solve the problem. But that's another thing.
- Now, since A is not in the encap file, I would expect any gateway to
masquerade the traffic to A to its own public IP, ensuring a regular
connection to it, like a connection to any internet host. The only issue in
this case is that the originator will not be identified as an ampr host, so
here come those direct tunnels into play, and of course, BGP can be used
internally to get dynamic routing. Allthough I favor OSPF for intranet
routing.
Maybe now I was more explicit :-)
Marius, YO2LOJ
-----Original Message-----
From: 44net-bounces+marius=yo2loj.ro(a)hamradio.ucsd.edu
[mailto:44net-bounces+marius=yo2loj.ro@hamradio.ucsd.edu] On Behalf Of
Heikki Hannikainen
Sent: Thursday, January 30, 2014 07:47
To: AMPRNet working group
Subject: Re: [44net] Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
Last time I checked, amprgw could not route out any unencapsulated packets
that have a destination address within 44/8. These would be packets from
the IPIP-connected gateways going to a BGP-only site (most IPIP sites can
not send unencapsulated outgoing packets with 44/8 source addresses due to
spoofing filtering at ISPs). The reason was that UCSD's internal network
routes all 44/8 destined packets to amprgw, so amprgw can not send packets
to 44/8 BGP sites at all.
As I understand it, currently all BGP sites must have an IPIP gateway too
to enable connectivity with all the rest of the non-BGP sites.
3:47 a.m.
New subject: Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
On Thu, 30 Jan 2014, Marius Petrescu wrote:
What I meant (and what seems logical to me) whas that
if you try to reach an
ampr host in the mesh network (let's call a host here M) from a bgp routed
subnet, than you have tis scenario:
- On the router of the upstram provider there shpuld be 2 routes. One to the
BGP'd subnet (let's assume it called A 44.1.1.0/24) and one to 44.0.0.0/8
which is amprgw which is the gw for any 44 traffic.
- If you try a connection from A to M, packets from A will go out "in the
wild" and will be routed to amprgw, encapsulated, and sent to M, like any
internet to ampr access. The responses will flow back the correct route
because of conntrack.
conntrack? I don't think the gateways out there are running any sort of
routing that would magically route back return packets "the same way". If
connection tracking is used, it might be applied to firewall rules, but
not routing. Besides, that "back the same way" would point to amprgw, and
again, amprgw is not able to route those return packets back to A.
M will use the routing table it has, which might be one of:
1) default route to the Internet (will get dropped at upstream ISP due
to the 44/8 source address)
2) some folks have a 44/8 route pointing to amprgw (won't work since
amprgw currently cannot send packets to the 44/8 BGP sites)
3) if the BGP site has a Gateways database entry and an IPIP receiving
endpoint, that will be used, and things work.
- Any two bgp enabled subntes will talk to each other
like any subnet ion
the internet.
Correct.
- Outgoing connections from M will not work since
amprgw does not allow
outgoing connects to 44 addresses. IMHO this rule should be refined to be
"drop outgoing connections to hosts present in the encap file" which will
solve the problem. But that's another thing.
amprgw itself does allow them, but its local upstream routers will give
all packets destined to 44/8 back to amprgw.
The reality is that all outgoing *packets* from amprgw to 44/8, _unless_
_encapsulated_, will not go anywhere, since UCSD routers route 44/8 to
amprgw and do not know about the more specific BGP routes.
That is not affected by the direction of a TCP connection, conntrack, or
anything else.
- Hessu
4:25 a.m.
New subject: Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
Hi, I would appreciate any advice from this thread from someone that
actually managed to introduce local BGP peering of a subnet and to set
up whatever it takes not to disconnect this subnet from the rest of the
44 network.
I am coordinating the Swedish delegation 44.140.0.0/16. Last year we
made a complete restart of this network since it was totally dead after
a very active pioneer period. I was unfortunately not part of that
period and lack experience from the tunnelled version of the network.
We now have an agreement both with ARDC and SUNET (the Swedish
University Network) to provide Internet transit for this space locally
via SUNET. SUNET is connected to Nordunet, which is connected to the
European academic backbone GÉANT, which is connected all over the world.
SUNET accepts announcements of subnets down to /24. This makes it
possible for us to spread SUNET/se.amprnet gateways at SUNET points of
presence all over the country. We are in the process of connecting
amateur clubs to such gateways that will serve their individual members.
We are also adding ipv6 support into this arrangement.
Is it correctly understood from this thread that we should set up a fat
tunnel in se.ampr between SUNET and 44.0.0.1? and route to 44.0.0.0/8
through it? There has been talk about both gre and ipip-tunnels. In
cases where we still have to tunnel in 44.140/16 (hopefully temporarily)
we are using gre-tunnels since they also support ipv6. But I guess we
can use either for this particular tunnel.
Anything else to think about to get it working?
Bjorn/sa0bxi
11:09 a.m.
New subject: Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
Let me get this straight: UCSD routers are not BGP enabled?
Marius, YO2LOJ
-----Original Message-----
From: 44net-bounces+marius=yo2loj.ro(a)hamradio.ucsd.edu
[mailto:44net-bounces+marius=yo2loj.ro@hamradio.ucsd.edu] On Behalf Of
Heikki Hannikainen
Sent: Thursday, January 30, 2014 10:48
To: AMPRNet working group
Subject: Re: [44net] Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
amprgw itself does allow them, but its local upstream routers will give
all packets destined to 44/8 back to amprgw.
The reality is that all outgoing *packets* from amprgw to 44/8, _unless_
_encapsulated_, will not go anywhere, since UCSD routers route 44/8 to
amprgw and do not know about the more specific BGP routes.
11:15 a.m.
New subject: Updated BGP design
On Thu, Jan 30, 2014 at 06:09:13PM +0200, Marius Petrescu wrote:
Let me get this straight: UCSD routers are not BGP
enabled?
The border routers are, of course, BGP enabled. However, there are
several routers between the campus border and amprgw, and those
intermediate routers don't do BGP because they're not on the border.
I hope to have that changed someday but re-engineering the entire
campus network architecture to accomodate one host isn't a high priority.
- Brian
1 Feb
1 Feb
7:55 a.m.
New subject: Updated BGP design
On 01/30/2014 05:15 PM, Brian Kantor wrote:
The border routers are, of course, BGP enabled.
However, there are
several routers between the campus border and amprgw, and those
intermediate routers don't do BGP because they're not on the border.
But
shouldnt it be possible to have a single fat tunnel between amprgw
and a border router of an other bgp island, for example se.ampr.org,
which is managed by the IGP (in our case ospf) rather than bgp?
29 Jan
29 Jan
7:14 p.m.
New subject: Updated BGP design (was 44Net Digest, Vol 3, Issue 20)
You have regional BGP routers that route subnets to
the internet. These could
then tunnel the subnets to end users via GRE. End users could route via an
IGP over this tunnel to the regional speaker(s). Multiple tunnels would give
redundancy.
Not throwing stones here but we all saw what happen when URO went down a few years ago.
This topology did not work well for us, I would have seen Hub0 as a regional routers.
People lives change all the time so keeping it simple work well and has worked well for a
lot of years.
I for one would prefer to route my own Gateway even if I have to deal with a little
latency.
Jerry, kd4yal
3758
days inactive
3761
days old
13 comments
10 participants
participants (10)
-
Bjorn Pehrson -
Brian Kantor -
Bryan Fields -
Chad -
Chad Starling
-
Heikki Hannikainen -
Jerald A DeLong -
K7VE - John -
Marius Petrescu -
Tom Hayward