> Excellent thank you I now have one on order.
> Does anyone know if the setup on the wiki is still current or given recent changes is there a new one?
Well, the text about MikroTik router setup on the wiki is not something you would want to do, as it only
installs a single tunnel to UCSD, but at the bottom it refers to the YO2LOJ site and the script you can
find there (get the latest version) and the directions for installing it are the way to go to get it on
the IPIP mesh.
Of course you can use it in a lot of other ways (also at the same time), depending on the local radio
network available. We use these routers (from their availability mostly RB750Gr3, before that some
RB2011 and RB750 versions) as routers in our radio network as well, each running a couple of links to
neighbors and using BGP as an autorouting protocol. In some areas it is also possible to setup a VPN
to some nearby gateway that is on the mesh and/or is directly routed on Internet.
Rob
> I think the RB750GR3 is the best bang for your buck if you don't need
> POE or SFP.https://routerboard.com/RB750Gr3
It sure is! Make sure you get the r3 version, not the earlier ones.
It is much more powerful than the RB750 (no G) and RB2011.
(of course it has only 5 ports and no WiFi)
Rob
> Try turning off RPF (return path filtering at the kernel level) if it goes out on one interface and comes in the other, then RPF is almost always at fault as it will by default drop the connection
I expect it is something invisible and nasty like that, but rp_filter is not active on that system.
(I never enable this because it is so difficult to monitor what it does... on gw-44-137 I use packet
marking using the "rpfilter" matcher in mangle and drop of the marked packets with logging in the filter)
Rob
> The strange thing is that ping works ok when TCP doesn't connect.
> My first suspicion would be a stateful firewall, but I'm sure you
> checked that. Could it be a TTL problem? I'm just guessing here.
The TTL of the inside IP packet is 63. I first traced on the external interface
and saw the encapsulated packet, then I traced on the tunl0 interface and saw
the decapsulated packet (same without the outer IP header), and it all looks OK.
I see the SYN going out,the SYN ACK coming in, but nothing more (ACK should go out).
The firewall is stateful but I added an explicit accept for -s 44.0.0.1 at the
top of all the rules to make sure it is not that. Also, I reset the firewall and
watched the counters, did not see any packets being dropped.
And indeed, ping works OK. It is strange. Maybe something is wrong due to the
gateway external address change, although I would not know what could produce
the above scenario. The system is up for about a year, maybe I should try
rebooting it. (usually this brings nothing when I try it... it isn't Windows :-)
Of course sometime it will become clear how this can be explained.
Rob
> I'm curious as to how long you waited when trying port 25. There
> is a five-second delay after the connection is established before
> it issues the 220 greeting
I noticed that. But on 44.137.40.2 the TCP connection does not even
establish.
I tried from home (44.137.41.97) and from the gw-44-137 (44.137.0.1)
and both were able to connect:
telnet 44.0.0.1 25
Trying 44.0.0.1...
Connected to 44.0.0.1.
Escape character is '^]'.
220 gw.ampr.org ESMTP Sendmail 8.15.2/8.15.2; Thu, 13 Jul 2017 09:20:58 -0700 (PDT)
quit
221 2.0.0 gw.ampr.org closing connection
Connection closed by foreign host.
However, on 44.137.40.2:
telnet 44.0.0.1 25
Trying 44.0.0.1...
(and nothing more. probably an error when I wait a minute or so)
When running tshark on tunl0 I see the SYN+ACK packets from 44.0.0.1 to 44.137.40.2
and when I do an insert of a rule matching source address 44.0.0.1 before all
other firewall rules I see it matching, but still no ACK is going back.
I have no idea what is going on there. Ping works OK.
Rob
> Remember that the system routing table and the encap routing table are
> separate on amprgw. The system routing table is in the kernel; the
> encap table is in user space. Packets arriving at amprgw which are for
> hosts listed in the encap hosts table are diverted to the encap router;
> all other packets are left to the kernel to route.
Ok that should work fine.
> On the other hand, packets leaving (originating at) amprgw are first
> looked up in the system routing table and if a route is found (as it
> would be for your bgp-advertised subnet), they are sent out the system
> Ethernet interface and not diverted into the encap router. Packets with a
> destination not in the system routing table are tested to see if they're
> in the encap list, and if they are, they're sent to the encap router
> to be forwarded. The rest are dropped. The encap list currently has
> 19000 host entries in it.
Why do you have entries for the BGP-advertised subnets? Would those not
be adequately handled by the default route? Or is this only to handle
IPIP tunnels with a BGP-advertised net-44 tunnel endpoint address? Maybe
route entries for only those addresses would be an option.
(after all, they are known as they are listed in encap.txt)
The algorithm could then be to lookup 44-net addresses in the IPIP table
when there is no specific route for them in the routing table. When no
match is found, they could still be sent out as BGP-routed when desired,
this could be achieved with a "special" entry in the lookup array for
BGP-routed addresses that are not mentioned in the encap list.
> I have experimented with forcing 44.137.40.2 into the encap router with
> an explicit route to the loopback interface. This seems to work; the
> pings go out a tunnel to 89.18.172.156. I've left this route in place
> for now so you can repeat your tests and see if you do indeed get replies
> over the tunnel the way you expect. Please let me know.
Interesting: I can now ping 44.0.0.1 from that system, but a "telnet 44.0.0.1 25"
still yields nothing. I have not yet been able to find why that is, I see
the SYN ACK coming back on the tunnel and I have allowed all traffic from
44.0.0.1 in the firewall but it simply fails to establish. No ACK is going
back to you. I can connect to other IPIP endpoints just fine. Strange.
It is not really required to fix it, I can easily route the mail via a
different path, but I was interested in finding why this goes wrong and how
it may affect others / other applications.
> It could be argued that I should scrap FreeBSD on amprgw and start over
> with a Linux host. I don't know enough about the intricacies of Linux
> IP handling to be at all confident of being able to do that.
I can understand that! It is certainly possible to get it running, and it
would be a lot easier to get IPIP running, but it is always a big step
to do such things.
Rob
> Yes, BGP overrides encap on amprgw.
Is there a rationale behind that or is it just because of some implementation detail?
In our gateway, all routes are in the same table but at different distance (metric).
Direct/BGP routes have higher precedence than IPIP routes, when for the same subnet.
But, a more specific route (smaller subnet) always has precedence over a less specific one.
Rob
> Rob, it looks like the problem might be nearer to you...
> Interestingly, when I traceroute to 44.137.40.2 with 169.228.34.84 as
> the source address, I get:
> ...
That is normal, it will be routed over internet to our BGP subnet 44.137.0.0/16
which is via that gw-44-137 which then forwards it to 44.137.40.2 over IPIP.
However, 44.137.40.2 does not reply to pings from internet.
> When I repeat that traceroute with 44.0.0.1 as the source address,
> it gets nearly as far, but not quite:
> ...
That is strange. Don't you have a route to 44.137.40.2 in your encap table?
I would expect you to send it into an IPIP tunnel direct to that system.
Or is there some magic in amprgw that makes it ignore IPIP tunnels within BGP
routed subnets? That would cause this to fail...
> I can ping 213.222.29.194 from 169.228.34.84, but not from
> 44.0.0.1.
But you can ping 44.137.0.1 (the amprnet address of that same system) from 44.0.0.1 ?
> It looks, at first glance, like there's a 44 path problem in
> 213.222.29.194.
When you route back to 44.137.40.2 via 213.222.29.194 it is explained, because that
system has a stateful firewall that does not expect replies to traffic that has not
gone out via that same system.
Rob
> I thought those problems were fixed as well, or I would not have changed
> the submission address. I am not aware of any changes that would have
> affected the connectivity of 44.0.0.1. It should be working.
> More details might be helpful, such as the actual IP addresses involved,
> or perhaps traceroutes showing where the routing stops working.
Yes, I think it worked but likely that was on the previous gateway.
The system is 44.137.40.2 (89.18.172.156)
The route is:
44.0.0.1 via 169.228.34.84 dev tunl0 proto ampr-ripd onlink
A traceroute immediately fails. (only stars)
tshark dump of a "ping" (looks OK):
Internet Protocol Version 4, Src: 89.18.172.156 (89.18.172.156), Dst: 169.228.34.84 (169.228.34.84)
Version: 4
Header Length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
Total Length: 104
Identification: 0x949e (38046)
Flags: 0x02 (Don't Fragment)
0... .... = Reserved bit: Not set
.1.. .... = Don't fragment: Set
..0. .... = More fragments: Not set
Fragment offset: 0
Time to live: 64
Protocol: IPIP (4)
Header checksum: 0xd40c [validation disabled]
[Good: False]
[Bad: False]
Source: 89.18.172.156 (89.18.172.156)
Destination: 169.228.34.84 (169.228.34.84)
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
Internet Protocol Version 4, Src: 44.137.40.2 (44.137.40.2), Dst: 44.0.0.1 (44.0.0.1)
Version: 4
Header Length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
Total Length: 84
Identification: 0x8f7f (36735)
Flags: 0x02 (Don't Fragment)
0... .... = Reserved bit: Not set
.1.. .... = Don't fragment: Set
..0. .... = More fragments: Not set
Fragment offset: 0
Time to live: 64
Protocol: ICMP (1)
Header checksum: 0x2a9e [validation disabled]
[Good: False]
[Bad: False]
Source: 44.137.40.2 (44.137.40.2)
Destination: 44.0.0.1 (44.0.0.1)
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
Internet Control Message Protocol
Type: 8 (Echo (ping) request)
Code: 0
Checksum: 0xbba6 [correct]
Identifier (BE): 23938 (0x5d82)
Identifier (LE): 33373 (0x825d)
Sequence number (BE): 1 (0x0001)
Sequence number (LE): 256 (0x0100)
Timestamp from icmp data: Jul 12, 2017 20:30:58.200360000 CEST
[Timestamp from icmp data (relative): 0.000371000 seconds]
Data (48 bytes)
0000 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 ................
0010 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 ........ !"#$%&'
0020 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33 34 35 36 37 ()*+,-./01234567
Data: 08090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f...
[Length: 48]
This system has our own gateway 213.222.29.194 as its default tunnel route
but otherwise it is a standard ampr-ripd setup with 2 route tables.
Rob
Recently I saw a header in the DNS robot reply mails that the mail address has changes and the
update mails now have to be sent @gw.ampr.org. I changed that in my script, but then I noticed
that my system on the IPIP tunnel network cannot send mail there. Of course gw.ampr.org is 44.0.0.1
and there have been issues with reaching that from the IPIP gateway systems, but I thought those were
fixed.
I have changed my mail config to send the mail via my ISP smarthost and that works, but what is the
status on this? Is it supposed to be working, does it work for others?
It only affects my gateway on the IPIP tunnel network. From the BGP routed gateway and the radio-connected
systems here in the Netherlands there is no problem, and neither from normal internet addresses.
Rob
