Rob,
You stated:
So all traffic received on IPIP tunnels should be from net44 only in our case. Unfortunately not all of it is.
Can you elaborate on the traffic that isn't, please?
Is this traffic from another operator...or a non-operator?
Can you also elaborate if this traffic forwards in any cases?
That's what we're tying to stop. Please note, I haven't identified this is related to any IPENCAP issue specifically (except that it appears we may have some operators that forward traffic not destined for them). While I understand your concern, I'm not sure it's related to IPENCAP 100%.
73,
- Lynwood
KB3VWG
> I'm now wondering how such a config is [incorrectly] made (i.e. the
inside Header has the incorrect SRC)....likely because of no route
policy...another discussion...
Easy: when you take a default Linux system and add something like IPIP
mesh with routes in the same table, and then you run services on the
same system, an outgoing connect to a system within net44 will just
consult the routing table, find an outgoing route and make a connection.
You then have to rely on the "source address selection" done by the
system, which may select your public IP as the source address.
This may also be configured in the service itself (when the socket is
not bound to 0.0.0.0 but to some specified address).
The outgoing connect will now be routed through the IPIP tunnel, but it
will have the public address as the source.
To prevent this, the service would have to be bound to the net44
address, or it would have to be set as a default source address in the
tunnel routes in the table.
When you run a separate system as the IPIP router and an AMPRnet
services host, you do not run into this problem because the services
host has the proper external address within net44 and the router will
not change it.
But with both combined in a single host, you can still get it working
correctly when you pay some attention. Which of course has to be done
when you want a single system that can both be a general-purpose
internet browsing system (directly via your ISP connection) and can be
an AMPRnet services host at the same time (also for services available
from public internet addresses). The routing has to be carefully set up
when doing this, and setting a preferred source address is only part of
that.
In our network the problem you mention w.r.t. AMPRGW does not occur
because internet traffic is routed directly to our gateway, not via an
IPIP tunnel.
The IPIP tunnel via AMPRGW only gets public internet traffic when our
BGP announcement is down for some reason, that is why I kept it
operational but it normally has zero traffic.
So all traffic received on IPIP tunnels should be from net44 only in our
case. Unfortunately not all of it is.
When I "just drop" the bad traffic it appears in a log and it appears
the originators of the traffic do not notice it, so it goes on and on.
As I mentioned, I sent mail to gateway owners about it, but it rarely
fixes the situation.
Rob
> One minor thing to bring up though, since I went and re-read the text. It says
> "ISPs don't configure their routers with publicly routable IP space for end users, why would you?"
> This is by and large false. Some do indeed use 1918 space for customer facing interfaces, but most do not, as this practice this can break PMTUD due to dropping PTB messages sent by 1918 numbered interfaces and is not generally not recommended.
It looks like this came from the HamWan people who apparently use a lot of RFC1918 addressing, I think at first even their whole network was on RFC1918 space and they introduced net44 capability only later.
However, in our network here we exclusively use net44 addresses, also for links and routers. Of course we do have appropriate firewalls in place.
The first line of defense is that at the internet router, all incoming traffic is blocked by default unless the destination is from an address list of only systems that provide services that are to be visible from internet.
And in the last router before the actual system there is an additional firewall that opens only certain ports for everyone, and restricts management ports to another address list that has only addresses of known operators, or at most the country subnet.
This makes the result similar to using RFC1918 addressing (which provides protection because it can only be routed locally), but without the disadvantages that you mention.
Anyway, I am not that much concerned about that part of the text, it will depend on local policies and we do not use that method of subnet allocation anyway.
My main concern at this time is that "outsiders" (not licensed radio amateurs) apparently find our network in their search for IPv4 address space, and make requests that we have to reject.
This rejection often leads to discussion (partly because there is no terminate-this-request option in the portal, a coordinator can only accept it or send it back to the requester for more detail).
Therefore I think it would be best when those outsiders immediately see that this system is not for them.
Rob
Rob,
I now understand what you meant by this. I'm not seeing [lots of] improper Header 1 traffic with a known gateway IP as a SRC IP (and if I do, I want to block as it's asymmetric, as I tax AMPRGW on the reply trip). I receive the packet from AMPRGW (i.e. they're using the Public Internet instead of AMPRNet to reach the NTP server).
* AMPR <> AMPR (OK, This allowed - as I list this service to my subnet in the Wiki)
* Registered_Public <> Registered_Public (OK; but closed for MOST services)
* Registered_Public <> AMPR (OK, not announced, taxes AMPRGW - may close soon; closed for some services)
* Tunnel (Header 0) <> Public IP (Header 1) (NO - I consider this a crafted packet or improperly configured gateway, creates asymmetry; and taxes AMPRGW on the reply) - but cannot be mitigate except by blocking registered gateways on the de-encapsulated side of tunl0
I would advise you block that traffic with an improper Header 1 from registered gateways of you flag it via Header 0.
But this discussion now causes me to recall discussions we've had before on:
1.) the security of seeing the Registered Public IP on both sides of the tunnel
2.) policy-based routing and the need to separate routing tables comes to mind....this comes from operators' failure to set that up.
I'm now wondering how such a config is [incorrectly] made (i.e. the inside Header has the incorrect SRC)....likely because of no route policy...another discussion...
Just note that bad IPENCAP is not the majority of bad traffic we're seeing on some nodes...in fact, bad IPENCAP is the least of this traffic (at least now).
- KB3VWG
Unfortunately this breaks legitimate traffic because some gateways are
incorrectly configured (as I mentioned before) and send tunneled traffic
with their own external address as source address, instead of the
AMPRnet address assigned to the gateway.
So such traffic is accepted as well here (i.e. traffic with a source
address of one of the gateways)
Are there any updated dns procedures with regards to coordinators? I have a new coordination and haven’t done it in a while and can no longer find the email with the procedure for adding dns. At one point I thought this was in the portal.
Best Regards
Elias
Kd5jfe
Louisiana AMPR coordinator
Sent from my iPhone
> The relevant page has been updated with your suggested dialogue
Thanks! It is at least an attempt, let's hope it works.
Other coordinators beware: there is a hosting company owner from the
Netherlands
that is apparently shopping for a subnet. After I rejected him, he
tried in Germany.
Rob
> Please keep in mind though, the malicious traffic I observed did not originally come from AMPRGW. I originally observed the nested IPENCAP traffic from a Polish Public IP that's still currently registered as an AMPRNet gateway.
As you know in later Linux kernels it became more difficult to see the
outer header of the IPIP packet in a firewall rule handling the tunneled
traffic.
To circumvent that, a couple of years ago I added a rule to the firewall
that sets a packet mark on traffic received from AMPRGW (matching the
source IP in the outer header).
This packet mark can then be checked in the firewall for the tunneled
traffic. Source addresses outside AMPRnet are only accepted when the
packet mark is set.
Unfortunately this breaks legitimate traffic because some gateways are
incorrectly configured (as I mentioned before) and send tunneled traffic
with their own external address as source address, instead of the
AMPRnet address assigned to the gateway.
So such traffic is accepted as well here (i.e. traffic with a source
address of one of the gateways)
We really should abandon the IPIP tunnel mesh and move on to something a
bit more secure (and easier to use on modern equipment)...
Rob
David,
1. OpenWrt is Linux; and my previous gateway ran Ubuntu Linux (no *nos) - I cannot speak to other Routing Planes running IPENCAP tunnels. Also note machines running tnos have the machine's kernel and userland software to possibly interact with a packet. I only surmise without firewalls, some of the basic routing concepts apply to other Network Operating Systems too.
2. We're running AMPR gateways, so routing would be ON - I'm lost at the remark about IPv4 Forwarding = OFF. It seems as if you're suggesting to "disable routing on the router". I can only speak for Linux-based devices that are also performing functions as an actual router. Servers (i.e. with net.ipv4.ip_forward=0 and net.ipv6.ip_forward=0) merely receiving all of your IPENCAP packets should not apply here. Even when I originally ran a Linux server as my APMRNet device - downstream of my border router, I still had a /24 (253 other usable IPs) - so I've always had the need to route some IPs.
You asked: Are these users trying to break into our AMPR hosts due to gaps in our firewalls or are they exploiting incomplete firewalls to use them as relay hosts to SOURCE attacks out on the Internet?
Answer: Both are logically possible with an unfirewalled router or device and/or policy route isolation of the interfaces (in the case of a router). I described a scenario where a malicious actor can use the asymmetry of the routing to get a login screen (e.g. of your router or server) only accessible locally. It also seems as if DDoS is a consequence, or the modus operandi, of some traffic (e.g. TCP reflections cause 100% open connections at the router or a machine).
2b. These would be firewall rules to mitigate.
3. I have provided: tcpdump outputs and netflow records, the tcpdump command to observe the nested IPENCAP and a netflow log of a packet which (had it not been firewalled) could have double routed across my device - let me know what additional filters are needed. Also, the device running the dump must be your registered gateway (to which these actors are directing the traffic in this discussion), not a random device on the Internet. https://linux.die.net/man/8/tcpdump
Nested IPENCAP:
tcpdump -vvv -n -i tunl0 proto 4
I haven't thought of a tcpdump to observe all Header 1 routing - as the vice versa SRC/DST could be valid in some instances - this command only shows traffic entering or exiting tunl0 without your IP...it will also hit FALSELY on the RIP44 packets, hence the last part:
tcpdump -vvv -n -i tunl0 src net not 44.60.44.0/24 and dst net not 44.60.44.0/24 and not host 224.0.0.9
4. I have posted the Second header scenario and it's remains available at the Wiki page. There should be no 3rd header scenario - that's the malicious activity the Second Header firewall rule blocks.
http://wiki.ampr.org/wiki/Firewalls (and see previous emails on this topic) - FYI, there's nothing 44net-specific to the rules I suggested -
# THIS PREVENTS NESTED IPENCAP (BCP 38)
iptables -t raw -I PREROUTING -p 4 -i tunl0 -j DROP
HTTP Rule (OpenWrt syntax - iptables-based):
config rule
option target 'ACCEPT'
option proto 'tcp'
option name 'AMPR_WWW'
option family 'ipv4'
option dest_ip '44.60.44.10'
option dest_port '80'
option src '*'
option dest 'amprnet'
option extra '-m limit --limit 25/minute --limit-burst 100 -j ACCEPT'
Block first TCP packet not SYN (again, OpenWrt syntax - iptables-based):
config rule
option proto 'tcp'
option name 'Block_In_Not_SYN'
option src '*'
option target 'DROP'
option extra '! --syn -m conntrack --ctstate NEW'
(Also the normal SYN Flood and INVALID rules).
5. Yes you can have DNS records and not receive traffic from AMPRGW - just block it and/or remove its route. Following the Wiki (for an Linux or OpenWrt setup), the only allowed traffic from anywhere by default are routes from AMPRGW, so long as the machine's default iptables rule is drop or reject. So it's alarming to hear inquiries like this. This will not stop the actual IPENCAPed packets from the Internet via AMPGW for the IPs in the DNS zone.
Please keep in mind though, the malicious traffic I observed did not originally come from AMPRGW. I originally observed the nested IPENCAP traffic from a Polish Public IP that's still currently registered as an AMPRNet gateway.
73,
- Lynwood
KB3VWG
All,
I wanted to make 2 notes regarding: dynamic IP addresses and AMPR TCP-based services.
* An operator mentioned that it may perhaps be malicious dynamic IPs (no longer used by the operators in question) that are sending this traffic...I also think the [malicious] experience "via RIP[44]" had not occurred was noted...
Since my gateway is in fact dynamic and still seeing issues...wouldn't it be prudent for those operators that are monitoring AND FIREWALLING their gateways to identify the rogue gateways for removal and dropping?
Because, if they're rogue operators...**we now understand how our routes are being leaked**...as they are successfully relocating my new Public IPs (even though I'm dropping routing traffic and IPs out of tunl0 that != my allocation).
* All operators running TCP services should ensure they are not reflecting SYN-ACK packets and/or experiencing half-open connections on the router and server as a result. Firewalling new TCP connections CONNECTION_STATE not SYN (unless you expect such traffic) and burst rules help. In OpenWrt, such activity can be easily seen on the Overview Page upon login at "Active Connections" graphic.
** I suggest as an extreme measure for those who have not blocked IPENCAP on tunl0 only to operators in the past - to do so; and/or renumber their router interface(s) possessing AMPR IPs (the IP does not need a DNS record if it doesn't need access from AMPRGW/Public Internet). This mitigates the malicious actors' ability to perform nested routing on your device.
** It is now apparent that 2nd and 3rd headers need to be properly firewalled on all gateways running IPENCAP. Be vigilant.
** Please make sure no internal LAN devices were compromised (thru asymmetric traffic that may have allowed a malicious person access to LAN machines).
** Nested IPIP can be firewalled (i.e. Header 2 - the third header); but there is no known solution to receiving a packet with a malicious Header 1 (i.e the second header - which could be expected to reach any device on Earth with a Public IP). Properly firewalling and monitoring traffic that you allow into your 44network (with 44net IPs) is important.
** If anyone running a Linux-based router can WireShark/tcpdump and observe MAC addresses directly on their tunl0, please let me know. This is believed a hash of the 4 SRC/DST IPs of Headers 0 and 1 - which could be used to authenticate traffic from gateways that != AMPRGW.
19:39 UTC 23MAR2020:
269 11.96 KB zone_amprwan_dest_DROP all * * 0.0.0.0/0 0.0.0.0/0 - AMPR_DropLoop
844 76.12 KB zone_amprwan_dest_DROP all * * !44.60.44.0/24 0.0.0.0/0 - Drop-AMPR_OUT_InvalidSRC
This is ~30 seconds of traffic of drops from my router to forwarding of invalid Header 1 receipts. The first rule is all traffic from you in which KB3VWG != DST IP; and hence you asked me to route it for you. (I only allow this by coordination). It should be 0. I should note the second rule includes drops from my AMPRLAN that != bogons - but should otherwise be 0 too (e.g. unfirewalled, the packets could forward to BGP 44 IPs that exist on an operator's routing table as "properly NATed traffic" from an allowed gateway).
73,
- Lynwood
KB3VWG
