> Please keep in mind though, the malicious traffic I observed did not originally come from AMPRGW. I originally observed the nested IPENCAP traffic from a Polish Public IP that's still currently registered as an AMPRNet gateway.
As you know in later Linux kernels it became more difficult to see the
outer header of the IPIP packet in a firewall rule handling the tunneled
traffic.
To circumvent that, a couple of years ago I added a rule to the firewall
that sets a packet mark on traffic received from AMPRGW (matching the
source IP in the outer header).
This packet mark can then be checked in the firewall for the tunneled
traffic. Source addresses outside AMPRnet are only accepted when the
packet mark is set.
Unfortunately this breaks legitimate traffic because some gateways are
incorrectly configured (as I mentioned before) and send tunneled traffic
with their own external address as source address, instead of the
AMPRnet address assigned to the gateway.
So such traffic is accepted as well here (i.e. traffic with a source
address of one of the gateways)
We really should abandon the IPIP tunnel mesh and move on to something a
bit more secure (and easier to use on modern equipment)...
Rob
David,
1. OpenWrt is Linux; and my previous gateway ran Ubuntu Linux (no *nos) - I cannot speak to other Routing Planes running IPENCAP tunnels. Also note machines running tnos have the machine's kernel and userland software to possibly interact with a packet. I only surmise without firewalls, some of the basic routing concepts apply to other Network Operating Systems too.
2. We're running AMPR gateways, so routing would be ON - I'm lost at the remark about IPv4 Forwarding = OFF. It seems as if you're suggesting to "disable routing on the router". I can only speak for Linux-based devices that are also performing functions as an actual router. Servers (i.e. with net.ipv4.ip_forward=0 and net.ipv6.ip_forward=0) merely receiving all of your IPENCAP packets should not apply here. Even when I originally ran a Linux server as my APMRNet device - downstream of my border router, I still had a /24 (253 other usable IPs) - so I've always had the need to route some IPs.
You asked: Are these users trying to break into our AMPR hosts due to gaps in our firewalls or are they exploiting incomplete firewalls to use them as relay hosts to SOURCE attacks out on the Internet?
Answer: Both are logically possible with an unfirewalled router or device and/or policy route isolation of the interfaces (in the case of a router). I described a scenario where a malicious actor can use the asymmetry of the routing to get a login screen (e.g. of your router or server) only accessible locally. It also seems as if DDoS is a consequence, or the modus operandi, of some traffic (e.g. TCP reflections cause 100% open connections at the router or a machine).
2b. These would be firewall rules to mitigate.
3. I have provided: tcpdump outputs and netflow records, the tcpdump command to observe the nested IPENCAP and a netflow log of a packet which (had it not been firewalled) could have double routed across my device - let me know what additional filters are needed. Also, the device running the dump must be your registered gateway (to which these actors are directing the traffic in this discussion), not a random device on the Internet. https://linux.die.net/man/8/tcpdump
Nested IPENCAP:
tcpdump -vvv -n -i tunl0 proto 4
I haven't thought of a tcpdump to observe all Header 1 routing - as the vice versa SRC/DST could be valid in some instances - this command only shows traffic entering or exiting tunl0 without your IP...it will also hit FALSELY on the RIP44 packets, hence the last part:
tcpdump -vvv -n -i tunl0 src net not 44.60.44.0/24 and dst net not 44.60.44.0/24 and not host 224.0.0.9
4. I have posted the Second header scenario and it's remains available at the Wiki page. There should be no 3rd header scenario - that's the malicious activity the Second Header firewall rule blocks.
http://wiki.ampr.org/wiki/Firewalls (and see previous emails on this topic) - FYI, there's nothing 44net-specific to the rules I suggested -
# THIS PREVENTS NESTED IPENCAP (BCP 38)
iptables -t raw -I PREROUTING -p 4 -i tunl0 -j DROP
HTTP Rule (OpenWrt syntax - iptables-based):
config rule
option target 'ACCEPT'
option proto 'tcp'
option name 'AMPR_WWW'
option family 'ipv4'
option dest_ip '44.60.44.10'
option dest_port '80'
option src '*'
option dest 'amprnet'
option extra '-m limit --limit 25/minute --limit-burst 100 -j ACCEPT'
Block first TCP packet not SYN (again, OpenWrt syntax - iptables-based):
config rule
option proto 'tcp'
option name 'Block_In_Not_SYN'
option src '*'
option target 'DROP'
option extra '! --syn -m conntrack --ctstate NEW'
(Also the normal SYN Flood and INVALID rules).
5. Yes you can have DNS records and not receive traffic from AMPRGW - just block it and/or remove its route. Following the Wiki (for an Linux or OpenWrt setup), the only allowed traffic from anywhere by default are routes from AMPRGW, so long as the machine's default iptables rule is drop or reject. So it's alarming to hear inquiries like this. This will not stop the actual IPENCAPed packets from the Internet via AMPGW for the IPs in the DNS zone.
Please keep in mind though, the malicious traffic I observed did not originally come from AMPRGW. I originally observed the nested IPENCAP traffic from a Polish Public IP that's still currently registered as an AMPRNet gateway.
73,
- Lynwood
KB3VWG
All,
I wanted to make 2 notes regarding: dynamic IP addresses and AMPR TCP-based services.
* An operator mentioned that it may perhaps be malicious dynamic IPs (no longer used by the operators in question) that are sending this traffic...I also think the [malicious] experience "via RIP[44]" had not occurred was noted...
Since my gateway is in fact dynamic and still seeing issues...wouldn't it be prudent for those operators that are monitoring AND FIREWALLING their gateways to identify the rogue gateways for removal and dropping?
Because, if they're rogue operators...**we now understand how our routes are being leaked**...as they are successfully relocating my new Public IPs (even though I'm dropping routing traffic and IPs out of tunl0 that != my allocation).
* All operators running TCP services should ensure they are not reflecting SYN-ACK packets and/or experiencing half-open connections on the router and server as a result. Firewalling new TCP connections CONNECTION_STATE not SYN (unless you expect such traffic) and burst rules help. In OpenWrt, such activity can be easily seen on the Overview Page upon login at "Active Connections" graphic.
** I suggest as an extreme measure for those who have not blocked IPENCAP on tunl0 only to operators in the past - to do so; and/or renumber their router interface(s) possessing AMPR IPs (the IP does not need a DNS record if it doesn't need access from AMPRGW/Public Internet). This mitigates the malicious actors' ability to perform nested routing on your device.
** It is now apparent that 2nd and 3rd headers need to be properly firewalled on all gateways running IPENCAP. Be vigilant.
** Please make sure no internal LAN devices were compromised (thru asymmetric traffic that may have allowed a malicious person access to LAN machines).
** Nested IPIP can be firewalled (i.e. Header 2 - the third header); but there is no known solution to receiving a packet with a malicious Header 1 (i.e the second header - which could be expected to reach any device on Earth with a Public IP). Properly firewalling and monitoring traffic that you allow into your 44network (with 44net IPs) is important.
** If anyone running a Linux-based router can WireShark/tcpdump and observe MAC addresses directly on their tunl0, please let me know. This is believed a hash of the 4 SRC/DST IPs of Headers 0 and 1 - which could be used to authenticate traffic from gateways that != AMPRGW.
19:39 UTC 23MAR2020:
269 11.96 KB zone_amprwan_dest_DROP all * * 0.0.0.0/0 0.0.0.0/0 - AMPR_DropLoop
844 76.12 KB zone_amprwan_dest_DROP all * * !44.60.44.0/24 0.0.0.0/0 - Drop-AMPR_OUT_InvalidSRC
This is ~30 seconds of traffic of drops from my router to forwarding of invalid Header 1 receipts. The first rule is all traffic from you in which KB3VWG != DST IP; and hence you asked me to route it for you. (I only allow this by coordination). It should be 0. I should note the second rule includes drops from my AMPRLAN that != bogons - but should otherwise be 0 too (e.g. unfirewalled, the packets could forward to BGP 44 IPs that exist on an operator's routing table as "properly NATed traffic" from an allowed gateway).
73,
- Lynwood
KB3VWG
Can someone please add some explanatory text to the relevant portal and wiki
pages that inform the visitor that AMPRNet is a closed network that is
only accessible
to licensed radio amateurs.
I got another of those requests that come from people that are just
after some free
IP space and submit an allocation request through the portal using some
"handle" like
used on whois as their "callsign". They probably find us using some
search and then
land on portal.ampr.org, make an account and file an allocation
request. When walking
that route, there is no information at all about the target audience,
all information is
just geared towards "how to request IP space", which they do.
In the latest case, when I rejected the request saying it is only for
licensed amateurs
the request was re-submitted with a callsign which is not assigned to
the person making
the request. He probable searched again to find what licensed radio
amateurs are and
found some callsign that he then added to the request.
(not knowing that we can do reverse lookups on those)
They should at least at some point encounter some text that explains
them that this
network is not for their hosting purposes.
Rob
All,
The following are RAW firewall hits indicating nested IPIP in IPIP packets.
I run a firewall that only allows IPIP from all of you (and rules that only allows the AMPRGW routes and my destination IPs); but since this is a RAW rule - it implies nothing of any operator.
I have not reviewed my Netflow records; but please be vigilant of this traffic. I warned of this issue in the "ancient" 44 mailing archives.
Table: RawChain PREROUTING (Policy: ACCEPT, 94829408 Packets, 59.38 GB Traffic)Pkts. Traffic Target Prot. In Out Source Destination Options Comment32 2.37 KB DROP 4 tunl0 * 0.0.0.0/0 0.0.0.0/0 - -
73 ::and elbow bump::,
- Lynwood
KB3VWG
> * Some of you query the NTP with your Public IP thru AMPRGW instead of directly to me thru the mesh (just a note to make your SRC IP an AMPR address, not your Public IP - I may disable your access thru AMPGW in the future, as it's announced as "AMPR-only")
Unfortunately it is quite common for gateway stations to send tunneled traffic towards 44net addresses (via IPIP) with a public IP as the source.
I normally block all such traffic, except when the public IP is the gateway public address (as I got tired of trying to reach sysops where this error was present).
People *should really use* proper source address selection and policy routing and NOT send tunnel traffic with other than 44net traffic (both source and destination) inside it to any gateway station except by prior agreement.
(e.g. AMPRGW can send traffic from a 44net address to public destination addresses, and so can some gateways)
To make life easier, DO NOT TRY to setup a gateway on the same system where your applications are also running, unless you have good knowledge of networking configuration and know about such concepts as policy routing (ip route rule, multiple route tables) and setting a preferred source address in a route.
When you use a separate router and application machine, such errors are much less likely to occur, and configuring the firewall is also much easier.
Get a separate Pi or MikroTik or whatever to run your gateway, and then have a PC or another system to run your BBS or conference or whatever you want to run on AMPRnet!
Rob
> Those with a dynamic address CAN participate as their public gateway
> can now be a FQDN for their dynamic service. I have some within the
> New York State subnet (44.68/16).
The issue is that IPIP tunnels have to be validate with their external
address
for at least SOME security, and this means that when the address changes
there
is nothing else we can do than drop their packets until the change comes
through
the portal and RIP system.
With a system that has those dynamic addresses connect only to one or two
VPN routers in a secure manner (e.g. L2TP/IPsec) we would not have that
problem.
Also, those address changes would not be important to other systems on
the network.
Out of the 561 registered gateways, we only ever receive traffic only
from 73 of them.
(the others could be either inactive or not be sending traffic to the
Netherlands)
Their address changes would not be important to us.
Rob
For a long time now, I have been allowing IPIP only from registered
gateways and
disallowed nested IPIP. Indeed I have seen in the past that IPIP
packets were sent with
the intention of being forwarded through "allow trusted subnets" rules
and then maybe
back out to internet hosts that were targeted for DDoS or similar.
When looking in the logs of those rules, I usually see dropped packets
from hosts that
are apparently on dynamic addresses and have changed address, but this
change has
not yet reached me through the ampr-rip announcements.
However, there are indeed also instances of apparently unrelated
intrusion attempts.
It remains my position that we should change from this IPIP mesh to a
more modern
VPN system where stations with dynamic addresses can participate through
a local
VPN server that participates in a network that uses standard protocols
to form a
dedicated AMPRnet tunnel network with automatic routing, that can be
used by standard
equipment and can be made more resilient against unwanted use (e.g. by
using GRE/IPsec
and L2TP/IPsec tunnels instead of the traditional IPIP).
However, I no longer want to beat a dead horse. We had the discussion a
while ago but
unfortunately it was then redirected onto a separate mailing list.
Rob
