Hi
I have acquired myself a problem. Recently I decided to move all web services away from the gateway to an ‘inner’ server. This is easily done by using port forwarding in the firewall on the gateway. Thus I forward all web services by using this line:
iptables -t nat -A PREROUTING -p tcp -m multiport --dport $Services -d $EXT_IP -j DNAT --to $webserver
where $Services is defined as
Services="smtp,domain,www,https,submission,imaps",
$EXT_IP is my external IP address and $webserver is the IP address of the ‘inner’ server.
I do the same for the ampr interface:
$iptables -t nat -A PREROUTING -p tcp -m multiport --dport $Services -i ampr0 -j DNAT --to $webserver
In this case it is only the www and https services that are relevant since I do not do mail on the 44-address.
All of this works of course as it should for ordinary IP addresses (non-44) and it also works fine when comming from a 44-address. Furthermore it works fine when I ping my 44-address from any IP address (which isn't surprising as ICMP is not forwarded). But it fails miserablywhen accessing my 44-address (44.145.40.3) from an ordinary IP address. The incomming request comes nicely in on the ampr interface, but the reply goes out the WAN interface to the Internet at largeand is thus not recognized as a reply on the originating host.
I have spent quite a bit of time trying to findout why this is so. I’m convinced it is something in my setup that is the cause of this behavior, but I’m at a loss as what it may be, so I hope some of you eagle-eyed people out there can spot the error.
Here are somedetails of my setup.
Routing rules:
$ip rule list 0: from all lookup local 44: from all to 44.0.0.0 /8 lookup ampr 45: from 44.145.40.3 lookup ampr 32766: from all lookup main 32767: from all lookup default
Main routing table:
$ip route list default via 86.48.99.33 dev enp0s7 metric 3 127.0.0.0/8 via 127.0.0.1 dev lo 192.168.19.0/24 dev enp0s6 proto kernel scope link src 192.168.19.6
‘ampr’ routing table (table 44) (excerpts):
$ip route list table ampr | head -n4 default via 169.228.34.84 dev ampr0 onlink 44.2.0.1 via 191.183.136.1 dev ampr0 proto 44 onlink window 840 44.2.2.0/24 via 216.218.207.198 dev ampr0 proto 44 onlink window 840 44.2.7.0/30 via 98.208.73.100 dev ampr0 proto 44 onlink window 840
My gateway is a Soekris Net5501 running Gentoo Linux updated to the latest versions last Monday.
I do hope I have expressed myself clearly enough and provided details enough so that one of you can point at something and say: ‘there is your error’. If some information is missing or lacking please ask for it and I’ll provide as much as I can.
Best 73 de Bent/OZ6BL
This is a normal system behavior.
For it to work, you need to mark incoming connections from internet addresses to the ampr tunnel interface, and then use this connection mark to add a routing mark on the replies, so they get routed back via your ampr table and to the ampr-gw.
I have no working example on a linux system to give you, but the hint is that it is done via iptables...
Marius, YO2LOJ
On 03.03.2019 22:13, Bent Bagger via 44Net wrote:
(...)
All of this works of course as it should for ordinary IP addresses (non-44) and it also works fine when comming from a 44-address. Furthermore it works fine when I ping my 44-address from any IP address (which isn't surprising as ICMP is not forwarded). But it fails miserablywhen accessing my 44-address (44.145.40.3) from an ordinary IP address. The incomming request comes nicely in on the ampr interface, but the reply goes out the WAN interface to the Internet at largeand is thus not recognized as a reply on the originating host.
You mean protocol forwarding. Ipencap is protocol 4. If your ampr gateway is behind a traditional NAT router, you need to find a way to forward protocol 4 to it.
An example is (where 192.168.1.10 is your ampr gateway):
iptables -t nat -A PREROUTING -p 4 -j DNAT --to 192.168.1.10
Depending on your router you might be able to this via some sort of script (firewall) input box or via the CLI. Short of that, you could always try pointing DMZ to your gateway.
The reply route for anything coming in via the internet must send the reply thru UCSD. So something like this must exist:
ip route add default dev tunl0 via 169.228.34.84 onlink table 44
And of course to get stuff off the internet to your gateway requires an ampr.org DNS entry.
On Sun, Mar 3, 2019 at 2:16 PM Bent Bagger via 44Net 44net@mailman.ampr.org wrote:
Hi
I have acquired myself a problem. Recently I decided to move all web services away from the gateway to an ‘inner’ server. This is easily done by using port forwarding in the firewall on the gateway. Thus I forward all web services by using this line:
iptables -t nat -A PREROUTING -p tcp -m multiport --dport $Services -d $EXT_IP -j DNAT --to $webserver
where $Services is defined as
Services="smtp,domain,www,https,submission,imaps",
$EXT_IP is my external IP address and $webserver is the IP address of the ‘inner’ server.
I do the same for the ampr interface:
$iptables -t nat -A PREROUTING -p tcp -m multiport --dport $Services -i ampr0 -j DNAT --to $webserver
In this case it is only the www and https services that are relevant since I do not do mail on the 44-address.
All of this works of course as it should for ordinary IP addresses (non-44) and it also works fine when comming from a 44-address. Furthermore it works fine when I ping my 44-address from any IP address (which isn't surprising as ICMP is not forwarded). But it fails miserablywhen accessing my 44-address (44.145.40.3) from an ordinary IP address. The incomming request comes nicely in on the ampr interface, but the reply goes out the WAN interface to the Internet at largeand is thus not recognized as a reply on the originating host.
I have spent quite a bit of time trying to findout why this is so. I’m convinced it is something in my setup that is the cause of this behavior, but I’m at a loss as what it may be, so I hope some of you eagle-eyed people out there can spot the error.
Here are somedetails of my setup.
Routing rules:
$ip rule list 0: from all lookup local 44: from all to 44.0.0.0 /8 lookup ampr 45: from 44.145.40.3 lookup ampr 32766: from all lookup main 32767: from all lookup default
Main routing table:
$ip route list default via 86.48.99.33 dev enp0s7 metric 3 127.0.0.0/8 via 127.0.0.1 dev lo 192.168.19.0/24 dev enp0s6 proto kernel scope link src 192.168.19.6
‘ampr’ routing table (table 44) (excerpts):
$ip route list table ampr | head -n4 default via 169.228.34.84 dev ampr0 onlink 44.2.0.1 via 191.183.136.1 dev ampr0 proto 44 onlink window 840 44.2.2.0/24 via 216.218.207.198 dev ampr0 proto 44 onlink window 840 44.2.7.0/30 via 98.208.73.100 dev ampr0 proto 44 onlink window 840
My gateway is a Soekris Net5501 running Gentoo Linux updated to the latest versions last Monday.
I do hope I have expressed myself clearly enough and provided details enough so that one of you can point at something and say: ‘there is your error’. If some information is missing or lacking please ask for it and I’ll provide as much as I can.
Best 73 de Bent/OZ6BL
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
No, I mean connection and routing marks. It has nothing to do with protocol forwarding.
On 03.03.2019 23:04, Steve L via 44Net wrote:
You mean protocol forwarding. Ipencap is protocol 4. If your ampr gateway is behind a traditional NAT router, you need to find a way to forward protocol 4 to it.
An example is (where 192.168.1.10 is your ampr gateway):
iptables -t nat -A PREROUTING -p 4 -j DNAT --to 192.168.1.10
Depending on your router you might be able to this via some sort of script (firewall) input box or via the CLI. Short of that, you could always try pointing DMZ to your gateway.
The reply route for anything coming in via the internet must send the reply thru UCSD. So something like this must exist:
ip route add default dev tunl0 via 169.228.34.84 onlink table 44
And of course to get stuff off the internet to your gateway requires an ampr.org DNS entry.
On Sun, Mar 3, 2019 at 2:16 PM Bent Bagger via 44Net 44net@mailman.ampr.org wrote:
Hi
I have acquired myself a problem. Recently I decided to move all web services away from the gateway to an ‘inner’ server. This is easily done by using port forwarding in the firewall on the gateway. Thus I forward all web services by using this line:
iptables -t nat -A PREROUTING -p tcp -m multiport --dport $Services -d $EXT_IP -j DNAT --to $webserver
where $Services is defined as
Services="smtp,domain,www,https,submission,imaps",
$EXT_IP is my external IP address and $webserver is the IP address of the ‘inner’ server.
I do the same for the ampr interface:
$iptables -t nat -A PREROUTING -p tcp -m multiport --dport $Services -i ampr0 -j DNAT --to $webserver
In this case it is only the www and https services that are relevant since I do not do mail on the 44-address.
All of this works of course as it should for ordinary IP addresses (non-44) and it also works fine when comming from a 44-address. Furthermore it works fine when I ping my 44-address from any IP address (which isn't surprising as ICMP is not forwarded). But it fails miserablywhen accessing my 44-address (44.145.40.3) from an ordinary IP address. The incomming request comes nicely in on the ampr interface, but the reply goes out the WAN interface to the Internet at largeand is thus not recognized as a reply on the originating host.
I have spent quite a bit of time trying to findout why this is so. I’m convinced it is something in my setup that is the cause of this behavior, but I’m at a loss as what it may be, so I hope some of you eagle-eyed people out there can spot the error.
Here are somedetails of my setup.
Routing rules:
$ip rule list 0: from all lookup local 44: from all to 44.0.0.0 /8 lookup ampr 45: from 44.145.40.3 lookup ampr 32766: from all lookup main 32767: from all lookup default
Main routing table:
$ip route list default via 86.48.99.33 dev enp0s7 metric 3 127.0.0.0/8 via 127.0.0.1 dev lo 192.168.19.0/24 dev enp0s6 proto kernel scope link src 192.168.19.6
‘ampr’ routing table (table 44) (excerpts):
$ip route list table ampr | head -n4 default via 169.228.34.84 dev ampr0 onlink 44.2.0.1 via 191.183.136.1 dev ampr0 proto 44 onlink window 840 44.2.2.0/24 via 216.218.207.198 dev ampr0 proto 44 onlink window 840 44.2.7.0/30 via 98.208.73.100 dev ampr0 proto 44 onlink window 840
My gateway is a Soekris Net5501 running Gentoo Linux updated to the latest versions last Monday.
I do hope I have expressed myself clearly enough and provided details enough so that one of you can point at something and say: ‘there is your error’. If some information is missing or lacking please ask for it and I’ll provide as much as I can.
Best 73 de Bent/OZ6BL
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
To be more explicit, let's get a system with a tunnel, a WAN and a 44net LAN. Public Router IP is 1.2.3.4
Router AMPR is 44.128.0.1
LAN is 44.128.0.0/24, web server on 44.128.0.2
Now the routes are:
44.128.0.0/24 on LAN
default is via WAN
table 44 has default via amprgw
44.128.1.0/24 is via 2.3.4.5
Rules are:
Any to 44.0.0.0 via table 44
Now the use cases:
1. Connection from Internet comes via WAN, gets port forward to web server, replies return via default, because it is not 44net. OK.
2. Connection comes from 44.128.1.2 via tunnel, port forward, reply follows rule and goes via table 44 and back to sender. OK
3. Connection comes from internet to 44net address. Gets port forward. But the source address is a public internet address, so the reply does not follow the rule, goes back via table main and to the WAN.
Which is the failure mode described by Bent.
To go around it, we need connection tracking, incoming connections get a connection mark in the prerouting chain, get dst-nated to the server.
The replies, due to conntrack will get the same connection mark. Now we need to additional steps:
- replies with connection mark will get a routing mark.
- replies with routing mark will follow an additional rule like: marked goes via table 44
This 2 step approach is needed because it is not possible to create rules based on connection marks, only on routing marks.
I hope this clarifies things a little...
Marius, YO2LOJ
On 04.03.2019 03:25, Marius Petrescu wrote:
No, I mean connection and routing marks. It has nothing to do with protocol forwarding.
On 03.03.2019 23:04, Steve L via 44Net wrote:
You mean protocol forwarding. Ipencap is protocol 4. If your ampr gateway is behind a traditional NAT router, you need to find a way to forward protocol 4 to it. ps://mailman.ampr.org/mailman/listinfo/44net
Bent, try something like this:
|# this tells the system to use table 44 for packets marked with 1 ip rule add fwmark 1 table 44 # this tells the system to apply a connection mark of 1 to incoming connections on tunl0 that are NOT in the 44/8 address space |iptables -t mangle -A PREROUTING -i tunl0 ! -s 44.0.0.0/8 -j CONNMARK --set-mark 1
# this tells to copy the connection mark (if any) to the packet mark so it will follow the rule above iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
Marius, YO2LOJ
Sorry, but mails got rearranged my the mailer....
ip rule add fwmark 1 table 44
iptables -t mangle -A PREROUTING -i tunl0 ! -s 44.0.0.0/8 -j CONNMARK --set-mark 1
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
On 04.03.2019 04:25, Marius Petrescu wrote:
Bent, try something like this:
|# this tells the system to use table 44 for packets marked with 1 ip rule add fwmark 1 table 44 # this tells the system to apply a connection mark of 1 to incoming connections on tunl0 that are NOT in the 44/8 address space |iptables -t mangle -A PREROUTING -i tunl0 ! -s 44.0.0.0/8 -j CONNMARK --set-mark 1
# this tells to copy the connection mark (if any) to the packet mark so it will follow the rule above iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
Marius, YO2LOJ
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Hi folks
Thanks for the answers. I have had a busy day, so I'll not be able to do anything until tomorrow.
Marius, you must be a night owl - unless of course you are in a completely different time zone.
Best 73,
Bent/OZ6BL
On 04/03/2019 03.29, Marius Petrescu wrote:
Sorry, but mails got rearranged my the mailer....
ip rule add fwmark 1 table 44
iptables -t mangle -A PREROUTING -i tunl0 ! -s 44.0.0.0/8 -j CONNMARK --set-mark 1
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
On 04.03.2019 04:25, Marius Petrescu wrote:
Bent, try something like this:
|# this tells the system to use table 44 for packets marked with 1 ip rule add fwmark 1 table 44 # this tells the system to apply a connection mark of 1 to incoming connections on tunl0 that are NOT in the 44/8 address space |iptables -t mangle -A PREROUTING -i tunl0 ! -s 44.0.0.0/8 -j CONNMARK --set-mark 1
# this tells to copy the connection mark (if any) to the packet mark so it will follow the rule above iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
Marius, YO2LOJ
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Hi
Success!
Well, not in the first place. I executed the three commands Marius suggested, and they certainly did have an effect! The forwarded packets that were supposed to go to my 'inner' server were also routed back to the AMPR GW, which of course did not know anything about my local addresses (192.168...).
However, after adding this line:
ip route add 192.168.19.0/24 dev enp0s6 table 44
everything felt in place and I'm now a happy man.
Thanks again for all the help.
Best 73 de Bent/OZ6BL
On 04/03/2019 03.29, Marius Petrescu wrote:
Sorry, but mails got rearranged my the mailer....
ip rule add fwmark 1 table 44
iptables -t mangle -A PREROUTING -i tunl0 ! -s 44.0.0.0/8 -j CONNMARK --set-mark 1
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
On 04.03.2019 04:25, Marius Petrescu wrote:
Bent, try something like this:
|# this tells the system to use table 44 for packets marked with 1 ip rule add fwmark 1 table 44 # this tells the system to apply a connection mark of 1 to incoming connections on tunl0 that are NOT in the 44/8 address space |iptables -t mangle -A PREROUTING -i tunl0 ! -s 44.0.0.0/8 -j CONNMARK --set-mark 1
# this tells to copy the connection mark (if any) to the packet mark so it will follow the rule above iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
Marius, YO2LOJ
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
-
Bent Bagger -
Marius Petrescu -
Steve L