I'm running into an interesting problem I thought maybe someone in this community would have run into before.
We're working on a project to redesign and re-IP our entire radio network. Part of the goal is to stop using OpenVPN and IPSec tunnels (long story) and move exclusively to GRE-based tunnels. The plan is to have two Linux VPS hosts running at a provider with our 44Net allocation and an IPv6 allocation advertised and then routed into the radio network over GRE tunnels to 3 different locations that are all backhauled together. One of the primary goals is to do dual-stack throughout the network.
I have one site setup working perfectly - "gre0". It's passing IPv4 and IPv6 traffic over the GRE tunnel. One *key* point to this is that the GRE connection endpoints are iPv6. IPv4 isn't doable for this connection (again, long story but not an option). When setting up the second GRE tunnel "gre2", nothing would work even though the configuration was the same EXCEPT for the fact that the second GRE tunnel was using IPv4 addresses for the GRE tunnel endpoints. Linux keeps spitting out a very odd error when I try to ping across the tunnel I cannot find reasonably documented anywhere:
ip6_tunnel: gre2 xmit: Local address not yet configured!
The key was the "ip6_tunnel" part that took me awhile to figure out. After experimenting, I've found that if I have one GRE tunnel using IPv6 endpoints (ip -6 tunnel add gre0 mod ip6gre) and one using IPv4 endpoints (ip tunnel add gre2 mode gre), only the gre0 tunnel will work and the gre2 tunnel seems to believe it's missing an IPv6 address. If I delete the gre0 tunnel, the gre2 tunnel immediately beings working with both IPv4 and IPv6 traversing the GRE. I cannot find any documentation that describes this behavior or why it would be the case.
I can't switch to IPIP tunnels because I haven't found a way to do a dual-stack tunnel between the endpoints - seems like you can only do one IPIP tunnel between two endpoint IPs regardless of type (ipip, ipip6, ip6ip6, sit)
Anyone have any deep wisdom on GRE tunnels?
Jason
I'm no gre expert. I have only used it over v4. But I am willing to look at code.
What kernel version are you using?
On Sat, Nov 16, 2019, 9:01 PM Jason McCormick via 44Net < 44net@mailman.ampr.org> wrote:
I'm running into an interesting problem I thought maybe someone in this community would have run into before.
We're working on a project to redesign and re-IP our entire radio network. Part of the goal is to stop using OpenVPN and IPSec tunnels (long story) and move exclusively to GRE-based tunnels. The plan is to have two Linux VPS hosts running at a provider with our 44Net allocation and an IPv6 allocation advertised and then routed into the radio network over GRE tunnels to 3 different locations that are all backhauled together. One of the primary goals is to do dual-stack throughout the network.
I have one site setup working perfectly - "gre0". It's passing IPv4 and IPv6 traffic over the GRE tunnel. One *key* point to this is that the GRE connection endpoints are iPv6. IPv4 isn't doable for this connection (again, long story but not an option). When setting up the second GRE tunnel "gre2", nothing would work even though the configuration was the same EXCEPT for the fact that the second GRE tunnel was using IPv4 addresses for the GRE tunnel endpoints. Linux keeps spitting out a very odd error when I try to ping across the tunnel I cannot find reasonably documented anywhere:
ip6_tunnel: gre2 xmit: Local address not yet configured!The key was the "ip6_tunnel" part that took me awhile to figure out. After experimenting, I've found that if I have one GRE tunnel using IPv6 endpoints (ip -6 tunnel add gre0 mod ip6gre) and one using IPv4 endpoints (ip tunnel add gre2 mode gre), only the gre0 tunnel will work and the gre2 tunnel seems to believe it's missing an IPv6 address. If I delete the gre0 tunnel, the gre2 tunnel immediately beings working with both IPv4 and IPv6 traversing the GRE. I cannot find any documentation that describes this behavior or why it would be the case.
I can't switch to IPIP tunnels because I haven't found a way to do a dual-stack tunnel between the endpoints - seems like you can only do one IPIP tunnel between two endpoint IPs regardless of type (ipip, ipip6, ip6ip6, sit)
Anyone have any deep wisdom on GRE tunnels?
Jason
44Net mailing list 44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Scott,
I did a little digging but my C is rusty and I’m not a kernel hacker either. It was Linux 4.19.0.6 amd64 as provided by Debian 10.
I ended up solving the problem by doing 4in6 (IPIP6) and then 6in4 as an inner layer to get tunneling of both v4 and v6 in from the gateway. For IPv4 outer endpoints is 4in4 (IPIP) and then 6in4 inside that. Works well and the packet header overhead isn’t problematic so far.
Jason
From: Scott Nicholas scott.nicholas@scottn.us Sent: Friday, November 22, 2019 7:58 AM To: AMPRNet working group 44net@mailman.ampr.org Cc: Jason McCormick jason@mfamily.org Subject: Re: [44net] Linux GRE Tunnels
I'm no gre expert. I have only used it over v4. But I am willing to look at code.
What kernel version are you using?
On Sat, Nov 16, 2019, 9:01 PM Jason McCormick via 44Net <44net@mailman.ampr.orgmailto:44net@mailman.ampr.org> wrote: I'm running into an interesting problem I thought maybe someone in this community would have run into before.
We're working on a project to redesign and re-IP our entire radio network. Part of the goal is to stop using OpenVPN and IPSec tunnels (long story) and move exclusively to GRE-based tunnels. The plan is to have two Linux VPS hosts running at a provider with our 44Net allocation and an IPv6 allocation advertised and then routed into the radio network over GRE tunnels to 3 different locations that are all backhauled together. One of the primary goals is to do dual-stack throughout the network.
I have one site setup working perfectly - "gre0". It's passing IPv4 and IPv6 traffic over the GRE tunnel. One *key* point to this is that the GRE connection endpoints are iPv6. IPv4 isn't doable for this connection (again, long story but not an option). When setting up the second GRE tunnel "gre2", nothing would work even though the configuration was the same EXCEPT for the fact that the second GRE tunnel was using IPv4 addresses for the GRE tunnel endpoints. Linux keeps spitting out a very odd error when I try to ping across the tunnel I cannot find reasonably documented anywhere:
ip6_tunnel: gre2 xmit: Local address not yet configured!
The key was the "ip6_tunnel" part that took me awhile to figure out. After experimenting, I've found that if I have one GRE tunnel using IPv6 endpoints (ip -6 tunnel add gre0 mod ip6gre) and one using IPv4 endpoints (ip tunnel add gre2 mode gre), only the gre0 tunnel will work and the gre2 tunnel seems to believe it's missing an IPv6 address. If I delete the gre0 tunnel, the gre2 tunnel immediately beings working with both IPv4 and IPv6 traversing the GRE. I cannot find any documentation that describes this behavior or why it would be the case.
I can't switch to IPIP tunnels because I haven't found a way to do a dual-stack tunnel between the endpoints - seems like you can only do one IPIP tunnel between two endpoint IPs regardless of type (ipip, ipip6, ip6ip6, sit)
Anyone have any deep wisdom on GRE tunnels?
Jason
_________________________________________ 44Net mailing list 44Net@mailman.ampr.orgmailto:44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
Just to circle back to this issue…
GRE does work as expected having v4 + v6 dual-stacked inside of a single v4 or v6 set of endpoints. And you can mix and match them. The key is that you cannot use the default gre0 device as it appears to hold some sort of templating info. If any of the endpoints use gre0, they all work inconsistently. However if you use, as in our case, gre1 – gre3 – everything works fine. The switch to GRE has been terrific for stability over using IPSec VTIs + OpenVPN. We were running into a lot of instability both with the VTI parts and OpenVPN on some of the Pi-Star boxes or some combination of everything in between those endpoints.
Thanks to a few of you who provided suggestions and ideas mostly off-list on my issue.
Jason N8EI
From: Scott Nicholas scott.nicholas@scottn.us Sent: Friday, November 22, 2019 7:58 AM To: AMPRNet working group 44net@mailman.ampr.org Cc: Jason McCormick jason@mfamily.org Subject: Re: [44net] Linux GRE Tunnels
I'm no gre expert. I have only used it over v4. But I am willing to look at code.
What kernel version are you using?
On Sat, Nov 16, 2019, 9:01 PM Jason McCormick via 44Net <44net@mailman.ampr.orgmailto:44net@mailman.ampr.org> wrote: I'm running into an interesting problem I thought maybe someone in this community would have run into before.
We're working on a project to redesign and re-IP our entire radio network. Part of the goal is to stop using OpenVPN and IPSec tunnels (long story) and move exclusively to GRE-based tunnels. The plan is to have two Linux VPS hosts running at a provider with our 44Net allocation and an IPv6 allocation advertised and then routed into the radio network over GRE tunnels to 3 different locations that are all backhauled together. One of the primary goals is to do dual-stack throughout the network.
I have one site setup working perfectly - "gre0". It's passing IPv4 and IPv6 traffic over the GRE tunnel. One *key* point to this is that the GRE connection endpoints are iPv6. IPv4 isn't doable for this connection (again, long story but not an option). When setting up the second GRE tunnel "gre2", nothing would work even though the configuration was the same EXCEPT for the fact that the second GRE tunnel was using IPv4 addresses for the GRE tunnel endpoints. Linux keeps spitting out a very odd error when I try to ping across the tunnel I cannot find reasonably documented anywhere:
ip6_tunnel: gre2 xmit: Local address not yet configured!
The key was the "ip6_tunnel" part that took me awhile to figure out. After experimenting, I've found that if I have one GRE tunnel using IPv6 endpoints (ip -6 tunnel add gre0 mod ip6gre) and one using IPv4 endpoints (ip tunnel add gre2 mode gre), only the gre0 tunnel will work and the gre2 tunnel seems to believe it's missing an IPv6 address. If I delete the gre0 tunnel, the gre2 tunnel immediately beings working with both IPv4 and IPv6 traversing the GRE. I cannot find any documentation that describes this behavior or why it would be the case.
I can't switch to IPIP tunnels because I haven't found a way to do a dual-stack tunnel between the endpoints - seems like you can only do one IPIP tunnel between two endpoint IPs regardless of type (ipip, ipip6, ip6ip6, sit)
Anyone have any deep wisdom on GRE tunnels?
Jason
_________________________________________ 44Net mailing list 44Net@mailman.ampr.orgmailto:44Net@mailman.ampr.org https://mailman.ampr.org/mailman/listinfo/44net
-
Jason McCormick -
Scott Nicholas