Hey all, I recently got my gateway and ip block allocations all setup including the DNS piece thanks to Brian and others.
I've got the tunnel defined on my router. When I run a TCPDUMP on the physical interface that is hooked up to m169.228.66.251y cable modem, I am not seeing any traffic from the 169.228.66.251 tunnel endpoint. Nor, obviously, am I seeing any traffic on my tunnel interface.
How do I troubleshoot this?
Thanks
Craig KC1ETB
On Thu, May 18, 2017 at 09:37:59AM -0400, Craig Brauckmiller wrote:
I've got the tunnel defined on my router. When I run a TCPDUMP on the physical interface that is hooked up to m169.228.66.251y cable modem, I am not seeing any traffic from the 169.228.66.251 tunnel endpoint. Nor, obviously, am I seeing any traffic on my tunnel interface.
How do I troubleshoot this?
Thanks
Craig KC1ETB
Your cable modem/router is rejecting incoming packets from 169.228.66.251:
# ping 96.86.86.53 PING 96.86.86.53 (96.86.86.53): 56 data bytes 36 bytes from 96-86-86-53-static.hfc.comcastbusiness.net (96.86.86.53): Destination Port Unreachable Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 20 0054 4bd2 0 0000 2e 01 9d4c 169.228.66.251 96.86.86.53
36 bytes from 96-86-86-53-static.hfc.comcastbusiness.net (96.86.86.53): Destination Port Unreachable Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 20 0054 4d86 0 0000 2e 01 9b98 169.228.66.251 96.86.86.53
36 bytes from 96-86-86-53-static.hfc.comcastbusiness.net (96.86.86.53): Destination Port Unreachable Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 20 0054 4ee7 0 0000 2e 01 9a37 169.228.66.251 96.86.86.53
However, pseudo-RIP and IPIP are being sent to you, so once you fix the problem with your modem/router you should be on your way. - Brian
Nope, that's my firewall dropping that. I don't allow ICMP to hit the 96.86.86.53 address.
My tests included both PING and HTTPS. HTTPS is allowed through. The bigger point I'm making here is that I ran TCPDUMP at the OS level which would show the packets before the firewall rules are applied.
As a test, you can go to : https://96.86.86.53 This is my VPN appliance and you will get a login page there. At least...you should.
Thanks
Craig
On Thu, May 18, 2017 at 9:47 AM, Brian Kantor Brian@ucsd.edu wrote:
(Please trim inclusions from previous messages) _______________________________________________ On Thu, May 18, 2017 at 09:37:59AM -0400, Craig Brauckmiller wrote:
I've got the tunnel defined on my router. When I run a TCPDUMP on the physical interface that is hooked up to m169.228.66.251y cable modem, I
am
not seeing any traffic from the 169.228.66.251 tunnel endpoint. Nor, obviously, am I seeing any traffic on my tunnel interface.
How do I troubleshoot this?
Thanks
Craig KC1ETB
Your cable modem/router is rejecting incoming packets from 169.228.66.251:
# ping 96.86.86.53 PING 96.86.86.53 (96.86.86.53): 56 data bytes 36 bytes from 96-86-86-53-static.hfc.comcastbusiness.net (96.86.86.53): Destination Port Unreachable Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 20 0054 4bd2 0 0000 2e 01 9d4c 169.228.66.251 96.86.86.53
36 bytes from 96-86-86-53-static.hfc.comcastbusiness.net (96.86.86.53): Destination Port Unreachable Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 20 0054 4d86 0 0000 2e 01 9b98 169.228.66.251 96.86.86.53
36 bytes from 96-86-86-53-static.hfc.comcastbusiness.net (96.86.86.53): Destination Port Unreachable Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 20 0054 4ee7 0 0000 2e 01 9a37 169.228.66.251 96.86.86.53
However, pseudo-RIP and IPIP are being sent to you, so once you fix the problem with your modem/router you should be on your way. - Brian
44Net mailing list 44Net@hamradio.ucsd.edu http://hamradio.ucsd.edu/mailman/listinfo/44net
On Thu, May 18, 2017 at 10:08:09AM -0400, Craig Brauckmiller wrote:
Nope, that's my firewall dropping that. I don't allow ICMP to hit the 96.86.86.53 address.
Then you have cut your own throat by disabling one of the internet's primary troubleshooting tools.
As a test, you can go to : https://96.86.86.53 This is my VPN appliance and you will get a login page there. At least...you should.
Yes, it connects from 169.228.66.251:
telnet 96.86.86.53 443 Trying 96.86.86.53... Connected to 96-86-86-53-static.hfc.comcastbusiness.net. Escape character is '^]'. ^] telnet> q Connection closed.
I have been told that comcast-provided CPE doesn't allow protocol 4 through it unless it is placed in raw bridge mode. - Brian
comcast likes to block a bunch of stuff...standard SIP is blocked as well (5060)
On 5/18/2017 10:17 AM, Brian Kantor wrote:
(Please trim inclusions from previous messages) _______________________________________________ On Thu, May 18, 2017 at 10:08:09AM -0400, Craig Brauckmiller wrote:
Nope, that's my firewall dropping that. I don't allow ICMP to hit the 96.86.86.53 address.
Then you have cut your own throat by disabling one of the internet's primary troubleshooting tools.
As a test, you can go to : https://96.86.86.53 This is my VPN appliance and you will get a login page there. At least...you should.
Yes, it connects from 169.228.66.251:
telnet 96.86.86.53 443 Trying 96.86.86.53... Connected to 96-86-86-53-static.hfc.comcastbusiness.net. Escape character is '^]'. ^] telnet> q Connection closed.
I have been told that comcast-provided CPE doesn't allow protocol 4 through it unless it is placed in raw bridge mode.
- Brian
44Net mailing list 44Net@hamradio.ucsd.edu http://hamradio.ucsd.edu/mailman/listinfo/44net
--- This email has been checked for viruses by AVG. http://www.avg.com
For clarity sake...
On Thu, 2017-05-18 at 07:20 -0700, Michael Fox - N6MEF wrote:
I have been told that comcast-provided CPE doesn't allow protocol 4 through it unless it is placed in raw bridge mode.
Comcast Business cable modems *do* allow protocol 4. I can't speak to the consumer service.
Comcast as a corporate policy does not have a filter on ip protocol 4 by default. While there are a good number of things they do filter, ip protocol 4 blocking is a byproduct of their socket watchdog timer... and it affects inbound only. Outbound is fine pending you're able to receive RIP. Unfortunately, there's no user menu control to disable this in their devices so one fix would be to place the Comcast CPE into bridge mode and supply your own device in which you have the ability to shut off the watchdog timers.
I wrote up documentation about this after spending a good part of this past saturday night working on some end point issues with BK. You may find it, and some tests, at https://n1uro.ampr.org/linuxconf/amprcable.html
One site in particular who was hit by this is 44.60.0.1 who after selecting the hardware solution mentioned in my document should now be reachable from the ipip mesh. Prior to this he was not.
Some of you may recall I experienced a similar issue a few years back. Fortunately I was able to get some information via an old friend of 44/8 who's one of their chief engineers in the Boston area to find solutions to this. While the effect appears that they filter ipencap/gre/etc they do not... it's the watchdog timers that do and it affects inbound traffic only.
While there may be many reasons for keeping these on and not allowing the end user to disable them, one logical one I can think of would be to allow the CPE to keep stale/old NAT sessions flushed so they're not hogging up ram resources unnecessarily. After all these aren't high end cisco or juniper devices.
Ok, is anyone on Google Chat aka Hangouts? I'd love to run some live troubleshooting.
Thanks
Craig KC1ETB@gmail.com for Hangouts.
On Thu, May 18, 2017 at 11:04 AM, Brian n1uro@n1uro.ampr.org wrote:
(Please trim inclusions from previous messages) _______________________________________________ For clarity sake...
On Thu, 2017-05-18 at 07:20 -0700, Michael Fox - N6MEF wrote:
I have been told that comcast-provided CPE doesn't allow protocol 4 through it unless it is placed in raw bridge mode.
Comcast Business cable modems *do* allow protocol 4. I can't speak to
the
consumer service.
Comcast as a corporate policy does not have a filter on ip protocol 4 by default. While there are a good number of things they do filter, ip protocol 4 blocking is a byproduct of their socket watchdog timer... and it affects inbound only. Outbound is fine pending you're able to receive RIP. Unfortunately, there's no user menu control to disable this in their devices so one fix would be to place the Comcast CPE into bridge mode and supply your own device in which you have the ability to shut off the watchdog timers.
I wrote up documentation about this after spending a good part of this past saturday night working on some end point issues with BK. You may find it, and some tests, at https://n1uro.ampr.org/linuxconf/amprcable.html
One site in particular who was hit by this is 44.60.0.1 who after selecting the hardware solution mentioned in my document should now be reachable from the ipip mesh. Prior to this he was not.
Some of you may recall I experienced a similar issue a few years back. Fortunately I was able to get some information via an old friend of 44/8 who's one of their chief engineers in the Boston area to find solutions to this. While the effect appears that they filter ipencap/gre/etc they do not... it's the watchdog timers that do and it affects inbound traffic only.
While there may be many reasons for keeping these on and not allowing the end user to disable them, one logical one I can think of would be to allow the CPE to keep stale/old NAT sessions flushed so they're not hogging up ram resources unnecessarily. After all these aren't high end cisco or juniper devices.
-- I don't have to worry about body fitness in 2017. All I do is show my body to itself in the mirror and it throws plenty of fits.
73 de Brian - N1URO/AFT1BR email: (see above) Web: http://www.n1uro.net/ Ampr1: http://n1uro.ampr.org/ Ampr2: http://nos.n1uro.ampr.org Linux Amateur Radio Services axMail-Fax & URONode http://uronode.sourceforge.net http://axmail.sourceforge.net AmprNet coordinator for: Connecticut, Delaware, Maine, Maryland, Massachusetts, New Hampshire, New Jersey, Pennsylvania, Rhode Island, and Vermont.
I might be able to run basic tests (better by email during the day, though), and I have ping going to your 44.44.7.17 from my 44.60.44.14, to see if you're getting my IPENCAP.
...if you prefer Hangouts, try LLeachii@gmail.com, I'll be in the field and to lunch. I don't have the app, but Google Voice should call my cell.
It's about noon in Washington, DC (15:50 UTC), if interested in a lunch chat. Just let me know.
- Lywnood KB3VWG
Sure...I can do a lunch chat. What time exactly? I'm outside of Boston.
Thanks
Craig
On Thu, May 18, 2017 at 11:52 AM, lleachii--- via 44Net < 44net@hamradio.ucsd.edu> wrote:
(Please trim inclusions from previous messages) _______________________________________________ I might be able to run basic tests (better by email during the day, though), and I have ping going to your 44.44.7.17 from my 44.60.44.14, to see if you're getting my IPENCAP.
...if you prefer Hangouts, try LLeachii@gmail.com, I'll be in the field and to lunch. I don't have the app, but Google Voice should call my cell.
It's about noon in Washington, DC (15:50 UTC), if interested in a lunch chat. Just let me know.
- Lywnood
KB3VWG
44Net mailing list 44Net@hamradio.ucsd.edu http://hamradio.ucsd.edu/mailman/listinfo/44net
I can just call you if you like. I've got a club meeting tonight. We are discussing our Field Day activities and don't want to miss that.
Maybe tomorrow or the weekend would be better?
Thanks
Craig
On Thu, May 18, 2017 at 11:56 AM, Craig Brauckmiller kc1etb@gmail.com wrote:
Sure...I can do a lunch chat. What time exactly? I'm outside of Boston.
Thanks
Craig
On Thu, May 18, 2017 at 11:52 AM, lleachii--- via 44Net < 44net@hamradio.ucsd.edu> wrote:
(Please trim inclusions from previous messages) _______________________________________________ I might be able to run basic tests (better by email during the day, though), and I have ping going to your 44.44.7.17 from my 44.60.44.14, to see if you're getting my IPENCAP.
...if you prefer Hangouts, try LLeachii@gmail.com, I'll be in the field and to lunch. I don't have the app, but Google Voice should call my cell.
It's about noon in Washington, DC (15:50 UTC), if interested in a lunch chat. Just let me know.
- Lywnood
KB3VWG
44Net mailing list 44Net@hamradio.ucsd.edu http://hamradio.ucsd.edu/mailman/listinfo/44net
Guys, I think I may be missing something here. I've seen a couple references to RIP being required. I didn't see that mentioned in any of the on-line docs. Perhaps I've missed it, and if so, I apologize. I'll go look at the Wiki to see if it is there.
Is this required or optional?
Thanks
Craig
On Thu, May 18, 2017 at 12:18 PM, Craig Brauckmiller kc1etb@gmail.com wrote:
I can just call you if you like. I've got a club meeting tonight. We are discussing our Field Day activities and don't want to miss that.
Maybe tomorrow or the weekend would be better?
Thanks
Craig
On Thu, May 18, 2017 at 11:56 AM, Craig Brauckmiller kc1etb@gmail.com wrote:
Sure...I can do a lunch chat. What time exactly? I'm outside of Boston.
Thanks
Craig
On Thu, May 18, 2017 at 11:52 AM, lleachii--- via 44Net < 44net@hamradio.ucsd.edu> wrote:
(Please trim inclusions from previous messages) _______________________________________________ I might be able to run basic tests (better by email during the day, though), and I have ping going to your 44.44.7.17 from my 44.60.44.14, to see if you're getting my IPENCAP.
...if you prefer Hangouts, try LLeachii@gmail.com, I'll be in the field and to lunch. I don't have the app, but Google Voice should call my cell.
It's about noon in Washington, DC (15:50 UTC), if interested in a lunch chat. Just let me know.
- Lywnood
KB3VWG
44Net mailing list 44Net@hamradio.ucsd.edu http://hamradio.ucsd.edu/mailman/listinfo/44net
It's not RIP as in RIPv2, it's AMPR-RIP or RIP44. It's not required, but it's a convenient alternative to downloading the ENCAP file periodically and installing it.
This is a mesh network, not a star, so there is no default route; you need a separate tunnel route to each gateway you want to communicate with, either maintained by the RIP44 daemon or by you periodically installing the ENCAP file.
The RIP44 packets are sent unsolicited to every gateway listed in the ENCAP file, every 5 minutes (and I've verified they're being sent to your gateway) so if you're not seeing any IPIP traffic, there is a problem on your end. - Brian
On Thu, May 18, 2017 at 12:49:15PM -0400, Craig Brauckmiller wrote:
Guys, I think I may be missing something here. I've seen a couple references to RIP being required. I didn't see that mentioned in any of the on-line docs. Perhaps I've missed it, and if so, I apologize. I'll go look at the Wiki to see if it is there.
Is this required or optional?
Thanks
Craig
I do that for security, but ok.
I just re-enabled ICMP.
Ping away! :)
Thanks
Craig
On Thu, May 18, 2017 at 10:17 AM, Brian Kantor Brian@ucsd.edu wrote:
(Please trim inclusions from previous messages) _______________________________________________ On Thu, May 18, 2017 at 10:08:09AM -0400, Craig Brauckmiller wrote:
Nope, that's my firewall dropping that. I don't allow ICMP to hit the 96.86.86.53 address.
Then you have cut your own throat by disabling one of the internet's primary troubleshooting tools.
As a test, you can go to : https://96.86.86.53 This is my VPN
appliance
and you will get a login page there. At least...you should.
Yes, it connects from 169.228.66.251:
telnet 96.86.86.53 443 Trying 96.86.86.53... Connected to 96-86-86-53-static.hfc.comcastbusiness.net. Escape character is '^]'. ^] telnet> q Connection closed.
I have been told that comcast-provided CPE doesn't allow protocol 4 through it unless it is placed in raw bridge mode. - Brian
44Net mailing list 44Net@hamradio.ucsd.edu http://hamradio.ucsd.edu/mailman/listinfo/44net
On Thu, May 18, 2017 at 10:25:05AM -0400, Craig Brauckmiller wrote:
I do that for security, but ok.
If you reply with port unreachable (if you reply at all) you've given away as much as answering a ping would -- that there is something there to try to break into. The only 'secure' way to hide it is to blackhole the ping request, if hiding is what you're trying to do. - Brian
On Thu, 18 May 2017, Brian Kantor wrote:
(Please trim inclusions from previous messages) _______________________________________________ On Thu, May 18, 2017 at 10:08:09AM -0400, Craig Brauckmiller wrote:
Nope, that's my firewall dropping that. I don't allow ICMP to hit the 96.86.86.53 address.
Then you have cut your own throat by disabling one of the internet's primary troubleshooting tools.
In addition to being an useful troubleshooting tool, ICMP packets are used for Path MTU Discovery (PMTUd), which is pretty good and essential Internet functionality when you have smaller MTUs involved. And, with our IPIP tunnels, we have just that! The encapsulation causes our link MTU to be smaller than the standard ethernet 1500 bytes.
https://en.wikipedia.org/wiki/Path_MTU_Discovery
"Many network security devices block all ICMP messages for perceived security benefits, including the errors that are necessary for the proper operation of PMTUD. This can result in connections that complete the TCP three-way handshake correctly, but then hang when data is transferred. This state is referred to as a black hole connection."
So, please do not block all of ICMP; ICMP Unreachable messages (type 3) should be permitted, especially on a network like ours.
- Hessu
Hessu,
It's my understanding that iptables should accept such a reply as ESTABLISHED or RELATED. I'll have to research.
This does not consider those who EXPLICITLY block ICMP as a top rule (or drop them in the RAW table), as you more closely described in your email.
...this is one advantage to setting up a DROP/REJECT-by-default iptables configuration.
Also, it prevents any errors in rules or their order from allowing a packet to go the opposite of a "blackhole..." i.e. actually navigate through the iptables chains to arrive at the unsecured LAN.
73,
- Lynwood KB3VWG
So, please do not block all of ICMP; ICMP Unreachable messages (type 3) should be permitted, especially on a network like ours.
Craig,
For clarity, are you using your own router, or one provided by the cable company?
If you're using your own router:
- Can you bypass the "router" and use the cable modem only, hook up a laptop and run Wireshark.
- We want to see if you're receiving IPENCAP packets. In Wireshark, they will appear as normal packets, but looking at the frames in the lower screen, the IPENCAP packet will have TWO IP headers instead of one.
- What is the make a model of the router (if they are separate devices)?
73,
- Lynwood KB3VWG
I've got the tunnel defined on my router. When I run a TCPDUMP on the physical interface that is hooked up to m169.228.66.251y cable modem
I am using my own router. Juniper SRX Firewall actually. The Comcast business modem is in "transparent mode". This means that it is a L2 device only. IP connectivity is handled by the SRX directly. I assigned my Comcast Business static IP to GE-0/0/6 on the SRX.
I can run TCPDUMP on the firewall at the Unix command prompt which means that I am seeing raw packets BEFORE the firewall rules are applied. Again, no traffic at all from the 44 net side of the world.
Thanks
Craig
On Thu, May 18, 2017 at 10:29 AM, lleachii--- via 44Net < 44net@hamradio.ucsd.edu> wrote:
(Please trim inclusions from previous messages) _______________________________________________ Craig,
For clarity, are you using your own router, or one provided by the cable company?
If you're using your own router:
- Can you bypass the "router" and use the cable modem only, hook up a
laptop and run Wireshark.
- We want to see if you're receiving IPENCAP packets. In Wireshark, they
will appear as normal packets, but looking at the frames in the lower screen, the IPENCAP packet will have TWO IP headers instead of one.
- What is the make a model of the router (if they are separate devices)?
73,
- Lynwood
KB3VWG
I've got the tunnel defined on my router. When I run a TCPDUMP on the
physical interface that is hooked up to m169.228.66.251y cable modem
44Net mailing list 44Net@hamradio.ucsd.edu http://hamradio.ucsd.edu/mailman/listinfo/44net
Craig,
Pinging (and connecting to port 443 on) 96.86.86.53 work from amprgw.
Tcpdump on amprgw shows that IPIP traffic is being sent to you from 169.228.66.251. If you're not seeing it, either your modem or the network are filtering it out. My bet is on the modem. - Brian
On Thu, May 18, 2017 at 10:43:19AM -0400, Craig Brauckmiller wrote:
I am using my own router. Juniper SRX Firewall actually. The Comcast business modem is in "transparent mode". This means that it is a L2 device only. IP connectivity is handled by the SRX directly. I assigned my Comcast Business static IP to GE-0/0/6 on the SRX.
I can run TCPDUMP on the firewall at the Unix command prompt which means that I am seeing raw packets BEFORE the firewall rules are applied. Again, no traffic at all from the 44 net side of the world.
Craig,
I can run TCPDUMP on the firewall at the Unix command prompt which means that I am seeing raw packets BEFORE the firewall rules are applied. Again, no traffic at all from the 44 net side of the world.
Yes, but are you CERTAIN that you run tcpdump *before* your Kernel process a tunneled packet?!?
The bigger point I'm making here is that I ran TCPDUMP at the OS level which would show the packets before the firewall rules are applied.
Not necessarily, which is why I suggested disconnecting the router. You seem to have implemented IPENCAP tunnel (probably a working of kmod-ipip for Juniper), and we can't be quite certain of how that operates (unless we have a Juniper OS Development Engineer available).
From encap.txt:
'route addprivate 44.44.7.16/29 encap 96.86.86.53'
I'll ping/trace 44.44.7.17 and see where that takes us, you should see me at the border at the cable modem PHY, if they are not blocked thru the ISP.
Also, how did you implement RIP44 (not the same as RIPv2) into Juniper?
73,
- Lynwood KB3VWG
-
Brian -
Brian Kantor -
Craig Brauckmiller -
Heikki Hannikainen -
Leon Zetekoff -
lleachii＠aol.com -
Michael Fox - N6MEF