If your gateway appears in the pkterrors.txt file, the packets which caused that error to be logged and the packet to be dropped are now available in a file you can retrieve with your web browser. They are binary log files, so you'll need a program to interpret them. The URL for a typical file is
https://gw.ampr.org/private/errors/67.164.64.8.bin
where of course the IP address part changes to whatever your gateway address is. The files are removed and start fresh at midnight Pacific time (GMT-7 or -8). For some error-prone sites, they get large-ish.
The format of each file is /* * 2 bytes error number (unsigned short) * 2 bytes packet length (unsigned short) * 4 bytes time (seconds since epoch) * 4 bytes fractional seconds (microseconds) * n bytes (packetlen) encapped IP packet in network byte order */
I have a 'C' program that will interpret the file, which you may have if you're interested. It calls some library routines that you probably don't have, so you'll have to modify it to get it to run on your system. In particular, the error number and packet content interpreters are up to you. I don't think the compiled code will run on Linux but you're welcome to it if you have a FreeBSD system to run it on.
Or if you like, I'll run it on your gateway error log file and mail you the output. That looks like this:
timestamp (GMT) len err error -------------------- ---- ---- ---------- 2017-05-08T19:04:32Z 40 [19] dropped: non-44 inner source address ver=4 hl*4=20 tos=00 ip_len=40 id=d431 off=0000 ttl=244 proto=6 cksum=7d51 [TCP] 184.105.139.107:44864 -> 44.118.5.2:16992
2017-05-08T19:05:51Z 40 [19] dropped: non-44 inner source address ver=4 hl*4=20 tos=00 ip_len=40 id=18f4 off=0000 ttl=43 proto=6 cksum=5b02 [TCP] 181.23.53.75:58147 -> 44.118.5.2:2222
etc.
So far, keeping these log files doesn't seem to burden the system very much. If that changes, I'll have to discontinue them. - Brian
On Mon, May 8, 2017 at 12:38 PM, Brian Kantor Brian@ucsd.edu wrote:
(Please trim inclusions from previous messages) _______________________________________________ If your gateway appears in the pkterrors.txt file, the packets which caused that error to be logged and the packet to be dropped are now available in a file you can retrieve with your web browser. They are binary log files, so you'll need a program to interpret them. The URL for a typical file is
https://gw.ampr.org/private/errors/67.164.64.8.bin
where of course the IP address part changes to whatever your gateway address is. The files are removed and start fresh at midnight Pacific time (GMT-7 or -8). For some error-prone sites, they get large-ish.
The format of each file is /*
- 2 bytes error number (unsigned short)
- 2 bytes packet length (unsigned short)
- 4 bytes time (seconds since epoch)
- 4 bytes fractional seconds (microseconds)
- n bytes (packetlen) encapped IP packet in network byte order
*/
Brian,
This is a neat feature!
Would you consider changing the format to pcap or pcapng? This would allow viewing the packets in Wireshark. The format isn't much more complicated than the format you've chosen:
https://wiki.wireshark.org/Development/LibpcapFileFormat
Tom KD7LXL
That's a good idea. Hadn't occured to me; I'll look into it. - Brian
On Mon, May 08, 2017 at 12:46:02PM -0700, Tom Hayward wrote:
Would you consider changing the format to pcap or pcapng? This would allow viewing the packets in Wireshark. The format isn't much more complicated than the format you've chosen:
Well, I have it creating pcap files, but tcpdump doesn't like them.
And the structure definitions in libpcap don't agree with those on the web page https://wiki.wireshark.org/Development/LibpcapFileFormat so I'm going to have to reverse-engineer tcpdump to find out just what it's really expecting. But we're partway there. - Brian
On Mon, May 08, 2017 at 12:46:02PM -0700, Tom Hayward wrote:
Would you consider changing the format to pcap or pcapng? This would allow viewing the packets in Wireshark. The format isn't much more complicated than the format you've chosen:
https://wiki.wireshark.org/Development/LibpcapFileFormat
Tom KD7LXL
Well, isn't *that* special. Turns out the system declared time structure is 64-bit and the on-disk capture files use 32-bit. Once I corrected for that, the router is now writing error files that tcpdump is very happy with. I assume wireshark will be too.
So now, if your gateway is sending packets that contain errors and would be dropped and are therefore showing up in the pkterrors.txt file, you can grab a capture file containing those erroneous packets from, for example, https://gw.ampr.org/private/errors/1.2.3.4.pcap (Of course, replace 1.2.3.4 with your own gateway address.)
The only thing missing is that pcap files have no way to contain an indication of what the error was that caused that particular packet to be rejected. You'll just have to correlate them with the errors reported in the pkterrors.txt file for your gateway.
Keep in mind that the pkterrors.txt file and the accumulated pcap files are deleted and start fresh every day at midnight Pacific time (GMT-7 or -8).
This is fun! - Brian
On Mon, May 08, 2017 at 12:46:02PM -0700, Tom Hayward wrote:
Would you consider changing the format to pcap or pcapng? This would allow viewing the packets in Wireshark. The format isn't much more complicated than the format you've chosen:
https://wiki.wireshark.org/Development/LibpcapFileFormat
Tom KD7LXL
This is interesting. No sooner did I post the message containing the URL when something/someone attempted to retrieve it. I mean, within *seconds* of the posting. And they attempted to retrieve the example address, which is clearly fake. So I'm going to list another URL https://gw.ampr.org/private/no-such-file.txt and see if it's an automated process that tries to fetch them.
Please excuse this bogus email. - Brian
Yup, within 19 seconds of my posting the message, something tried to fetch the no-such-file. I doubt it was a person. Very interesting. - Brian
gw.ampr.org-ssl 70.39.157.194 - - [08/May/2017:17:56:38 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381
On Mon, May 08, 2017 at 05:56:19PM -0700, Brian Kantor wrote:
This is interesting. No sooner did I post the message containing the URL when something/someone attempted to retrieve it. I mean, within *seconds* of the posting. And they attempted to retrieve the example address, which is clearly fake. So I'm going to list another URL https://gw.ampr.org/private/no-such-file.txt and see if it's an automated process that tries to fetch them.
Please excuse this bogus email.
- Brian
Is the content of the list posted on a website? Like for archiving purposes? Otherwise someone might have a mailbox from the list configured with a scraper.. Can you see which is the source IP for the requests? Maybe this can lead you to the correct person..
73,
Ruben - ON3RVH
-----Original Message----- From: 44Net [mailto:44net-bounces+on3rvh=on3rvh.be@hamradio.ucsd.edu] On Behalf Of Brian Kantor Sent: dinsdag 9 mei 2017 3:09 To: AMPRNet working group 44net@hamradio.ucsd.edu Subject: Re: [44net] gateway errors detail available
(Please trim inclusions from previous messages) _______________________________________________ Yup, within 19 seconds of my posting the message, something tried to fetch the no-such-file. I doubt it was a person. Very interesting. - Brian
gw.ampr.org-ssl 70.39.157.194 - - [08/May/2017:17:56:38 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381
On Mon, May 08, 2017 at 05:56:19PM -0700, Brian Kantor wrote:
This is interesting. No sooner did I post the message containing the URL when something/someone attempted to retrieve it. I mean, within *seconds* of the posting. And they attempted to retrieve the example address, which is clearly fake. So I'm going to list another URL https://gw.ampr.org/private/no-such-file.txt and see if it's an automated process that tries to fetch them.
Please excuse this bogus email.
- Brian
_________________________________________ 44Net mailing list 44Net@hamradio.ucsd.edu http://hamradio.ucsd.edu/mailman/listinfo/44net
Yes, the host 'hamradio.ucsd.edu' where the mailing list is hosted also archives all messages and makes them available via the web.
It's amusing: despite not being real, from the following Apache log entries, no-such-file has been a popular target this evening:
gw.ampr.org-ssl 70.39.157.194 - - [08/May/2017:17:56:38 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 66.85.73.59 - - [08/May/2017:17:58:59 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 220.233.167.221 - - [08/May/2017:17:59:43 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 4.79.123.0 - - [08/May/2017:18:09:35 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 68.40.58.30 - - [08/May/2017:18:56:26 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 4.79.123.0 - - [08/May/2017:21:42:57 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 165.225.80.161 - - [08/May/2017:21:48:27 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381
The first one above was logged just 19 seconds after I posted the message to the list. The others came later; they could be humans. Note that none of them logged in; they just tried to fetch the file and went away. Oh, I'm not concerned, I just was a bit surprised. And curious. - Brian
On Tue, May 09, 2017 at 04:42:26AM +0000, Ruben ON3RVH wrote:
Is the content of the list posted on a website? Like for archiving purposes? Otherwise someone might have a mailbox from the list configured with a scraper.. Can you see which is the source IP for the requests? Maybe this can lead you to the correct person.. 73, Ruben - ON3RVH
Brian,
Guilty as charged - 220.233.167.221.
I'm clicking on pretty much every link in these postings to try to come to grips with the complexity (to me) of ampr.net as I try to decide if I am capable of setting up a system and maintaining it.
Ray vk2tv
On 09/05/17 16:27, Brian Kantor wrote:
(Please trim inclusions from previous messages) _______________________________________________ Yes, the host 'hamradio.ucsd.edu' where the mailing list is hosted also archives all messages and makes them available via the web.
It's amusing: despite not being real, from the following Apache log entries, no-such-file has been a popular target this evening:
gw.ampr.org-ssl 70.39.157.194 - - [08/May/2017:17:56:38 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 66.85.73.59 - - [08/May/2017:17:58:59 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 220.233.167.221 - - [08/May/2017:17:59:43 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 4.79.123.0 - - [08/May/2017:18:09:35 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 68.40.58.30 - - [08/May/2017:18:56:26 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 4.79.123.0 - - [08/May/2017:21:42:57 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 165.225.80.161 - - [08/May/2017:21:48:27 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381
The first one above was logged just 19 seconds after I posted the message to the list. The others came later; they could be humans. Note that none of them logged in; they just tried to fetch the file and went away. Oh, I'm not concerned, I just was a bit surprised. And curious.
- Brian
Ray,
Don't let the complexity deter you; a lot of the fun in ham radio and amateur networking experimentation is doing something that no one else has done. It seems to me that just about every system on AMPRNet is unique in some way, so it's not too surprising that there's no turnkey solution. We all have different equipment and skills.
That having been said, I'm quite conscious of the lack of some of the documentation that could make things easier for newcomers. That's why we set up the wiki, so people who have solved problems can post their advice and knowledge for others. But writing clearly takes time and effort, so it's slow going.
Hang in there! - Brian
On Tue, May 09, 2017 at 05:03:19PM +1000, vk2tv wrote:
I'm clicking on pretty much every link in these postings to try to come to grips with the complexity (to me) of ampr.net as I try to decide if I am capable of setting up a system and maintaining it.
Ray vk2tv
66.85.73.59 is me through my dedicated server VPN
:)
On May 9, 2017, at 2:27 AM, Brian Kantor Brian@ucsd.edu wrote:
(Please trim inclusions from previous messages) _______________________________________________ Yes, the host 'hamradio.ucsd.edu' where the mailing list is hosted also archives all messages and makes them available via the web.
It's amusing: despite not being real, from the following Apache log entries, no-such-file has been a popular target this evening:
gw.ampr.org-ssl 70.39.157.194 - - [08/May/2017:17:56:38 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 66.85.73.59 - - [08/May/2017:17:58:59 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 220.233.167.221 - - [08/May/2017:17:59:43 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 4.79.123.0 - - [08/May/2017:18:09:35 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 68.40.58.30 - - [08/May/2017:18:56:26 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 4.79.123.0 - - [08/May/2017:21:42:57 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381 gw.ampr.org-ssl 165.225.80.161 - - [08/May/2017:21:48:27 -0700] "GET /private/no-such-file.txt HTTP/1.1" 401 381
The first one above was logged just 19 seconds after I posted the message to the list. The others came later; they could be humans. Note that none of them logged in; they just tried to fetch the file and went away. Oh, I'm not concerned, I just was a bit surprised. And curious.
- Brian
On Tue, May 09, 2017 at 04:42:26AM +0000, Ruben ON3RVH wrote:
Is the content of the list posted on a website? Like for archiving purposes? Otherwise someone might have a mailbox from the list configured with a scraper.. Can you see which is the source IP for the requests? Maybe this can lead you to the correct person.. 73, Ruben - ON3RVH
44Net mailing list 44Net@hamradio.ucsd.edu http://hamradio.ucsd.edu/mailman/listinfo/44net
On Mon, May 8, 2017 at 6:09 PM, Brian Kantor Brian@ucsd.edu wrote:
Yup, within 19 seconds of my posting the message, something tried to fetch the no-such-file. I doubt it was a person. Very interesting. - Brian
Maybe one of the subscribers here has a mail server with a spam/virus detection system that checks all URLs in incoming messages for nefarious content.
Tom KD7LXL
Yes, that seems possible. I haven't heard of one that does that, but who knows what's out there these days? - Brian
On Mon, May 08, 2017 at 10:05:24PM -0700, Tom Hayward wrote:
Maybe one of the subscribers here has a mail server with a spam/virus detection system that checks all URLs in incoming messages for nefarious content.
Tom KD7LXL
-
Brian Kantor -
KK6NEI (Dylan) -
Ruben ON3RVH -
Tom Hayward -
vk2tv