Time limit is when the content on the site was updated
Ruben - ON3RVH
> On 7 May 2019, at 19:37, Bill Vodall via 44Net <44net(a)mailman.ampr.org> wrote:
>
> <mime-attachment>
> _________________________________________
> 44Net mailing list
> 44Net(a)mailman.ampr.org
> https://mailman.ampr.org/mailman/listinfo/44net
> Rob,
> Sorry, I forgot to say which site: separs.ampr.org <http://separs.ampr.org/>.
> Yes, there are indexed ampr.org <http://ampr.org/> subdomains, but I don’t seem to be getting my point access. Those domains are older and indexing no longer works.
You can keep iterating that, but it is simply not true.
You undoubtedly have difficulties and I'm sure they are difficult to solve, but they aren't ampr.org wide.
> Eventually according to Google, they will fall off their indexing. Google indexing can no longer reach them or any new site.
Maybe your friend has told you that, but he has told you other things that are wrong. So I would not count on that.
> 5) I asked Brian to add a TXT entry, as per the Google Search console instructions.
It is in fact not required to do that, you can create a textfile with a name that is the same as what you put in the TXT record and it will work the same way.
I have added sites to the search console in the past and used that method, it worked fine.
> 6) Initialy Brian did not want to do it, saying he didn’t want Google crawling all the 44net IPs
Contrary to what some "researchers" (and some of our fellow amateur radio operators) enjoy doing, Google is not portscanning the IP space to find sites to crawl.
Google indexes HTTP links and follows them. Of course this takes bandwidth, but that is not much when compared to the many many black-hat and white-hat portscanners.
> 9)I attempted to have the Google Search Console crawl the site. It declined indicating "no errors" but an “anomaly”
I cannot help you with that. Your site appears OK from the outside, and my site(s) are indexed just fine.
There is only one thing I cannot check: there could be some firewall filter that drops Google indexing (usually from 66.249.64.0/19) in the ampr gateway or in another system in the path to you.
(our local sites within 44.137.0.0/16 are not routed via that gateway so we are not subject to that filter if it exists)
However I don't think that is the case because there are other sites that are being indexed.
Rob
Hi there
Has anyone done / do Gateway with home router running OpenWrt ?
If yes I have some questions as the WIKI is not so clear to non software man like me .
How the routing made ? does the ADSL interface get the home address (i.e 192.168.1.x in my case) and the lan port get the 44 Net address ?
or one Interface deal with Both network like i do in my MikroTik ?
what about configuration ? is there any way to do it via the GUI ? or only with the line mode ?
I mean that the Router sit on the home DMZ (that supplied from the main home router) and the Adress of the outside world arrive to it for the tunnel from UCSD
I saw that there is IPIP Package that support IPIP so what the difference and why the Kernel_ipip mode needed ?
and last where can i get executable of the ripd ? i can not compile ...
the router planed for the task is D-Link DSL2650 with broadcom 6358 That I Flashed for OpenWRT
I have heard that the OPENWRT is not a stable system and get stuck and needed reset here and there can someone confirm that ?
Thanks in advance
Ronen - 4Z4ZQ
http://www.ronen.org
Ronen Pinchooks (4Z4ZQ) WebSite<http://www.ronen.org/>
ronen.org (Ronen Pinchooks (4Z4ZQ) WebSite) is hosted by domainavenue.comwww.ronen.org
> All,Don't these sites all appear to be indexed in the past?73,- LynwoodKB3VWG
> null
When checking for a site within our network, I see that Googlebot comes by almost every day, and
when I check the site on Google and retrieve the page "from cache", I get pages dated April 9th and 10th.
This reconfirms my earlier observation that it may take a couple of weeks for crawled pages to appear
on the Google site.
Rob
> When one asked what your site was, you didn't reply. If it's va7llb.ampr.org,
> it is unreachable from the internet today. That'd stop more than Google
> indexer...
As we still did not get that information, I checked the DNS zone and there are only two
TXT records for google site verification. One is for va4wan.ampr.org and their site is
indexed by Google, no problem there. You can lookup site:va4wan.ampr.org to confirm that.
(again: the "site:" in that is important, don't omit it)
The other is for separs.ampr.org and Google does not yet know about them.
So probably he is discussing that. It appears to work OK when visited inside and outside
of AMPRnet and its robots.txt also appears OK.
I think it is just external linking that is required. Try to get a link to your site from
the city website where it had an information page before. Make links from your local
club, from private sites of your members, etc. Then Google becomes more convinced that
this is a useful site that people may want to visit, rather than some scam that wants to
have quick visitors and then disappear. (of which they probably get thousands a day submitted)
And of course you need patience. It will not happen overnight.
We all know that it is difficult to communicate with them and that they do what they themselves
consider appropriate. You will have to live with that, we all do.
Rob
Hi all,
I’m a bit sorry to have brought the issue of google’s inability to crawl ampr.org subdomains. I thought others might also feel that it would be nice to have their ARES group or other Ham group represented on the Ham network (44Net) for the work to see. Instead it feels like there is no interest at all in figuring out why this happens, instead blaming it on Google, or me for brining the issue forward and trying to contact google for help (you do realize that the only way to actually talk to google is to know someone there, so I’m sorry that I have “a friend” at google that tried to help with some inside information that the google search console does not provide.)
Roger
VA7LBB
> On May 4, 2019, at 12:00, 44net-request(a)mailman.ampr.org wrote:
>
> Send 44Net mailing list submissions to
> 44net(a)mailman.ampr.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://mailman.ampr.org/mailman/listinfo/44net
> or, via email, send a message with subject or body 'help' to
> 44net-request(a)mailman.ampr.org
>
> You can reach the person managing the list at
> 44net-owner(a)mailman.ampr.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of 44Net digest..."
> Today's Topics:
>
> 1. Re: Google indexing (Roger)
> 2. Re: Google indexing (Roger)
> 3. Re: Google indexing (Steve L)
> 4. Re: Google indexing (Rob Janssen)
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> _______________________________________________
> 44Net mailing list
> 44Net(a)mailman.ampr.org
> https://mailman.ampr.org/mailman/listinfo/44net
Rob,
When I put ampr.org in the google search almost all come back as first the TLD and wiki. The others are sites talking about ampr.org and the group that are subdomains are old dead links.
As I said in the first email, I did consult the google search console and submitted the site. I’ve submitted the site and Brian was kind enough to add. a TXT line (as per google search console) with the code proving I have the right to submit the domain. That is several weeks ago and the search console shows it attempted to crawl. Googles server now (unlike years ago) honor request to not crawl. It attempted the crawl and declined to proceed.
You probably know that it’s nearly impossible to contact google for support. Therein lies my luck to have a friend that could look behind the scene at the google server. His answer is that this is not a google problem but an issue with some sort of blocking at the TLD. He just can’t tell what that is except, the google crawler is being told to not crawl subdomains in some way.
All I was asking was that if someone more knowledgeable than me with access to the DNS could see what possibly would cause this denial.
Have a careful look through the google search response to ampr.org. There are none.
Roger.
VA7LBB
On May 4, 2019, at 02:19, Rob Janssen <pe1chl(a)amsat.org> wrote:
>> My friend at google isn’t 100% sure what’s happening
>
> That is very obvious!
> Maybe you should consult the google search engine itself, instead of your friend.
>
> Enter this in the Google search bar: site:ampr.org
>
> This returns 17700 results of all kinds of different sites below .ampr.org.
> You see the domain names xxxx.ampr.org listed in the results.
>
> So there really is no problem indexing ampr.org sites.
>
> I think what you encounter is the difficulty to add a NEW site to the index, not any generic
> problem with .ampr.org.
> They probably aren't that eager to add a new site, of which you specify only a start URL, to
> their index. And rightly so. That capability was likely abused widely.
>
> Instead, you should (ask to) put links to your site from other sites relevant to the subject
> of your own site. E.g. the page of your local club, other people's private sites, etc.
> Then, after some time, indexing will occur.
>
> Rob
>
>
>
>
> My friend at google isn’t 100% sure what’s happening
That is very obvious!
Maybe you should consult the google search engine itself, instead of your friend.
Enter this in the Google search bar: site:ampr.org
This returns 17700 results of all kinds of different sites below .ampr.org.
You see the domain names xxxx.ampr.org listed in the results.
So there really is no problem indexing ampr.org sites.
I think what you encounter is the difficulty to add a NEW site to the index, not any generic
problem with .ampr.org.
They probably aren't that eager to add a new site, of which you specify only a start URL, to
their index. And rightly so. That capability was likely abused widely.
Instead, you should (ask to) put links to your site from other sites relevant to the subject
of your own site. E.g. the page of your local club, other people's private sites, etc.
Then, after some time, indexing will occur.
Rob
Brian,
My friend at google isn’t 100% sure what’s happening, just that when the crawler attempts the crawl, it comes back with the same code it gets for a robots.txt or something similar. So to clarify, google claims, something at the “top domain” (their words) has prevented almost all subdomains from being crawled. And whatever that is, it clearly has an affect because I really don’t find any results for ampr.org, except the main site, portal and wiki.
Jann:
I realize there are many other reasons and uses for amprnet. I was just talking about webserving some sort of content out to the internet at large. The issue I’m having affects anyone using 44Net, including those in your examples, that are wanting it discoverable by the world at large.
73
Roger
VA7LBB
> On May 2, 2019, at 19:31, Brian Kantor <Brian(a)bkantor.net> wrote:
>
> Having the robots.txt file at the site AMPR.ORG apply to pages
> served by that host is completely reasonable.
>
> Having the robots.txt file at the site AMPR.ORG apply to pages
> served from an entirely separate site called, e.g., VA7LBB.AMPR.ORG
> is one of the most colossal pieces of internet engineering stupidity
> I have ever encountered.
>
> Yet it would appear that is what the person at Google is telling
> you is happening to your site.
>
>> From Wikipedia:
> "A robots.txt file covers one origin. For websites with multiple
> subdomains, each subdomain must have its own robots.txt file.
> If example.com had a robots.txt file but a.example.com did not,
> the rules that would apply for example.com WOULD NOT APPLY to
> a.example.com. In addition, each protocol and port needs its own
> robots.txt file; http://example.com/robots.txt does not apply
> to pages under http://example.com:8080/ or https://example.com/"
>
> If that's not what their crawlers are doing, their crawlers are broken.
> They broke it, they get to fix it. It's not our problem.
> - Brian
>
>
>
>
>> On Thu, May 02, 2019 at 06:13:40PM -0700, Roger Andrews wrote:
>> Hi,
>> I recently talked with Brian briefly about this and wanted to
>> throw it out to the group. It’s incredibly rare to see any of the
>> tunnels that have been created, represented in a Google search.
>> While I understand and agree that any site that will become a high
>> volume site has no place on Amprnet (we have to share resources)
>> it also seems pointless to create a website that is undiscoverable.
>> After all, isn’t the primary purpose of a website to share it’s
>> content with others. I recently created a website on a 44net gateway
>> and after several weeks, (and even convincing Brian to add a meta
>> TXT entry allowing me to ask google to crawl), I am not seeing any
>> content on Google. I put in a service request to google (not the
>> easiest task) and I was advised that robots.txt or some other
>> prevention device is blocking indexing all the subdirectories on
>> amp.org. I was told that the few gateways that I see in the results
>> were likely crawled before the restriction on ampr.org was applied.
>> I created the website for our ARES group and placed it on an ampr
>> gateway because we don’t have funds, and in reality, see very little
>> traffic. We had a .net site last year and averaged about 50 visitors
>> a month. My question is - is it really necessary to prevent the
>> whole of ampr.org from being crawled (except of course the top
>> domain which does show up). So many ip addresses, but almost none
>> visible seems a real pity.
>> Thanks for listening. My only hope is that this creates a little
>> bit of debate around the issue.
>>
>> 73
>> Roger
>> VA7LBB
>
