GoogleBot - (Automatically?) running VRS Reports?

Want to post something that doesn't quite fit into the other forums? This is the place for that.
Post Reply
SkierInAvon
Posts: 11
Joined: Fri Jun 28, 2013 5:30 pm

GoogleBot - (Automatically?) running VRS Reports?

Post by SkierInAvon » Thu Feb 12, 2015 3:11 pm

General Awareness....

Just caught this - this morning (12FEB2015)...
As we all know...Google has been indexing the Web and Web sites for many years...

Today - I saw the GoogleBot - no only indexing my VRS web site - BUT also (RUNNING REPORTS??!!!)

The reports I saw the GoogleBot running....

/DateReport.htm
/IcaoReport.htm
/ReportRows.json

The VRS Reports being run were being from from Goggle at (66.249.69.12 66.249.69.233 66.249.69.248) respectively.
Just run a simple (Trace Route) back to those IP address and you'll see it's Google...

Not sure (how?) Google is actually running reports (filling out a web form?) in addition to Google's normal indexing of web pages... Curious

Your comments welcome.
Yes, I know I can lock down my VRS web site using passwords required for access.

I didn't know just how cleaver Google was getting.

Your comments welcome.

-pete :shock:

agw
Posts: 2241
Joined: Fri Feb 17, 2012 3:20 am

Re: GoogleBot - (Automatically?) running VRS Reports?

Post by agw » Sat Feb 14, 2015 5:57 pm

Those reports are links in the site's HTML, it's just following those links. I think this may be a side-effect for sites in the directory on the VRS web site, Google spiders the VRS web site fairly often and it will follow the links from the directory page to your site, and from there start following links to the reports.

You can tell Google what it can and cannot spider, I'll configure the web site to tell it not to follow the links on the directory page. I'll add a robots.txt to VRS as well to stop search engine following links to the reports.

Meghna
Posts: 2
Joined: Mon Jun 15, 2015 10:05 am
Contact:

Re: GoogleBot - (Automatically?) running VRS Reports?

Post by Meghna » Fri Jun 26, 2015 8:12 am

Blocking Googleboat is the best solution. Here is the code for Robots.txt....

user-agent:Googleboat
Disallow: /DateReport.htm
Disallow: /IcaoReport.htm
Disallow: /ReportRows.json

Post Reply