http://www.alexa.com, an amazon website, gives stats on where websites rank for traffic. They also provide some tools for webmasters like they have an xml datafeed that you can get which gves you site statistics for all kinds of stuff. They also have a tool where they’ll index your site and provide you a report with broken links to amazon pages, tell you of all 404 errors it finds, and a few other things. It’s not that interesting…except that they say they’ll index your whole site and report on the whole thing. Well, I thought, “ok, lets see if they index my whole site” for my ebay ste, which has about 730,000 pages right now that it could visit. So I turned it loose on my site yesterday.
Within an hour it had hit my site over 1000 times. Then a few hours later it had hit my site over 4300 times. That’s when I started getting worried. It seemed as if they were going to index my whole site yesterday. It worried me because I can only query ebay 30,000 times/month. Then, I really got worried when an hour later they had hit me over 8500 times. That was over 4000 hits that hour alone!!!
Then I started watching it closely but it slowed down right after that and stopped soon thereafter.
It’s an interesting thing to do because you never know what something like alexa indexing your whole site will do for your traffic/google ranking/google indexing of your site. Hopefully it’l increase my traffic. If not, it just took a lot of my bandwidth.