Create A Forum - Support Forums

I'm curious to know how Web Crawlers, Spiders, and Bots work. More often than not, when checking to see who is online, Guest (Spiders) will be attempting to post in existing threads. Is this a glitch in the software as I can't really think there are that many dumb people out there that would continually try to do this if they haven't yet registered on the site.


I also get a lot of people "Registering to the forum", but never get any email notification. And when I check in Admin and look in the Waiting for Approval, there isn't any there to approve.


Just odd.

Well the bots click on every link they see. They also record all text on the page, so if there's the word "mario" a hundred times on a page and I go to google and type in "mario" then I can expect your forum to be in between pages 1 - 30 on google because the bot can expect there'll be more information on mario which would make your forum more relevant.

Well the bot doesn't have common sense, cause it's a bot.... If it sees a link, it goes there. They might be stuck at that link because they can't get past it.

They might be smart enough to all pile up on a website and overload the server

They automatically come to your forum. No need to activate them whatsoever

They'll be listed under guests. If you see a random guest every now and then, it's quite possible they are bots. You just have to check their IPs.

I can't confirm if this list is valid but this is a list of known ip addresses for Google bots: http://www.iplists.com/google.txt

Thanks.