Blocking Spambots?

[jschodde]

What do you guys do to prevent spambots from spidering your site looking for email addresses? I read that you can add info to your .htaccess file as described in this article:

http://www.clockwatchers.com/robots_bad.html

What do you think?

[pgolovko]

I'd suggest to use online forms to send email messages from user to you. If you want to display your email address, then display it on an image. Masking your email address like this user[AT]host.com will not help as spam bots get smarter and can pick up this kind of emails too. .htaccess file can help only if spam bot identifies it self as one, but if it tells your server that it is a web browser, then .htaccess protection wont work.

[jschodde]

What about the database listings? Are the email addresses "visible" to the bots? I was more concerned about that more than anything.

[GremlinGod]

I posted some information on using the robots.txt file a few weeks ago at this URL: http://www.firewallforums.com/forums/showthread.php?p=8

It may help give you some ideas. I am still going to post a howto to do this with htaccess which actually stops them rather than depend on the robots.txt honor system. I'll post the URL here as soon as I am done. Been pretty busy with eSyndicat mods lately. :D

[Simon Gooffin]

just a note. from my own experience with esyndicat.com

we have all emails listed on our page. Before we had refunds@esyndicat.com email and every day we got few spam emails. But I never got a single spam email to my sgooffin box (fortunately). It seems bots are really smart and sometimes they look for common used emails containing support, webmaster, admin, info, refund, etc. words :)
So try to make an experiment. Create an email address like nfullname@domain.com and check if you get spammed. :)

[pgolovko]

just a note. from my own experience with esyndicat.com

we have all emails listed on our page. Before we had refunds@esyndicat.com email and every day we got few spam emails. But I never got a single spam email to my sgooffin box (fortunately). It seems bots are really smart and sometimes they look for common used emails containing support, webmaster, admin, info, refund, etc. words :)
So try to make an experiment. Create an email address like nfullname@domain.com and check if you get spammed. :)

This gives me an idea for the my Spider engine in Links Wizard 3. It is possible to validate an email address by quering the MX record at the SMTP servers. Your server can validate any registered email address, no matter how funny your email address look like: thisisnotarealaddress@esyndicat.com, lol@esyndicat.com, bgates@microsoft.com, gwb@whitehouse.gov, etc. :crazy-be:

[Simon Gooffin]

offtopic:
I think you can contact Vincent. He has a great experience with mail servers so it's possible he will advise anything.