I had to reconfigure the .htaccess for the APACHE server we run a few days ago.
A few system administrators still dont know how to correctly modify the .htaccess and apply it.
I am including a working template that anybody can use, and it is commented too.
For people who want to download get it from here (view it with notepad) : http://www.mediafire.com/?49lxrdqk542
The .htaccess is also posted below for anyone to read also, and perhaps see some ideas that might be useful for their servers, or even see there are a few things that can be done without cluttering up the robots.txt. I for one perfer to put most things into the .htaccess because nobody else can see it, where with the robots.txt, anyone can see it and also because spammers and spam bots / spammer user-agents usally ignore the robots.txt file
## DISABLE HOTLINKING
RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?PUTyourDomainHERE/.*$ [NC]
RewriteRule \.(gif|jpg|jpeg|png|js|css)$ - [F]
## SITE REFERRER BANNING
RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} info\.biyad\.com [NC]
RewriteRule .* - [F]
## USER IP BANNING
<Limit GET POST>
order allow,deny
deny from 74.54.131.194
allow from all
</Limit>
## AN EXAMPLE OF A REDIRECT
Options +Indexes
IndexOptions -FancyIndexing
Redirect permanent /www.PUTyourDomainHERE/oldPAGE.html http://www.PUTyourDomainHERE/newPage.html
Redirect permanent /www.PUTyourDomainHERE/OLDPage.html http://www.PUTyourDomainHERE/someNEWdirectory/NewPage.html
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP:IGNORE_INCON846393} !^Yes$
RewriteCond %{REQUEST_METHOD} ^(GET|POST)$ [NC]
RewriteCond %{REQUEST_URI}
!.*\.(css|jpg|ai|doc|bin|ogg|mp3|pdf|rtf|ps|ppt|jpeg|png|gif|rar|zip|bz|bz2|gz|tar|iso|chm|z|cue|deb|rpm|ac3|au|snd|mid|midi|wav|bmp|jpe|psd|avi|mpg|mpeg|mpe|mp4|ico|rdf)$ [NC]
## AN EXAMPLE OF A REDIRECT OF DYNAMIC PAGES TO STATIC PAGES
RewriteCond %{query_string} ^option=com_content&task=view&id=23$
RewriteRule ^index.php$ /page1? [R=301,L]
RewriteCond %{query_string} ^option=com_content&task=view&id=22&Itemid=36$
RewriteRule ^index.php$ /page2? [R=301,L]
RewriteCond %{query_string} ^option=com_content&task=view&id=14&Itemid=30$
RewriteRule ^index.php$ /page3? [R=301,L]
RewriteCond %{query_string} ^option=com_content&task=view&id=20&Itemid=34$
RewriteRule ^index.php$ /page4? [R=301,L]
RewriteCond %{query_string} ^option=com_content&task=view&id=13&Itemid=29$
RewriteRule ^index.php$ /page5? [R=301,L]
RewriteCond %{query_string} ^option=com_content&task=view&id=24&Itemid=38$
RewriteRule ^index.php$ /page6? [R=301,L]
RewriteCond %{query_string} ^option=com_sefservicemap&Itemid=37$
RewriteRule ^index.php$ /sitemap.html? [R=301,L]
## AN EXAMPLE OF A PERMANENT REDIRECTs AKA 301
redirect 301 /oldpage.html http://www.yourDOMAIN/newpage.html
redirect 301 /oldpage2.html http://www.yourDOMAIN/newpage2.html
redirect 301 /oldpage3.html http://www.yourDOMAIN/newpage3.html
redirect 301 /oldpage4.html http://www.yourDOMAIN/newpage4.html
## A WORKING EXAMPLE OF BLOCKING BAD BOTS OR USER AGENTS KNOWN TO BE RELATED TO SPAM
SetEnvIfNoCase User-Agent “^EmailSiphon” bad_bot
SetEnvIfNoCase User-Agent “^EmailWolf” bad_bot
SetEnvIfNoCase User-Agent “^ExtractorPro” bad_bot
SetEnvIfNoCase User-Agent “^CherryPicker” bad_bot
SetEnvIfNoCase User-Agent “^NICErsPRO” bad_bot
SetEnvIfNoCase User-Agent “^Teleport” bad_bot
SetEnvIfNoCase User-Agent “^EmailCollector” bad_bot
SetEnvIfNoCase User-Agent “^LinkWalker” bad_bot
SetEnvIfNoCase User-Agent “^Zeus” bad_bot
<Limit GET POST>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>