Moin

Da auf weniger berühmten Webservern wie z.B. diesem hier mindestens 90% des Traffics von Robotern irgendwelcher Marketingarschlöcher generiert wird und die Bots auch sehr gerne versuchen, was kaputtzumachen, scheint es mir sinnvoll zu sein, die nervigsten der Dinger mit einem Stinkefinger zu begrüßen und unfreundlich in die Mülltonne zu treten.
Die folgende Liste trägst du einfach in die Konfigurationsdatei deines Apache-Webservers ein. Z.B. bei debianbasierten Systemen in /etc/apache2/sites-enabled/000-default.conf oder wo du es immer hinhaben möchtest, z.B. über ein "Include custom.d/botblock".
Download as file

#Insert this into you apache2-config somewhere

BrowserMatchNoCase "80legs" badbots
BrowserMatchNoCase "A6-Indexer" badbots
BrowserMatchNoCase "ADmantX" badbots
BrowserMatchNoCase "Aboundex" badbots
BrowserMatchNoCase "AboutUsBot" badbots
BrowserMatchNoCase "AcoiRobot" badbots
BrowserMatchNoCase "Acoon\ Robot" badbots
BrowserMatchNoCase "Add\ Catalog" badbots
BrowserMatchNoCase "AhrefsBot" badbots
BrowserMatchNoCase "AlkalineBOT" badbots
BrowserMatchNoCase "AnzwersCrawl" badbots
BrowserMatchNoCase "Apache-HttpClient" badbots
BrowserMatchNoCase "Arachnoidea" badbots
BrowserMatchNoCase "ArchitextSpider" badbots
BrowserMatchNoCase "Autonomy\ Spider" badbots
BrowserMatchNoCase "baidu.sogo.uc.UCBrowser" badbots
BrowserMatchNoCase "Baiduspider" badbots
BrowserMatchNoCase "BecomeBot" badbots
BrowserMatchNoCase "BlackWidow" badbots
BrowserMatchNoCase "Bork-edition" badbots
BrowserMatchNoCase "Bot\ mailto\:craftbot@yahoo.com" badbots
BrowserMatchNoCase "CATExplorador" badbots
BrowserMatchNoCase "CAUpdate" badbots
BrowserMatchNoCase "CareerBot" badbots
BrowserMatchNoCase "Charlotte" badbots
BrowserMatchNoCase "ChinaClaw" badbots
BrowserMatchNoCase "ContextAd" badbots
BrowserMatchNoCase "ConveraCrawler" badbots
BrowserMatchNoCase "Covario" badbots
BrowserMatchNoCase "Custo" badbots
BrowserMatchNoCase "DISCo" badbots
BrowserMatchNoCase "DOC" badbots
BrowserMatchNoCase "DigExt" badbots
BrowserMatchNoCase "DoCoMo" badbots
BrowserMatchNoCase "DotBot" badbots
BrowserMatchNoCase "Download\ Demon" badbots
BrowserMatchNoCase "Download\ Ninja" badbots
BrowserMatchNoCase "EasouSpider" badbots
BrowserMatchNoCase "EirGrabber" badbots
BrowserMatchNoCase "EmailSiphon" badbots
BrowserMatchNoCase "EmailWolf" badbots
BrowserMatchNoCase "Exabot" badbots
BrowserMatchNoCase "Express\ WebPictures" badbots
BrowserMatchNoCase "ExtractorPro" badbots
BrowserMatchNoCase "EyeNetIE" badbots
BrowserMatchNoCase "Ezooms" badbots
BrowserMatchNoCase "Fetch" badbots
BrowserMatchNoCase "Fetch\ API" badbots
BrowserMatchNoCase "FlashGet" badbots
BrowserMatchNoCase "FollowSite\ Bot" badbots
BrowserMatchNoCase "FreeBSD" badbots
BrowserMatchNoCase "GT\:\:WWW" badbots
BrowserMatchNoCase "Gaisbot" badbots
BrowserMatchNoCase "GarlikCrawler" badbots
BrowserMatchNoCase "GetRight" badbots
BrowserMatchNoCase "GetWeb!" badbots
BrowserMatchNoCase "Gigabot" badbots
BrowserMatchNoCase "Go!Zilla" badbots
BrowserMatchNoCase "Go-Ahead-Got-It" badbots
BrowserMatchNoCase "GrabNet" badbots
BrowserMatchNoCase "Grafula" badbots
BrowserMatchNoCase "GrapeshotCrawler" badbots
BrowserMatchNoCase "HMView" badbots
BrowserMatchNoCase "HTTP\:\:Lite" badbots
BrowserMatchNoCase "HTTrack" badbots
BrowserMatchNoCase "IBM\ EVV" badbots
BrowserMatchNoCase "IDBot" badbots
BrowserMatchNoCase "ISC\ Systems\ iRc\ Search\ 2.1" badbots
BrowserMatchNoCase "ImageFetcher" badbots
BrowserMatchNoCase "Image\ Stripper" badbots
BrowserMatchNoCase "Image\ Sucker" badbots
BrowserMatchNoCase "Indy\ Library" badbots
BrowserMatchNoCase "InterGET" badbots
BrowserMatchNoCase "Internet\ Ninja" badbots
BrowserMatchNoCase "IstellaBot" badbots
BrowserMatchNoCase "JOC\ Web\ Spider" badbots
BrowserMatchNoCase "Java\/1.7.0_21" badbots
BrowserMatchNoCase "Java\/1.7.0_45" badbots
BrowserMatchNoCase "JetCar" badbots
BrowserMatchNoCase "LeechFTP" badbots
BrowserMatchNoCase "LinkWalker" badbots
BrowserMatchNoCase "MFC_Tear_Sample" badbots
BrowserMatchNoCase "MIDown\ tool" badbots
BrowserMatchNoCase "MJ12bot" badbots
BrowserMatchNoCase "MOREnet" badbots
BrowserMatchNoCase "MSIECrawler" badbots
BrowserMatchNoCase "Mass\ Downloader" badbots
BrowserMatchNoCase "Microsoft\ URL\ Control" badbots
BrowserMatchNoCase "Missigua" badbots
BrowserMatchNoCase "Missigua\ Locator" badbots
BrowserMatchNoCase "Mister\ PiX" badbots
BrowserMatchNoCase "MojeekBot" badbots
BrowserMatchNoCase "NPBot" badbots
BrowserMatchNoCase "Navroad" badbots
BrowserMatchNoCase "NearSite" badbots
BrowserMatchNoCase "NetAnts" badbots
BrowserMatchNoCase "NetSpider" badbots
BrowserMatchNoCase "NetZIP" badbots
BrowserMatchNoCase "Net\ Vampire" badbots
BrowserMatchNoCase "NextGenSearchBot" badbots
BrowserMatchNoCase "Nutch" badbots
BrowserMatchNoCase "Octopus" badbots
BrowserMatchNoCase "Offline\ Explorer" badbots
BrowserMatchNoCase "Offline\ Explorer" badbots
BrowserMatchNoCase "Offline\ Navigator" badbots
BrowserMatchNoCase "OpenfosBot" badbots
BrowserMatchNoCase "PECL\:\:HTTP" badbots
BrowserMatchNoCase "PHP/" badbots
BrowserMatchNoCase "PHPCrawl" badbots
BrowserMatchNoCase "PageGrabber" badbots
BrowserMatchNoCase "Papa\ Foto" badbots
BrowserMatchNoCase "PaperLiBot" badbots
BrowserMatchNoCase "PetalBot" badbots
BrowserMatchNoCase "PiplBot" badbots
BrowserMatchNoCase "PredictYourBabySearchToolbar" badbots
BrowserMatchNoCase "ReGet" badbots
BrowserMatchNoCase "RealDownload" badbots
BrowserMatchNoCase "Referrer\ Karma" badbots
BrowserMatchNoCase "SEOENGWorldBot" badbots
BrowserMatchNoCase "Scanmine" badbots
BrowserMatchNoCase "ScoutJet" badbots
BrowserMatchNoCase "SearchBot" badbots
BrowserMatchNoCase "SemrushBot" badbots
BrowserMatchNoCase "Servage\ Robot" badbots
BrowserMatchNoCase "SeznamBot" badbots
BrowserMatchNoCase "SiteExplorer" badbots
BrowserMatchNoCase "SiteSnagger" badbots
BrowserMatchNoCase "SiteSnagger" badbots
BrowserMatchNoCase "Slurp" badbots
BrowserMatchNoCase "SmartDownload" badbots
BrowserMatchNoCase "Snoopy" badbots
BrowserMatchNoCase "Sosospider" badbots
BrowserMatchNoCase "Spiceworks" badbots
BrowserMatchNoCase "SuperBot" badbots
BrowserMatchNoCase "SuperHTTP" badbots
BrowserMatchNoCase "SuperPagesUrlVerifyBot" badbots
BrowserMatchNoCase "SuperPagesUrlVerifyBot" badbots
BrowserMatchNoCase "Surfbot" badbots
BrowserMatchNoCase "SurveyBot" badbots
BrowserMatchNoCase "SurveyBot" badbots
BrowserMatchNoCase "Synapse" badbots
BrowserMatchNoCase "Synthomatic" badbots
BrowserMatchNoCase "Tagoobot" badbots
BrowserMatchNoCase "Teleport" badbots
BrowserMatchNoCase "TeleportPro" badbots
BrowserMatchNoCase "Teleport\ Pro" badbots
BrowserMatchNoCase "TweetmemeBot" badbots
BrowserMatchNoCase "TwengaBot" badbots
BrowserMatchNoCase "URI\:\:Fetch" badbots
BrowserMatchNoCase "URLAppendBot" badbots
BrowserMatchNoCase "UbiCrawler" badbots
BrowserMatchNoCase "User-Agent" badbots
BrowserMatchNoCase "Vigil" badbots
BrowserMatchNoCase "VoidEYE" badbots
BrowserMatchNoCase "VoilaBot" badbots
BrowserMatchNoCase "WBSearchBot" badbots
BrowserMatchNoCase "WEP\ Search" badbots
BrowserMatchNoCase "WWWOFFLE" badbots
BrowserMatchNoCase "WebAuto" badbots
BrowserMatchNoCase "WebCopier" badbots
BrowserMatchNoCase "WebCopier" badbots
BrowserMatchNoCase "WebFetch" badbots
BrowserMatchNoCase "WebGo\ IS" badbots
BrowserMatchNoCase "WebLeacher" badbots
BrowserMatchNoCase "WebReaper" badbots
BrowserMatchNoCase "WebSauger" badbots
BrowserMatchNoCase "WebStripper" badbots
BrowserMatchNoCase "WebStripper" badbots
BrowserMatchNoCase "WebWhacker" badbots
BrowserMatchNoCase "WebZIP" badbots
BrowserMatchNoCase "WebZIP" badbots
BrowserMatchNoCase "Web\ Image\ Collector" badbots
BrowserMatchNoCase "Web\ Sucker" badbots
BrowserMatchNoCase "Webmin" badbots
BrowserMatchNoCase "Website\ Quester" badbots
BrowserMatchNoCase "Website\ eXtractor" badbots
BrowserMatchNoCase "Wells\ Search\ II" badbots
BrowserMatchNoCase "Widow" badbots
BrowserMatchNoCase "Xaldon\ WebSpider" badbots
BrowserMatchNoCase "Xenu" badbots
BrowserMatchNoCase "YahooCacheSystem" badbots
BrowserMatchNoCase "YandexBot" badbots
BrowserMatchNoCase "YandexImages" badbots
BrowserMatchNoCase "YesupBot" badbots
BrowserMatchNoCase "YisouSpider" badbots
BrowserMatchNoCase "YodaoBot" badbots
BrowserMatchNoCase "Zao" badbots
BrowserMatchNoCase "Zealbot" badbots
BrowserMatchNoCase "Zeus" badbots
BrowserMatchNoCase "ZumBot" badbots
BrowserMatchNoCase "ZyBORG" badbots
BrowserMatchNoCase "adidxbot" badbots
BrowserMatchNoCase "aihit" badbots
BrowserMatchNoCase "archive" badbots
BrowserMatchNoCase "benderthewebrobot" badbots
BrowserMatchNoCase "bitlybot" badbots
BrowserMatchNoCase "botje" badbots
BrowserMatchNoCase "catchbot" badbots
BrowserMatchNoCase "changedetection" badbots
BrowserMatchNoCase "commoncrawl" badbots
BrowserMatchNoCase "crawler" badbots
BrowserMatchNoCase "data\ mining\ development\ project" badbots
BrowserMatchNoCase "discobot" badbots
BrowserMatchNoCase "discoveryengine" badbots
BrowserMatchNoCase "eCatch" badbots
BrowserMatchNoCase "eurobot" badbots
BrowserMatchNoCase "filterdb" badbots
BrowserMatchNoCase "findfiles" badbots
BrowserMatchNoCase "findlinks" badbots
BrowserMatchNoCase "flightdeckreports" badbots
BrowserMatchNoCase "genieBot" badbots
BrowserMatchNoCase "gigablast" badbots
BrowserMatchNoCase "hailoo" badbots
BrowserMatchNoCase "heritrix" badbots
BrowserMatchNoCase "houxou" badbots
BrowserMatchNoCase "ia_archiver" badbots
BrowserMatchNoCase "ia_archiver" badbots
BrowserMatchNoCase "id-search" badbots
BrowserMatchNoCase "internetmemory" badbots
BrowserMatchNoCase "k2spider" badbots
BrowserMatchNoCase "larbin" badbots
BrowserMatchNoCase "larbin" badbots
BrowserMatchNoCase "leberwurst" badbots
BrowserMatchNoCase "libghttp" badbots
BrowserMatchNoCase "libwww" badbots
BrowserMatchNoCase "libwww-perl" badbots
BrowserMatchNoCase "linkdexbot" badbots
BrowserMatchNoCase "linko" badbots
BrowserMatchNoCase "lwp-trivial" badbots
BrowserMatchNoCase "meanpathbot" badbots
BrowserMatchNoCase "metadatalabs" badbots
BrowserMatchNoCase "msnbot" badbots
BrowserMatchNoCase "musobot" badbots
BrowserMatchNoCase "naver" badbots
BrowserMatchNoCase "niki-bot" badbots
BrowserMatchNoCase "omni-explorer" badbots
BrowserMatchNoCase "panscient" badbots
BrowserMatchNoCase "panscient.com" badbots
BrowserMatchNoCase "pavuk" badbots
BrowserMatchNoCase "pcBrowser" badbots
BrowserMatchNoCase "picsearch" badbots
BrowserMatchNoCase "pipl" badbots
BrowserMatchNoCase "pmoz" badbots
BrowserMatchNoCase "pmoz" badbots
BrowserMatchNoCase "proximic" badbots
BrowserMatchNoCase "psbot" badbots
BrowserMatchNoCase "reverseget" badbots
BrowserMatchNoCase "rogerbot" badbots
BrowserMatchNoCase "seexie" badbots
BrowserMatchNoCase "seoprofiler" badbots
BrowserMatchNoCase "shopwiki" badbots
BrowserMatchNoCase "sindice" badbots
BrowserMatchNoCase "sistrix" badbots
BrowserMatchNoCase "smart.apnoti.com" badbots
BrowserMatchNoCase "spbot" badbots
BrowserMatchNoCase "suggybot" badbots
BrowserMatchNoCase "swebot" badbots
BrowserMatchNoCase "tAkeOut" badbots
BrowserMatchNoCase "twiceler" badbots
BrowserMatchNoCase "uptimerobot" badbots
BrowserMatchNoCase "urllib" badbots
BrowserMatchNoCase "winHTTP" badbots
BrowserMatchNoCase "yBot" badbots
BrowserMatchNoCase "yacybot" badbots
BrowserMatchNoCase "yandex" badbots
BrowserMatchNoCase "yolinkBot" badbots
BrowserMatchNoCase "youdao" badbots
Order Allow,Deny
Allow from ALL
Deny from env=badbots

---