[INFO] Using .htaccess to block bad bots

[elboriyorker]

Adding the following directives to your .htaccess files allows you to control which bots are denied and allowed to access your forum.

    #   Deny and Allow bots by User-Agent
    SetEnvIfNoCase User-Agent "bot|crawler|fetcher|headlesschrome|inspect|search|spider" bad_bot
    SetEnvIfNoCase User-Agent "duckduckgo|googlebot|yahoo" good_bot
    Deny from env=bad_bot
    Allow from env=good_bot

Adding the following directives to your .htaccess files allows you to control which bots are denied and allowed to access your forum.

View attachment 17429

    #   Deny and Allow bots by User-Agent
    SetEnvIfNoCase User-Agent "bot|crawler|fetcher|headlesschrome|inspect|search|spider" bad_bot
    SetEnvIfNoCase User-Agent "duckduckgo|googlebot|yahoo" good_bot
    Deny from env=bad_bot
    Allow from env=good_bot

If you would like to add additional bad bots, you add them on this line. Keep in mind that by having "bot" already entered, that will cover any bot with the work "bot" in the user agent.

SetEnvIfNoCase User-Agent "bot|crawler|fetcher|headlesschrome|inspect" bad_bot

Just add the | symbol followed by the name of the bad bot.

If you would like to add good bots, you add them on this line.

SetEnvIfNoCase User-Agent "bingbot|duckduckgo|googlebot|yahoo" good_bot

Just add the | symbol followed by the name of the good bot. Note that I prefer to remove the bingbot as I consider it a bad bot.

To deny IP addresses, follow this example:

#   Deny and Allow bots by User-Agent
SetEnvIfNoCase User-Agent "bot|crawler|fetcher|headlesschrome|inspect|search|spider" bad_bot
SetEnvIfNoCase User-Agent "bingbot|duckduckgo|googlebot|yahoo" good_bot
Deny from env=bad_bot
Deny from 47.76.
Allow from env=good_bot

Notice how only the first two octets are used followed by a period. In this example 47.76. will deny the following rage:

47.76.0.0 to 47.76.255.255.