Using mod_rewrite to control access

This document supplements the mod_rewrite reference documentation. It describes how you can use mod_rewrite to control access to various resources, and other related techniques. This includes many examples of common uses of mod_rewrite, including detailed descriptions of how each works.

Forbidding Image "Hotlinking"

Description:

The following technique forbids the practice of other sites including your images inline in their pages. This practice is often referred to as "hotlinking", and results in your bandwidth being used to serve content for someone else's site.

Solution:

This technique relies on the value of the HTTP_REFERER variable, which is optional. As such, it's possible for some people to circumvent this limitation. However, most users will experience the failed request, which should, over time, result in the image being removed from that other site.

There are several ways that you can handle this situation.

In this first example, we simply deny the request, if it didn't initiate from a page on our site. For the purpose of this example, we assume that our site is www.example.com.

RewriteCond "%{HTTP_REFERER}" "!^$"
RewriteCond "%{HTTP_REFERER}" "!www.example.com" [NC]
RewriteRule "\.(gif|jpg|png)$"    "-"   [F,NC]

In this second example, instead of failing the request, we display an alternate image instead.

RewriteCond "%{HTTP_REFERER}" "!^$"
RewriteCond "%{HTTP_REFERER}" "!www.example.com" [NC]
RewriteRule "\.(gif|jpg|png)$"    "/images/go-away.png"   [R,NC]

In the third example, we redirect the request to an image on some other site.

RewriteCond "%{HTTP_REFERER}" "!^$"
RewriteCond "%{HTTP_REFERER}" "!www.example.com" [NC]
RewriteRule "\.(gif|jpg|png)$" "http://other.example.com/image.gif"   [R,NC]

Of these techniques, the last two tend to be the most effective in getting people to stop hotlinking your images, because they will simply not see the image that they expected to see.

Discussion:

If all you wish to do is deny access to the resource, rather than redirecting that request elsewhere, this can be accomplished without the use of mod_rewrite:

SetEnvIf Referer "example\.com" localreferer
<FilesMatch "\.(jpg|png|gif)$">
    Require env localreferer
</FilesMatch>

Blocking of Robots

Description:

In this recipe, we discuss how to block persistent requests from a particular robot, or user agent.

The standard for robot exclusion defines a file, /robots.txt that specifies those portions of your website where you wish to exclude robots. However, some robots do not honor these files.

Note that there are methods of accomplishing this which do not use mod_rewrite. Note also that any technique that relies on the clients USER_AGENT string can be circumvented very easily, since that string can be changed.

Solution:

We use a ruleset that specifies the directory to be protected, and the client USER_AGENT that identifies the malicious or persistent robot.

In this example, we are blocking a robot called NameOfBadRobot from a location /secret/files. You may also specify an IP address range, if you are trying to block that user agent only from the particular source.

RewriteCond "%{HTTP_USER_AGENT}"   "^NameOfBadRobot"
RewriteCond "%{REMOTE_ADDR}"       "=123\.45\.67\.[8-9]"
RewriteRule "^/secret/files/"   "-"   [F]

Discussion:

Rather than using mod_rewrite for this, you can accomplish the same end using alternate means, as illustrated here:

SetEnvIfNoCase User-Agent "^NameOfBadRobot" goaway
<Location "/secret/files">
    <RequireAll>
        Require all granted
        Require not env goaway
    </RequireAll>
</Location>

As noted above, this technique is trivial to circumvent, by simply modifying the USER_AGENT request header. If you are experiencing a sustained attack, you should consider blocking it at a higher level, such as at your firewall.

Denying Hosts in a Blacklist

Description:

We wish to maintain a blacklist of hosts, rather like hosts.deny, and have those hosts blocked from accessing our server.

Solution:

RewriteEngine on
RewriteMap    hosts-deny  "txt:/path/to/hosts.deny"
RewriteCond   "${hosts-deny:%{REMOTE_ADDR}|NOT-FOUND}" "!=NOT-FOUND" [OR]
RewriteCond   "${hosts-deny:%{REMOTE_HOST}|NOT-FOUND}" "!=NOT-FOUND"
RewriteRule   "^"  "-"  [F]

##
## hosts.deny
##
## ATTENTION! This is a map, not a list, even when we treat it as such.
## mod_rewrite parses it for key/value pairs, so at least a
## dummy value "-" must be present for each entry.
##

193.102.180.41 -
bsdti1.sdm.de -
192.76.162.40 -

Discussion:

The second RewriteCond assumes that you have HostNameLookups turned on, so that client IP addresses will be resolved. If that's not the case, you should drop the second RewriteCond, and drop the [OR] flag from the first RewriteCond.

Referer-based Deflector

Description:

Redirect requests based on the Referer from which the request came, with different targets per Referer.

Solution:

The following ruleset uses a map file to associate each Referer with a redirection target.

RewriteMap  deflector "txt:/path/to/deflector.map"

RewriteCond "%{HTTP_REFERER}" !=""
RewriteCond "${deflector:%{HTTP_REFERER}}" "=-"
RewriteRule "^" "%{HTTP_REFERER}" [R,L]

RewriteCond "%{HTTP_REFERER}" !=""
RewriteCond "${deflector:%{HTTP_REFERER}|NOT-FOUND}" "!=NOT-FOUND"
RewriteRule "^" "${deflector:%{HTTP_REFERER}}" [R,L]

The map file lists redirection targets for each referer, or, if we just wish to redirect back to where they came from, a "-" is placed in the map:

##
##  deflector.map
##

http://badguys.example.com/bad/index.html    -
http://badguys.example.com/bad/index2.html   -
http://badguys.example.com/bad/index3.html   http://somewhere.example.com/