Rewrite acting flakey

[Eric Pauker]

Is there a problem with the rewrite engine on utopia? I am using it to make sure no one tries to hot link to any images on my sites, but every once in a while my own sites are blocked from getting their own images. Here is what my .htaccess file looks like:


RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://ericpauker.net/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://ericpauker.net/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https://www.ericpauker.net/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https://www.ericpauker.net/.*$ [NC]
RewriteRule .*\.(jpg|jpeg|gif|png|bmp)$ - [F,NC]


Have I got something wrong in there? I don't see any reason for it to work most of the time and not work sometimes. I've noticed that the problem happens more often with a page generated by PHP or Perl, but it still doesn't always happen, only once in a while.

[lookout]

The code you've posted has some flaws, which would account for some of the flakeyness. Try these tweaks, mostly having to do with slashes that may or may not be there, and not fully checking whether or not a www is present in the URL.

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https://ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https://www.ericpauker.net.*$ [NC]
RewriteRule \.(jpg|jpeg|gif|png|bmp)$ - [F,NC]

You may also want to allow empty referers thru like so (same as above, just with a new second line):

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https://ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https://www.ericpauker.net.*$ [NC]
RewriteRule \.(jpg|jpeg|gif|png|bmp)$ - [F,NC]

This is NOT the only way to do this. Just tweaks what you already have in place.

[G.Bloke]

You may also want to allow empty referers thru like so (same as above, just with a new second line):

Actually very important as some programs dubiously block referer data ...

[Eric Pauker]

Thanks for the help. I'll try it out and hopefully from now on I'll always see the images on my site.

[Eric Pauker]

OK, I've finally had a chance to test it out and have discovered that having the line
RewriteCond %{HTTP_REFERER} !^$
is allowing anyone to link to my images from any site. If I remove that line then no one can any longer link to images on my site... BUT... web pages created with PHP and Perl scripts on my site still SOMETIMES, not always, won't display the images.
Anyone got any ideas about whats happening?

[lookout]

That line allows empty referrers thru. Some browsers and several popular firewalls do not send the referrer with their requests by default. If you choose to not include that line, you will certainly be reducing the opportunities for hotlinking, but you will also be blocking large numbers of perfectly legitimate visitors (those using such browsers, or behind such firewalls) from viewing your images.

This really comes down to how much you feel you must lock things down on your site.

[lookout]

BTW, either approach will reduce hotlinking and bandwidth stealing. One is simply more aggressive in its approach than the other. Neither is very effective at protecting your images from those who might copy them and post them elsewhere.

If you include the line, hotlinking will still be possible but not that reliable. However, it won't affect your everyday visitor experience as much as the other method. If you don't include that line, your hotlink blocking efforts will be more reliable, but they will be at the cost of driving some of your own visitors away.

[lookout]

Not sure if I was entirely clear.

That line will NOT let everyone thru, just those who aren't sending referer information with their URL requests (lots of people like that). In testing, be sure your own firewall and browser aren't interfering with the passage of the referer info.

[G.Bloke]

Also make sure your CLEAR the cache between tests. Often images are cached making it seem as though the htacess isn't working in blocking hot-linking.

[lookout]

Yes, good point.

[Connie]

Can a domain be blocked from hot linking? My main problem is with
forums that are sex oriented hot linking. I would rather block those
specific sites rather than blocking all hot linking in general as I do now.

[lookout]

I haven't tested this code snippet myself, but this might be one approach to clssam's question. This example redirects the offenders to the site's home page, assuming it's called index.html. You could redirect them wherever you like, of course.

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://www.forbidden-domain.com.* [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://forbidden-domain.com.* [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://www.another-forbidden-domain.com.* [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://another-forbidden-domain.com.* [NC]
RewriteRule ^.*http://www.mydomain.com/index.html [R,L]

Note the OR between conditions, except the last one. Also note the lack of the ! that appeared in the previous examples, used to reverse the match condition.

Be aware that this approach depends entirely on being able to process the referer info in the first place, something you can't be guaranteed will be passed on with the URL request.

[Eric Pauker]

Originally posted by lookout
The code you've posted has some flaws, which would account for some of the flakeyness. Try these tweaks, mostly having to do with slashes that may or may not be there, and not fully checking whether or not a www is present in the URL.

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https://ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https://www.ericpauker.net.*$ [NC]
RewriteRule \.(jpg|jpeg|gif|png|bmp)$ - [F,NC]

You may also want to allow empty referers thru like so (same as above, just with a new second line):

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https://ericpauker.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https://www.ericpauker.net.*$ [NC]
RewriteRule \.(jpg|jpeg|gif|png|bmp)$ - [F,NC]

This is NOT the only way to do this. Just tweaks what you already have in place.

OK, I just finished some more testing and discovered that if the slashes are left off as you suggested then ANYONE can link to my images in their web page. I've tested this on a bunch of different computers in different locations using a couple of websites that weren't mine, but had pages with links to images on my site and they could all use my images. If I put the slashes back in, then they couldn't.
I also talked with a professor at the local university who teaches a web server installation and maintainance course and he said the line allowing a blank referer through would always let everyone through, thus defeating trying to stop hot-linking. As far as pages generated by PHP and Perl sometimes working and sometimes not, he had no ideas. He said he would need to see the main server configuration files first.

[G.Bloke]

Sorry to disagree with your professor but that emtpy refer line is essential and DOESN'T let everyone through. I've used it for years and I think you'll find that most examples available on the internet are identical. Without that line you block a percentage of legitimate visitors from seeing your images.

I could set up a demonstration of my htacess if you really want ... it works flawlessly and except for the last line, it is identical to the above.

[G.Bloke]

As I dug it out for another post might as well post it here:

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://.*.yoururl.com.*$ [NC]
RewriteRule .*\.*$ http://yoururl.com/stolen.gif [R]