Logs and Grep question

[kiwi li]

I'm wonder what is the command to save one's referrer log to a file, I'm just trying out the CPanel's raw log manager, but I don't know when it does it (everyday?) and what name it would be put under.

My second question is, I would like to use the grep command to delete the referrers that I know that are good and so that would leave me with the questionables that I could look at... however, I'm not too good at that too. ^^

I'm trying to find all referrers that do not have the word "cityof" in the URL and append it to a text file that I could read. ^^

[jason]

The grep command would be something like this:

grep [^(cityof)] originalfile.log >newfile.log

The pattern will match any line of the file that does not contain the text "cityof" in the file "originalfile.log." Normally this would be printed to the screen, but in this case I've redirected output to "newfile.log" so that you can use those results for further analysis.

I haven't tested this and I'm not a regular expression pro by any means, but I think that should for you.

--Jason

[Galen]

If you're wanting to keep a specific log of just http_referers to your site, it might be easier to write your own script for logging that information. I do that and it's MUCH easier to get information from than the cPanel logs. Just write a logging script and use include() to put it in the header of your page (assuming you use PHP for your site instead of html). Here's my logging script if you want it:

<?php 

$ip = getenv(REMOTE_ADDR);
$ref = getenv(HTTP_REFERER);
$ua = getenv(HTTP_USER_AGENT);
$date=date ("l, F d, Y");
$time=date ("H:i T");
$url = $HTTP_SERVER_VARS["HTTP_HOST"] . $HTTP_SERVER_VARS["REQUEST_URI"];

// write to text file
$file=fopen("/path/to/logs.txt", "a");
fwrite($file, "
<b>File Accessed:  <a target=\"_blank\" href=\"http://$url\">$HTTP_SERVER_VARS["REQUEST_URI"]</a>
</b>
<br>
Date/Time:  $date at $time
<br>
IP:  <a target=_blank href=http://www.whois.sc/$ip>$ip</a>
<br>
Agent:  $ua
<br>
Referrer:  <a target=_blank href=$ref>$ref</a>
<hr>
<br>");

?>

I include the .txt file in a page of the admin section of my site so the html gets parsed correctly and viewing my logs couldn't be easier

[gerilya]

grep command has a handy "-v" flag that inverts the match.
For example, use

grep -v string filename.txt

to show all lines in the file filename.txt NOT containing string string.