Log Scanning

[AncientLady]

I daily download the following logs from my /var/log directory:

bfd_log
exim_rejectlog
messages
secure

Plus /usr/local/apache/logs/error_log.

I then run a perl script that searches for instances where someone has tried to get in using various methods or tries to use my mail server without authority. It also reads in a file that contains previously discovered 'attacks' and IP addresses that have been determined to be 'bad' by this script. It generates an updated attack list (with IP address, entry target type, date, first and last times for that date), updates the bad ip list and creates the base for a new hosts.deny file, with e-mail notification when one of the IPs tries to get back in (this worked well today, a hacker who was caught by the script yesterday tried to get in, was rejected and I got the e-mail notification).

For cases where the source of the attack is named instead of an IP address, the script does an nslookup in an attempt to get the IP address. For ones that it can't, it reports that problem and it is up to me to try to find the IP address.

Ideally, I will get the script to ftp the logs by itself. Then all I would need to do is kick off the script and check the results. I've done automagic FTPing from PHP, but not from perl, yet. This is the first I've needed to.

The next step will be to us the IP address to get the details from one or more whois source and set up e-mails to report abuse. That one will take some more research. Plus, we wouldn't want to automagically send the e-mails. The script should just create the text files including the e-mail address to send to, the abuse warning and the relevant log entries.

Perling can be fun! (I've been doing it for more than 15 years and it just gets better. )

I will gladly share this file, with the proviso that you need to make your own mods to make sure it does what you want, if there is any interest.

Time for some of our VPs fine home-brewed brew.

[jason]

I'd be curious to see the script, even though I'm not running any server/VPS' on which I could use it right now.

My oone question for you, though, is why FTP the logs in the first place? Why not run the tool through a shell session (or even cron) on the server itself?

I'd also suggest doing the download with SCP instead of FTP so that potentially sensitive data in the logs (and your password) isn't sent in the clear.

--Jason

[AncientLady]

I use CuteFTP Pro with a SSH2 connection for my ftp'ing. As far as I can tell, none of the logs I download contain passwords of any sort.

I ftp the files down because it's somewhat easier to do the edit-test-edit cycle locally. It took a while to get it right. It's just easier for me to work locally.

Also, I keep the old logs so I can re-run the script in cases where I discover an error.

I don't want to set it up in a cron job until I'm certain that I've worked out all the wrinkles (like not flagging the IP address here as bad and locking myself out, which happened after I flubbed my password one day and it was caught as an 'attack').

I'm actually just adding the error_log search stuff now. I was only finally able to find the Apache error log when I searched for 'error_log' instead of 'error.log' (which is what it is on all my local systems). Once I'm happy with it's behavior, I'll figure out how to upload the file in this forum.

[Ron]

I'd be wary about banning IPs based on attacks; they are frequently spoofed. The attacker doesn't care about a return packet from the server.

[Vin DSL]

I will gladly share this file, with the proviso that you need to make your own mods to make sure it does what you want, if there is any interest...

I'm always interested in stuff like this... share away!

[Vin DSL]

I'd be wary about banning IPs based on attacks; they are frequently spoofed...

Turks attacked my site, the other night, using a dial-up account in Cali... friggin' UNION MySQL injection crafted, in a rather unique way -- something I've never seen before!

Gotta hand it to 'em! That was a 'good' one!

[AncientLady]

I understand about how a site can be hijacked. I sent some e-mails to report abuse to a few of places yesterday and one responded that the user's site had been hacked and that it had been fixed. One set of responses was in Korean (I think) and I don't know what they were asking.

I know just denying all apparent attempts is not the nice thing to do. I'll probably modify that later. I think a quick fix will be just making my script less sensitive to some things (there is a parameter in the main subroutine call which allows you to decide the number of times the 'error' appears before it is considered an attack). I can also manually edit the resulting file to leave in stuff, or edit hash in the script that allows me to ignore some IPs.

The script is only a few days into it's growth. I just wanted a way to identify probable problems.

A lot of the IPs are coming back (when I check the right whois) as university type places (especially China). Apparently there have been a lot of hack attempts generating out of China lately (according to our VP who also works in the telecoms industry).

I'll be refining my script as we go along. I'll upload the current version now. It produces a new list detailing attempts to use my mailserver for unauthorized relaying.

Comments are welcome and I'll try to answer any questions about the script.

Due to upload limitations, I've tacked '.txt' onto the file name.

[AncientLady]

I've updated the script to allow for IP addresses changing for e-mail forwarding attempts, plus renaming the 'target' when checking the exim_rejectlog.