Down memory lane

[jagged]

Hi,
I've been having memory issues with my VPS (again) and keep hitting the black zone with privvmpages. I've tried tweaking httpd.conf and my.conf but every couple of days the problem reappears.

Before I ask for more memory, can anyone check my VPS limits look OK based on these results from Plesk? It should be 768mb Guaranteed Ram and 2048 MB Burstable Ram.

vmguarpages
Soft Limit 196,608
Hard Limit 2,147,483,64
4KB pages
Memory allocation guarantee

privvmpages
Soft Limit 262,144
Hard Limit 292,912
4KB pages
Memory allocation limit

Also can anyone point me towards some info on finding exactly where my problem lies?

My httpd.conf has worker and prefork MPM which confuses things, and the my.conf has innodb settings that I don't know how (or if) to optimise. The settings given in the VPS FAQ don't help much here and the standard settings didn't work at all

Cheers
Jagged

[Frank Broughton]

vmguarpages

196,608 * 4 / 1024 = 768

correct Guaranteed Ram

[jagged]

Thanks, that's what I thought. To be honest I don't think it's a lack of memory as everything ran fine on my old Apollo VPS, but I wanted more speed/disk space (LOL).

privvmpages has gone into the black zone 4 times since yesterday.

I'm going to have a look around the Plesk forum to see if this is down to the standard Jaguar configuration or some issue with the Plesk control panel. Try as I might I can't find a configuration that is stable

Cheers
Jagged

[XoK]

privvmpages

Not sure.. but too many filesystem user names?
(each domain creates a user by default associated with it)

I had a similar problem but I dont remember if it is privvmpages.

[Vin DSL]

vmguarpages

196,608 * 4 / 1024 = 768

correct Guaranteed Ram

HuH?

Does not compute...

[Frank Broughton]

HuH?

Does not compute...

196,608 * 4 / 1024 = 768

okay

196,608 times 4 equals 786,432 divided by 1024 equals 768

It is the formula to find your RAM on a VPS using your vmguarpages settings.

Got it now?

[jagged]

OK, So I thought I had it by gradually pulling resources from mysql until I had OK speed and no blackouts, then today the QoS alerts have been bouncing in and out of the black zone all day despite below average visitors.

Any body know what this means in the server error logs - there are thousands of entries like this and I'm guessing it holds the answer:

[Fri Jan 11 16:24:11 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:24:13 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
Empty input file
[Fri Jan 11 16:24:14 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
Empty input file
[Fri Jan 11 16:26:48 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:26:49 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:26:50 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:26:51 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:26:52 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:26:53 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:26:54 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:26:55 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:26:56 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:26:59 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:27:00 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:27:09 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:27:10 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:27:11 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:28:32 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:28:33 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:28:34 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:28:35 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:28:36 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:28:42 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:28:43 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:28:48 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:28:49 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:28:50 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:28:51 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:00 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:01 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:02 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:03 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:20 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:21 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:34 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:35 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:36 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:37 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:38 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:39 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:40 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:41 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:42 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:43 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:44 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:45 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:46 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:47 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:48 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:49 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:50 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:51 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:52 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:53 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:54 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:55 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:56 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:57 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:58 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:29:59 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:30:00 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:30:05 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:30:06 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:30:07 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:30:08 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:30:09 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:30:10 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:31:08 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:31:09 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:31:10 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:31:11 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:31:12 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:31:13 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/
[Fri Jan 11 16:31:14 2008] [error] [client 127.0.0.1] Directory index forbidden by Options directive: /var/www/html/

[jason]

It means that something in your VPS is trying to hit the homepage of the VPS (http://vpsname.domainname.tld) and that a) a homepage doesn't exist and b) your Apache config prohibits directory browsing in the root.

I'm not sure why you are getting internal hits to your site. It could be a mod_rewrite or cron job gone bad. Since it is happening every second, I'd say something is up, though.

--Jason

[Vin DSL]

196,608 * 4 / 1024 = 768

okay

196,608 times 4 equals 786,432 divided by 1024 equals 768

It is the formula to find your RAM on a VPS using your vmguarpages settings.

Got it now?

Whatever...

Heh!

I see mathematics isn't your thing... probably didn't pass basic math.

Nobody's perfect, Frankie!

[jagged]

I'm not sure why you are getting internal hits to your site. It could be a mod_rewrite or cron job gone bad. Since it is happening every second, I'd say something is up, though

Thanks,
According to Jag support 'These messages should not cause or result of some memory issue'.

This is a shame as since the data centre outage the other day I've been getting between 80-240 QoS alerts a day and I was really hoping it was the cause

I've tweaked SQL and httpd to reduce the load, but still it increases suddenly and the alerts start coming. I wouldn't mind if I'd had an increase in traffic to warrant it!

It means that something in your VPS is trying to hit the homepage of the VPS (http://vpsname.domainname.tld)

Do you have any links giving more info on this as I've little else to go on? FWIW substituting my vpsname in this address returns Server not found

Cheers
Jagged

[jason]

I'm just reading the logs you posted and responding to what I see. I have no links to share because I honestly don't know what is happening.

127.0.0.1 is what they call the local loopback address or localhost. Virtually every computer will respond to that address internally, so if you start seeing that in your logs it means that the requests are coming from the server, not from outside.

Accessing your server by vpsname.domainname.tld will only work if you've set up an A record for your VPS name in your DNS, but if you had, with most configurations, you'd be able to put web content in /var/www/html and serve it up from the VPS address. This is essentially what is happening, but it appears to be happening from inside your server.

Are you, by any chance, running a server monitoring script that checks Apache and reports if it crashes? If so, it looks as though that script is programmed to check http://localhost every second and, since there is no index page there and since Apache is configured not to show its standard directory listing in the absence of a homepage, the error is being generated.

Like support said, it probably isn't that big a deal in the grand scheme of things because nothing is really happening. If you are using a monitoring script it probably doesn't need to check Apache every second, but doing so probably isn't the root of your problems.

If you want to suppress the error message--you'll continue to get the hits as well as entries in your access log (that you didn't post but that I assume are there), but you won't get the messages you posted in your error log anymore--just run this command a s root:

touch /var/www/html/index.html

That will create an empty index.html file in your "root" web directory so that when whatever is causing the errors runs it will get a valid hit. Like I said, whatever is running will still run, but it will stop throwing errors.

--Jason

[Frank Broughton]

Whatever...

Heh!
I see mathematics isn't your thing... probably didn't pass basic math.
Nobody's perfect, Frankie!

Huh, does not compute and I have not been called that since I took Math in school. I took Math all my years of High School and each of my kids here in the homeschool are taking 5 Math classes in the four years of HS.

The Math is correct.

[jagged]

Are you, by any chance, running a server monitoring script that checks Apache and reports if it crashes?

Not unless one was installed on the VPS by default?

If so, it looks as though that script is programmed to check http://localhost every second and, since there is no index page there and since Apache is configured not to show its standard directory listing in the absence of a homepage, the error is being generated.

I've created the index page and will be taking a close look at chron jobs this evening - the only ones I've created (to backup SQL) only run once a day.

I thought I had it after removing a clients catchall email addresses that have been getting 1000+ spam messages a day, but this afternoon things went back to normal

Mon Jan 14 11:57:16 2008 Green zone privvmpages
Mon Jan 14 11:56:56 2008 Yellow zone privvmpages
Mon Jan 14 11:45:35 2008 Green zone privvmpages
Mon Jan 14 11:45:16 2008 Yellow zone privvmpages
Mon Jan 14 11:44:36 2008 Green zone privvmpages
Mon Jan 14 11:44:35 2008 Yellow zone privvmpages
Mon Jan 14 11:44:26 2008 Green zone privvmpages
Mon Jan 14 11:44:16 2008 Yellow zone privvmpages
Mon Jan 14 11:43:56 2008 Green zone privvmpages
Mon Jan 14 11:43:26 2008 Yellow zone privvmpages
Mon Jan 14 11:39:14 2008 Green zone privvmpages
Mon Jan 14 11:36:16 2008 Yellow zone privvmpages
Mon Jan 14 11:31:06 2008 Black zone privvmpages
Mon Jan 14 11:28:16 2008 Yellow zone privvmpages
Mon Jan 14 11:28:06 2008 Green zone privvmpages
Mon Jan 14 11:27:35 2008 Yellow zone privvmpages
Mon Jan 14 11:23:16 2008 Green zone privvmpages
Mon Jan 14 11:22:46 2008 Yellow zone privvmpages
Mon Jan 14 11:21:56 2008 Green zone privvmpages
Mon Jan 14 11:21:36 2008 Yellow zone privvmpages
Mon Jan 14 11:20:06 2008 Green zone privvmpages
Mon Jan 14 11:19:56 2008 Yellow zone privvmpages
Mon Jan 14 11:19:46 2008 Red zone privvmpages
Mon Jan 14 11:18:36 2008 Yellow zone privvmpages
Mon Jan 14 11:18:35 2008 Red zone privvmpages
Mon Jan 14 11:17:16 2008 Yellow zone privvmpages
Mon Jan 14 11:17:06 2008 Green zone privvmpages
Mon Jan 14 11:16:06 2008 Yellow zone privvmpages
Mon Jan 14 11:15:56 2008 Green zone privvmpages
Mon Jan 14 11:15:26 2008 Yellow zone privvmpages
Mon Jan 14 11:14:36 2008 Green zone privvmpages
Mon Jan 14 11:13:46 2008 Yellow zone privvmpages
Mon Jan 14 11:13:26 2008 Green zone privvmpages
Mon Jan 14 11:11:46 2008 Yellow zone privvmpages
Mon Jan 14 11:11:26 2008 Green zone privvmpages
Mon Jan 14 11:11:16 2008 Yellow zone privvmpages
Mon Jan 14 11:10:36 2008 Green zone privvmpages
Mon Jan 14 11:10:35 2008 Yellow zone privvmpages

etc.............

Cheers
Jagged

[Vin DSL]

Huh, does not compute...

The Math is correct.

Let's try this...

Take your pithy response (and sophomoric mathematical analysis):

196,608 * 4 / 1024 = 768

okay

196,608 times 4 equals 786,432 divided by 1024 equals 768

It is the formula to find your RAM on a VPS using your vmguarpages settings.

Got it now?

This explains nothing about the 'soft limit' in vmguarpages.

Any logical person would wonder HOW they came up with the magic number "196,608", no?

A more proper formula would have been:

(768 X 1024 X 1024)/4096 x 4 /1024 = 768

Make more sense now?

[Frank Broughton]

No Vin it does not. The question was about verifying the size of RAM on a VPS and that is done via the formula I gave him.