Processor Not Keeping Up w/ Frontpage

[bbextra]

My new VPS w/ jaguar (welovebigbrother.com) on the Freedom Plan is working great. However, it seems that when I use certain tasks in Frotpage, it puts the processor at more than 100%. I know Frontpage isn't the "way to go," but that is how this site was built.

When I use FP to update an "includes" page that is included on about 50 pages or so, it kills the processor...sometimes i even have to reboot the VPS. Should this be the case? The normal processor load says averages at about 16%. Should this task really push the VPS to a hault?

I had a shared server with the same website and had no problems.

Thanks!

[bbextra]

By the way, after I restart the VPS and try to update my site with Frontpage, it usually works the second time.

[Rebel007]

I have to say it sounds like your site is very badly optimised, I use CPanel to back up my sites on a very similar setup to you, and have none of these problems.

I have never "killed" the vps to the extent it needed rebooting and never reached 100% loading, so quite what your site does is frightening.

In your position I would be very concerned, simply because if your site is doing all of this you must be affecting other users, and once JPC starts getting complaints and others leaving because of a single site....well the conclusions are fairly obvious.

[Gwaihir]

He's on a VPS, thus he won't be affecting other users. He's on a VPS, which is in many ways a small server in itself and because of that small size easier to overload.

I dunno about FP; stopped using it eyons ago. All I have is a wild guess: isn't there a setting in there somewhere that limits the number of actions it will take simultaneously (often labeled threads)? I.e. is it currently hitting your server with 50 or so requests all at once when it tries that update, or does it make one, wait for the server's reply, do the next one, etc.? Are you using the special frontpage extensions for access via HTTP, or do you use its FTP way? I've allways thought of the latter as more robust.

[Ron]

Wim,
Even thoguh he's on a VPS and gets CPU Memory and Bandwidth guarantees, he's still on a physical server that allows bursts.

The physical server might have to swap in his process and swap in his memory (or clear out someone else's bursting memory to disk). The physical server's disk drives might be overworked, so he's getting a ton of Wait I/O. CPU and bandwidth SHOULD be transparent though. So as he might be affected by other users, he also could be affecting other users.

The question is when his CPU "goes to 100%" is it in "user" or "sys" or is it in "wait"? Is it burning too much "sys" as it swaps in memory? Things need to be delved into detail before it can be said that it's his own VE that is the root of the problem. It could be a Virtuozzo or server problem.

[bbextra]

Hey thanks for the replies. I guess I should clear some things up.

The website is updated from within the Frotpage software (not FTP.) I know this isn't the best solution, but it allows multiple users from different locations to login at different times and change stuff.

The VPS doesn't "quit" working when the CPU goes to 100%, but it is very slow. If I kill the frontpage process, everything is fine again.

I have used the Optimizing VPS thread on JagPC forums before to optimize my VPS.

I will admit that I'm new to server admin-ing. I have had a shared server w/ JagPC since forever and wanted to expirament w/ the VPS/dedicated environment.

Let me know what other info I can give you guys!

Thanks much!

[Ron]

SSH into your VPS and use "top" to see what is going on.
These are the important lines to post (top 5 lines from the output):

top - 11:59:54 up 10 days,  4:01,  1 user,  load average: 3.26, 2.23, 2.03
Tasks: 188 total,   1 running, 184 sleeping,   0 stopped,   3 zombie
Cpu(s): 23.3% us,  2.7% sy,  0.0% ni, 58.6% id, 15.1% wa,  0.2% hi,  0.0% si
Mem:   3960540k total,  3593512k used,   367028k free,   271936k buffers
Swap:  4096532k total,      144k used,  4096388k free,  1933644k cached

[Ron]

I hope top reports on your VPS or on your server and not a combination of the two... something is "ringing a bell" with me that there may be an issue with top and VPSes. But I cant' recall what it is, if it was anything.

[Gwaihir]

So as he might be affected by other users, he also could be affecting other users.

Sure there is a burstable and in that sense shared component to it. However there are clear guarantees to the minimums and maximums one is entitled to and to my knowledge the Virtuozzo protects these quite well.

What bbextra just said about that confirms this:

The VPS doesn't "quit" working when the CPU goes to 100%, but it is very slow. If I kill the frontpage process, everything is fine again.

The VPS slows down because IT is at its limits. In the shared environment this load would simply have come from the big pie and might have affected other users at that point in time. In the VPS environment the virtualization software shields the rest of the machine from this local stress, assigning resources for it only up to a certain level that's acceptable for the machine as a whole.

So, bbextra, all you have to consider is whether you can afford to load the VPS that much during those updates, which depends on how much of a problem you think it is for your own users.


Only in the true shared environment one can claim an unfair part of a server's resources and get a just response from JagPC about such. One doesn't have to feel bad towards other users about putting some strain on a VPS, especially not so occasionally. I've seen a claim made before (perhaps that was Rebel007 too?) that one should restrain VPS use out of some social behaviour and think it's BS. Actually frightening off fellow users by suggesting that JagPC would come by and slap one over the fingers is IMHO over the top and prompted me to set the record straight.


top.. hmm.. IIRC that can't handle very well that max CPU and memory are not fixed enitities. I don't think there is a way to get such data from inside the VPS. Isn't the nice thing of Virtuozzo over OpenVZ that it has a nice panel to report such things to you?

[Ron]

"The VPS slows down because IT is at its limits."

That's not very good logic...
There are no disk I/O guarantees, it may be trying like heck and competing the best it can with other VEs.

Also, FP may be exceeding the max RAM, so his VE is paging like crazy. If his VE is paging like crazy, his VE can consume a large percentage of the server's I/O capacity, competing with everyone and affecting everyone.

You can't make statements like that without knowing more about what is going on.

[Spathiphyllum]

Is I/O capacity capped in a VPS? I mean, how isolated is the VPS from the core regulating it, and wouldn't the perfect multi-server manager cap everything so that the whole isn't taken down by the part... any part?

[Ron]

I don't know

[Gwaihir]

Are you sure about that, Ron? I think the max ram is a very hard limit which already includes whatever swap is made available (which is managed machine wide). For the precise reason you give, it would be very akward otherwise; many a VPS could start swapping in and out of its own private swap disk like mad, with serious inpact on machine wide performance.

As for I/O: there are limits on diskspace, number of inodes, number of open files and a few more, but the main cap on I/O capacity seems to be a more general one: there's a cap on the total number of process and threads. So, they could be pretty much all I/O processes, but there is a cap. I know I'm venturing on very thin ice here, but I dare assume that in most situations this works out fine enough, or they (VZ) would have build a more specific cap into their software by now.

http://download.openvz.org/doc/OpenVZ-Users-Guide.pdf
http://my.thomas-krenn.com/upload/do...ourcesMgmt.pdf
Memory management is on p. 57 of the first document, a list of all manageable resource paramenters on p. 8 of the second.

BTW: please do remind me of this in a few months as I expect to be stress testing some applications on a VPS before the year is out. We're writing some things for which we expect heavy use and we'll want to know where the bottlenecks are. Our testing environment for this is an OpenVZ VPS on a local server that runs a few other development related VPSses as well. (OpenVZ is the open version of Virtuozzo, it mainly lacks GUI management tools.) I've got no specific reason to assume it will end up I/O bound, but nevertheless, I'll have some first hand experience to share on stressing a VPS up to and beyond its limits, as well as how that affects the rest of the machine.

[Ron]

OK, did you read (with comprehension) your cited documents, or just skim through them?

cpuunits
This is a positive integer number that determines the minimal guaranteed share of the CPU time the corresponding Virtual Private Server will receive.

cpulimit
This is a positive number indicating the CPU time in per cent the corresponding VPS is not allowed to exceed.

Source:http://download.openvz.org/doc/OpenVZ-Users-Guide.pdf page 54
What if you set cpuunits to N and cpulimit to 100? Do you get to burst to 100%? Yeeessssssss....

vmguarpages
The memory allocation guarantee, in pages (one page is 4 Kb). VPS applications are guaranteed to be able to allocate additional memory so long as the amount of memory accounted as privvmpages (see the auxiliary parameters) does not exceed the configured barrier of the vmguarpages parameter. Above the barrier, additional memory allocation is not guaranteed and may fail in case of overall memory shortage.

"Above the barrier, additional memory allocation is not guaranteed" means that it can get as much memory as is available.
Source:http://download.openvz.org/doc/OpenVZ-Users-Guide.pdf page 55

Another possible attack is the attack on the disk bandwidth, attempting to leave as little bandwidth to other [VEs] as possible. This attack is possible theoretically, but is difficult to be conducted efficiently. [...]In any case, the other [VEs] and the host system will continue to function under this attack (although, with bad responsiveness of the applications).Future Virtuzzo versions will provide full automatic protection from disk bandwidth attacks, too

Source:http://my.thomas-krenn.com/upload/do...ourcesMgmt.pdf page 4
Sooooo, understanding what is REALLY being said, doing some things can bring the entire physical server to it's knees (as long as you don't exceed the number of files and/or the number of inodes, etc.).

Do you think it's possible or probable that frontpage is exceeding memory limitations on his VE? (LOL ok, that's a little dig against fp)

I dunno either. That's why I've said now X number of times: We need to see what is happening on machine when his problem occurs in order to accurately determine what the problem is, i.e. what resource is the bottleneck.

(PS I'd like to see how they implement disk usage quotas given my "superficial" understanding of UNIX systems architecture and management.)

Anyway, there ya go.

[Gwaihir]

OK, did you read (with comprehension) your cited documents, or just skim through them?

Lol; of those documents I didn't read everything (as in every page). I did study those that seemed relevant to this matter at first glance. Same as you did just now, I trust? And then actually a nice bit more (not relevant now), as I've been dealing with this VPS stuff a bit for work over the past two weeks.

Guess we're just looking upon it from different sides. I believe one of the main reasons behind the VPS idea is to shield users on a machine from eachothers actions, make it impossible for a user to take more than he pays for / is entitled to. You're saying that system almost certainly isn't perfect. I'm saying it's almost certainly pretty decent. So.. what are we arguing about, really? Or do you really think one must worry about overloading the machine doing fairly ordinairy stuffs before having received any solid feedback that one indeed is?

What if you set cpuunits to N and cpulimit to 100? Do you get to burst to 100%? Yeeessssssss....

Yeah, ... so?
A) You'd only get a burst to 100% from one VPS if no other VPSes are claiming a share. Each other VPS first gets its guaranteed share, only once each has gotten that they start sharing what's left.
B) If you don't want that much bursting, then don't set the limit that high. According to specs Jag doesn't..

Those are guarantees a shared environment doesn't have. That's a lot of pretty effective protection users get to make sure no fellow VPS takes an unfair share that would cripple a user's VPS, right? Especially given all that, I just don't see how occasionally allowing ones VPS to compete for all it can get is supposed to be unfair. We're talking about site updates here, something very incidental, not some process that's 24/7 hogging the server for all it can squeeze out. If someone were to put that on, then we should perhaps be having this discussion.

"Above the barrier, additional memory allocation is not guaranteed" means that it can get as much memory as is available.

If the memory is sitting idle, why not? At the same time it means that when memory is not in abundance, the VPS that's over its max limits is the first that gets abruptly denied its wishes. That's exactly what you want but were afraid wouldn't happen, isn't it?

doing some things can bring the entire physical server to it's knees

As an attack, which to me means deliberate sabotage, possible theoretically and even then difficult to be conducted efficiently. To me that reads as I guessed before: it seems to work out fine in real world scenarios, no point in scaring our TS with such stuff.


Yes, it would be cool if one could study what happens in detail. Then again, if FP is to stay, what difference does it make? The TS is just asking if he's doing anything wrong and / or perhaps could do something different short of abandoning FP. We've been over that now, no?

A) He's prolly doing nothing wrong. FP simply taxes the server that hard and the small size of the VPS makes load visible that wasn't visible on the big server. AFAIK Frontpage extensions have always been a pain on a server because they do a big scramble when FP starts up to see what changed since last time. Does it matter? Well, that's for him to decide based on how much it inconveniences the primary victims of the slow down, his own users.

B) Can he do something differently? Perhaps. There's bound to be some settings in FrontPage that ease it up to some extent, using FTP instead of the HTTP extentions being one possibility that *might* help. So just experiment and see what works for you.

Anyway, there ya go.

Oh.. is that perhaps a reference to the PM I sent you, Ron? That was about a very different thread (see link)..