VPS problems

[gaucho]

I'm having some sort of problems in last 2-3 weeks, but in last 2 days it's completely down...

I had one big IT forum, but I suspended it, and the same problem again, I posted ticket but I think tech. sup is more confused than me...

ticket:
2009-02-06/00:28:41
hi, last few hours (5-10h) my vps was "dead", only place where I could connect to was vzpp and in it "System usage" was 99-100%
numtcpsock - yellow zone
kmemsize - black zone

no services was working and nothing couldn't start
also few weeks I get these errors or even vps restart email, but I managed somehow explain it to my customers, but now I don't now what's going on, I limited few stuff and managed to get it stable:

root@vps [/home/**user**]# ulimit -a
...
max memory size (kbytes, -m) 700000
...
virtual memory (kbytes, -v) 400000

&

RLimitMEM 33554432

but now when I try to connect to some site e.g. jeftinhost.net it shows 500 error

I'm exhausted fixing that vps and explaining customers what's going on, I need a good 20h sleep
so please help me to finish this suffer
thx
2009-02-06/00:52:51
A customer service representative has begun working on this issue.

--- Tech --- : M.Asif. 2009-02-06/02:34:44
Hi,
I have checked and found that your server is hitting the Qos parameters for memory so you need to upgrade the RAM for your server on emergency basis, Pleas update so we assist you further in this issue.

let us know for further assistance.

Regards,
M.Asif
Technical Support | JaguarPC

+++ Client +++ : ***** 2009-02-06/04:20:41
Yes, I know that, but there are no any process that could do that, only one big IT forum that I currently disabled
try now, now it's working again, and again 500 error


- - - SYSTEM - - - 2009-02-06/05:02:56
A customer service representative has begun working on this issue.

--- Tech --- : M.Asif. 2009-02-06/05:45:40
Hi,
I have checked and you were using the php values in .htaccess for /home/******/public_html/email/.htaccess which is not allowed to use, but to use in php.ini. I have removed the values from the .htaccess and the issue is resolved.

But the domain is again giving the Internal server error and now this is because of short of memory on the server as the logs show, So now you need to get more RAM in order to fix this issue.

let us know for further assistance.

Regards,
M.Asif
Technical Support | JaguarPC

+++ Client +++ : ******* 2009-02-06/10:03:45
but let's say now:
in vzpp:
CPU Usage
4.0%
Load Average 0.23, 0.32, 0.15

System Usage
Resource Capacity
30.56%

in ssh:
custom vmem command:
Limit mem. .....: 768 MB (VPS shared: 2048 MB)
now used...........: 120 MB (15%) (requested: 536 MB)
free ..: 648 MB (85%)
-----------
httpd status:

root@vps [~]# service httpd status

Apache Server Status for localhost

Server Version: Apache/2.2.9 (Unix) mod_ssl/2.2.9 OpenSSL/0.9.8b
mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635

Server Built: Aug 12 2008 12:16:46
________________________________________ _________________________

Current Time: Friday, 06-Feb-2009 16:00:45 CET
Restart Time: Friday, 06-Feb-2009 15:57:15 CET
Parent Server Generation: 0
Server uptime: 3 minutes 29 seconds
Total accesses: 239 - Total Traffic: 211 kB
CPU Usage: u.02 s.07 cu0 cs0 - .0431% CPU load
1.14 requests/sec - 1033 B/second - 904 B/request
44 requests currently being processed, 12 idle workers

R___RRRCRR_RRRRKRRRRRRRR_RRRRRR_RRR_R_R_ RRRRRRR.KR_RW_R_R.......
........................................ ........................
........................................ ........................
........................................ ........................

Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process
-------

so can you say me what's wrong with this, I don't see memory problem, I also found few problems like mine on your forum, but nothing is explained, users just write that tech. stuff raised QoS limit...

- - - SYSTEM - - - 2009-02-06/10:13:03
A customer service representative has begun working on this issue.

+++ Client +++ : ************* 2009-02-06/10:39:23
and also in vzpp system processes is about 100+ same processes with command:
/usr/local/apache/bin/httpd -k start -DSSL

and now memory usage 96MB; 672MB free,
Load Average 0.2, 0.25, 0.16

and
System Usage
Resource
The resource is in "Black Zone"
20.54%

WHY black zone when 20% used, and cpu load is low???

----
30secs later:

System Usage
Resource
The resource is in "Green Zone"
20.54%

with same %

+++ Client +++ : ************ 2009-02-06/10:41:09
also in vzpp system services shows that httpd is down but I connect through ssh and # service httpd status shows everything right it should be

+++ Client +++ : ******** 2009-02-06/10:48:30
There is some misconfiguration of apache of php...
because only php sites (and that's 99% of server pages) are error 500; but file listing is ok:
http://www.kolekcija.org/dernek/
and also shoutcast is ok:
http://yupunk.info:8000/ (worked also when resources were in black zone and all other services down)

-----------

I found few more topics here and on whtalk considering jaguarpc, but there wasn't any useful information

so please help me with this, because now again "black zone" apears, and there are ~200 processes runing (100+ again of that httpd mentioned above) and of them all only one process (shoutcast) is using 3,5% and max memory use is spamd with 0,2%, and two others; everithing together 0,5% memory and max 5% cpu

[thisisit3]

First of all, level 1 support are "not very clever" and will always respond with something along the lines "you have reached your hosting limits, you need to upgrade", just ignore them and do your own investigation first.

Now, what you need to do is check what kind of limits your VPS is hitting, as we have already mentioned in other VPS related threads, the default VPS plans are too limited but support is superb and they will always raise any limits not related to memory.

So, first give us the output of this command:

cat /proc/user_beancounters

then run the following command (but to protect the privacy of your users, don't post the results)

netstat -ntu | awk '{print $5}' | tail --lines=+3 | cut -s -d: -f1 | sort | uniq -c | sort -n -b

The first command will show us all the limits and we'll be able to detect what causes the "black zone" effect.

The second command will give you a list of IP addresses and the number of connections they have to your server, that should give you an indication of a DDoS attack or something similar. Don't post the list, just check if there are IP addresses with more than 80-100 hits...

then we can give you some more things to run

[thisisit3]

Why is this thread moderated and the above post reported as spam? its definitely not spam.

[JPC-Masood]

Why is this thread moderated and the above post reported as spam? its definitely not spam.

too much junk characters in the post marked it as spam automatically.

[gaucho]

ssh is down at the time, so I cannot check that, but I got response from your tech support manager
--- Tech --- : Rizwan.Z 2009-02-06/12:03:14
Hi,

We are investigating into this and will update you shortly on our findings on this , along with details of what exact parameters are hitting their limits.

Regards,
Rizwan Z.

--- Tech --- : Rizwan.Z 2009-02-06/12:17:08
Hi,

There was a dos on your vps from IP '201.143.224.*' , it has been blocked in the firewall , and we are checking it further.

Few errors indicated a possible corruption of libraries , but we can confirm after investigation further.


Regards,
Rizwan Z.

-------

but I instaled tons of ddos protections, and other stuff; I don't even remember everything