Cobra

[DaveL]

Looks like /home2 is full.

Ticket has been raised on this.

Monitoring?

DaveL

[IDB]

Just raised a ticket on this as well.

[Cobra] has been one of the best systems for a long time until recently - have many new or exisitng accounts suddenly been transferred to [Cobra] or is there some other problem we should be aware of.

[IDB]

I can access my site on Cobra, but the MySQL databases are unavailable.

[DaveL]

I've been watching the state of /home2 recently and noted it creeping up steadily towards the mid 80% area. It looks like a whole load of stuff has been dumped on it to fill it up so quickly. Let's hope they can muck something out and get the show back on the road - quickly!

DaveL

[jkeller]

My guess is that there has been a recent influx, but "/home2" has been pretty full for a long time. I got the impression that its user base was static; my guess is that there's someone (or more than one) who's recently expanded their use of their account(s).

I agree that it's been frustrating to see the downtimes come to cobra. I'm on there, but I haven't been affected by the db issues because I don't have any db things up yet. IDB or DaveL: would you be able to post a link to something of yours that uses the db?

(edit: I retract this bit; see below)
I'm not able to mod this area. This thread isn't a Suggestion or Feedback, but General Support (bordering on Downtimie Issue, due to the db downtime). If another mod sees this, could they move it over to the appropraite section, depending on the ability to see if db access is down via a sample link? Thanks...

[BitStar]

http://64.119.172.149/cgi-bin/ib3/ikonboard.cgi

This is one of the errors we have been seeing since about four hours ago

[jkeller]

Thanks, BitStar.

My brain wasn't screwed in straight when I wrote that -- it's something I've been writing for all db outages lately. Then I remembered that, because I'm on cobra, I could confirm myself throught the control panel.

Duly confirmed.

I've grouped and arranged threads in "Downtime Issues" on this cobra outage. For all those users coming to this thread, please post there (I don't have the ability to do anything in the "Suggestions & Feedback" forum, so I can't move/lock this thread).

[DaveL]

Hi John. As a 'newby' to Aletia forums, I wasn't too sure where to post as, at first, many of the posts seem to be about downtime issues across several forums. I haven't taken much interest in the Aletia forums until the ordure passed through the personal fan.

I think I now have it worked out for next time!

I've been watching /home2 filling progressively on Cobra's stats and thought that if growth proceeded in linear fashion, it had a week or two left. Maybe someone moved something to Cobra and got the calculations wrong, because it grew and blew very quickly.

[IDB]

I can confirm the database is accessible now, but the \home2 partition remains 100% full. I'm now reluctant to do anything with the databse in case it fails because of the space problems. It would be nice to see some comment on what they are doing about the space problem. Until is rectified the site is effecitely dead as I can't update it.

[jkeller]

Originally posted by DaveL
Hi John. As a 'newby' to Aletia forums, I wasn't too sure where to post as, at first, many of the posts seem to be about downtime issues across several forums.

That's ok. That's why the mods are in charge of moving threads around. I just don't have that ability outside a limited area.

I generally try and make comments for future guidance, not to chastise for current accidents. There's always a learning curve, and it's good to have another active member -- even if under unfortunate circumstances.

I've been watching /home2 filling progressively on Cobra's stats and thought that if growth proceeded in linear fashion, it had a week or two left. Maybe someone moved something to Cobra and got the calculations wrong, because it grew and blew very quickly.

Yeah, I'd noticed it a while back when it was around 80+% full. When I'd made a comment about how cobra seemed to be the only one with a "/home2" (at the time, at least), I learned that it was where some "refugees" from another had been moved to.

So, I figured it was only ever meant for that group, since there's still comparatively so much room left on "/home". That's where my guess (and only a guess, not as studied as your observations) led to thinking it was sudden growth within that limited group.

[jkeller]

Originally posted by IDB
I can confirm the database is accessible now, but the \home2 partition remains 100% full. I'm now reluctant to do anything with the databse in case it fails because of the space problems.

My understanding is that databases exist on the "/var" partitions, but I could be wrong. I know I've seen some machines with only a "/backup" and no "/var".

But the result is effectively the same, as you say. If the server's having problems due to space, it could still upset any db updates. Sorry you've been caught in the grinder.

It would be nice to see some comment on what they are doing about the space problem.

I agree: I would like to see more frequent and more regular postings on systems status. Recognition of the problem, attempted solutions, and final success/closure note. Each no more than one or two sentences, but 3 x 5 minutes would go far in helping people feel that there's a safety net in place.

[jkeller]

...And with that, I retract my request to mods to move this thread. I think the one already in "Downtime Issues" is doing a good job, and that this thread is serving a better purpose by remaining in this forum as is.

[DaveL]

Agreed, John. I think most users would accept a couple more minutes downtime during a long outage to be informed of progress during repairs. It does engender both confidence and understanding. A frustrated user, punching holes in the proverbial fog, can easily allow emotive comment to boil over onto any perceived outlet, and I'm sure that if the techies see it, they feel hurt, especially if their effort has been of the monumental variety. Silence hurts everyone in the end.

Most of us can identify with the situation of telephoning the hospital to enquire the status of someone dear, so most of us will have been told 'as well as can be expected' and the frustration that accompanies such a meaningless statement.

Here, we're just told to hold the line and hold and hold and hold .........

[Sandman]

I'm on cobra, but how do I find the server status page (with disk stats etc) on the new control panel, i thought it had gone?

[jkeller]

Originally posted by Sandman
I'm on cobra, but how do I find the server status page (with disk stats etc) on the new control panel, i thought it had gone?

The link does appear to have disappeared (an oversight, I think, along with some other rough edges that I've noticed).

You can, however, just copy the machine name and paste it into your address bar. As a rule, machines at Aletia are named [animal].nocdns.com, where [animal] is the shorthand machine name.

I've bookmarked my machine status page, since it's not necessary to log in to the control panel to see it. I find myself looking at the status more often than changing parameters, anyway.