Moving hosts because of IMAP process issues - Shared, VPS, or something else?

[dhewlette]

I am about to switch hosts for a issue relating to IMAP connections to a shared server. Basically, I have a company site that has about 20 people who all check email via IMAP (over SSL) on both their
personal computer as well as on mobile devices (iphones and ipads primarily). These devices are apparently known for creating IMAP connections that they do not close when done checking email--resulting in many open but unused IMAP processes. This is behavior that can not be changed on the device side. The problem we have with our current host is that, even though these IMAP connections say they use 0 CPU and 0 memory, they eat away at our shared hosting "process limits". I'm told that a VPS eliminates this, but could still be effected in the long run. How true is this statement? Is there some fancy way of dealing with this issue that a VPS allows shared hosting doesn't? How common is this problem?

The process limits of shared servers end up creating Internal server (500)
errors on the site. Not good. We think switching to VPS will eliminate this,but dont want to end up with 1000 IMAP connections that end up slowing our
VPS (or other alternative) down in the long run. We run wordpress for the website itself and its not a huge site (storage or traffic wise). So moving to something other than shared hosting is only necessary if it will solve our IMAP issues (and thus save the website from being blocked by a 500 error). But looking at our options, the additional costs start really adding up (i.e. for cpanel, mysql, IP for SSL, etc.).

Does Jaguar shared hosting separate the mail server from the web server (thus eliminating this issue for a shared plan)? Is moving to the most basic VPS the right move here? Is that overkill for dealing with a simple IMAP issue? Is there a middle ground? We are not web experts, so don't want to move to a VPS and then be stuck with any complex administration tasks a VPS requires over "hands-off" shared hosting. Will this be an issue?

Thanks for any help or advice!

[JPC-Masood]

Answer to your question depends on the traffic on your website. If it is not that large then shared hosting at JaguarPC can handle those 20 IMAP connections. Or you can go with our semi-dedicated hosting:

Semi Dedicated Web Hosting Services

which is basically shared hosting but fewer accounts are setup per server giving everyone more hardware resources to share (cpu, memory). VPS may become overkill if this is just for IMAP issue.

[dhewlette]

Masood, thanks for the quick reply. What are the process limits on shared hosting here? These IMAP connections don't close...so they build up on each other.

For example, one email account may connect at 1pm and that opens a new IMAP process. They connect again at 1:15pm and a new process is created. Then again at 2pm and a third process is opened. At 2pm they are only actually using the 3rd IMAP process, but the other 2 processes are still open. So this issue builds on itself.

I can't imagine this is a issue unique to our company, as we have no custom software or apps or anything out of the ordinary...just standard IMAP connections. Can/will these old IMAP processes be automatically killed off over time?

I assumed that this IMAP process issue would be handled by some sort of software or mail server component, but apparently this is not the case?

Thanks.

[JPC-Zachary]

Masood, thanks for the quick reply. What are the process limits on shared hosting here? These IMAP connections don't close...so they build up on each other.

For example, one email account may connect at 1pm and that opens a new IMAP process. They connect again at 1:15pm and a new process is created. Then again at 2pm and a third process is opened. At 2pm they are only actually using the 3rd IMAP process, but the other 2 processes are still open. So this issue builds on itself.

I can't imagine this is a issue unique to our company, as we have no custom software or apps or anything out of the ordinary...just standard IMAP connections. Can/will these old IMAP processes be automatically killed off over time?

I assumed that this IMAP process issue would be handled by some sort of software or mail server component, but apparently this is not the case?

Thanks.

While Masood may have a technical way to solve it through IMAP on his side and I'm sure he will respond here if he does. Personally, I would put the sites on a VPS anyway. That would allow you to determine when upgrades happen and you have less of a chance of another site affecting your site. The IMAP issue could be solved quickly by setting up a cron job to restart IMAP each morning at like 4:00 am when no one is using it. Then you can sleep well knowing they are not building up.

[JPC-Sabrina]

Dhewlette,

It would be nice if you can post back here when you have made your decision as to the best course of action to resolve this issue. It may be a problem for others as well. Your resolution can set a standard or simply act as a guide line.

[Ron]

I Noticed the *intermittent* persistent IMAP connection from my iPhone, but thought it was just me!

But it's not there all the time. I wonder if Apple is aware of the issue, and if it will go away (fingers crossed) in iOS5 (coming very soon, I'd guess)

[dhewlette]

Apple is definitely aware of the issue. This has apparently been going on for a while (meaning it was in earlier iOS versions). I searched around a good amount on Apple's own support boards as well as online in general and found quite a few threads about it. Its a classic IT situation: device maker says its a service provider issue, and service provider says it is a device issue. In Mac Mail you can explicitly disable the IDLE command which reportedly solves this issue. IOS devices do not have any such settings you can disable.

This is one of those issues that you never think about until it affects you...and it only affected me when we expanded our IT environment to include company issued iOS devices configured for IMAP access to company email. It definitely seems like a sporadic issue, but I can't be sure due to the limited reporting I have on my current shared server. But, this being a business environment, we can't risk our website or email going down due to an artificial process limit.

This is also an issue that doesn't lend itself to easy explanation. Most sales or intro tech support people don't fully understand what a persistent, but essentially empty process (like these IMAP ones are) means, nor how there servers are configured to deal with them. So searching for a new hosting provider that will explicitly support this is difficult to say the least.

[dhewlette]

Also, as a side note, I still can't believe that there isn't an easy solution to this problem. Going by my current provider's TOS, IMAP is essentially useless if you have any more than 2 email addresses. So, while you can create unlimited addresses, you can really only use a couple at a reasonable level via IMAP (POP is different story, but still not immune from these process limits if enough people check email at the same time). There has to be some elegant solution to killing off unused IMAP connections every few hours without disrupting in-use connections (i.e, a blunt "killall IMAP" cron job won't do). These large "email" only providers must have something like this in place...so I must just not know enough of the tech details to search for the right thing script wise.

[Ron]

Probably can find something of use in this script:
Finding the age of a unix process, killing old processes, killing zombie processes - The UNIX and Linux Forums

Looking at the stime you can probably kill procs older than 1 hour. There's probably some risk that one of those processes might be in use at that split second, but if it is I'd guess that the iDevice will just reconnect.

[Ron]

I wonder if Apple is aware of the issue, and if it will go away (fingers crossed) in iOS5 (coming very soon, I'd guess)

I guess we may find out the answer to this soon as iOS5 comes out in 6 days.

Any luck with the script idea?