Problem with zip downloads since server upgrade

[shwn]

Some of my site visitors have been reporting problems with zip file downloads ("file corrupted", "not a valid zip archive") since the server upgrade (I'm on newsaturn). It appears this problem only occurs for those using Internet explorer for the download and possibly either winzip or pkzip to decompress the file.

No problems for those downloading using Firefox, etc. or for those unzipping with 7-zip, windows compressed folders, etc. It seems that adding a .gz to the .zip file extension renders the file usable for winzip/pkzip.

Evidently this has to do with mod_gzip? What can I do? (still Googling...)

[shwn]

If I add the following to my htaccess file it seems to resolve the issue for zip files:

<Files *.zip>
SetEnv no-gzip
</Files>

but not if the file is obtained using the following:

function provideFile($file) {
$path = $GLOBALS['PATH_TO_HIDDEN'] . 'downloads/';
$filename = 'dw_' . $file . '_lic.zip';
$url = $path . $filename;
$this->status = 'no_file';
if ( file_exists($url) ) {
$len = @filesize($url);
$type = "application/zip"; //$type="application/x-zip-compressed"; // mac ie chokes (old)

header("Cache-Control: must-revalidate, post-check=0, pre-check=0, max-age=0");
header("Cache-Control: public", false);
header("Pragma: public");
header("Expires: 0");
header('Last-Modified: '.gmdate('D, d M Y H:i:s') . ' GMT');
header("Content-Type: $type");
header("Content-Length: $len");
header("Content-Disposition: attachment; filename=$filename");
header("Content-Transfer-Encoding: binary");

set_time_limit(0);
$ok = @readfile($url);
$this->status = 'read_er';
if ( $ok ) {
$this->status = 'ok';
return true;
}
}
return false;
}

Any suggestions?

[shwn]

Put the following in htaccess for the relevant directory for php header and readfile:

SetEnv no-gzip dont-vary

Both methods for zip download seem okay now. Just groping in the dark here

[shwn]

I wonder if anyone else has encountered problems with their zip file downloads on shared servers at jag? My guess, if so, they are not likely to be attributing it to a problem on the server.

I have spent considerable time the past month troubleshooting and researching the issue. I just want to share some of the information that led me to conclude that it is due to server configuration so that if anyone else encounters the problem and gets nowhere with support ("I don't know why your script isn't working." "It can't be a server issue if other browsers aren't having a problem." ...) hopefully this can save you some time in crafting your solution.

It took me quite a while to narrow down the problem to users downloading with Internet Explorer and unzipping with either winzip or pkzip. Some links with helpful info on the issue and ways to address it:

• Zip files corrupted on download ...
• Corrupt Zip file when downloaded from Apache with Internet Explorer
• Fix: ZIP file download corrupted in IE ...
• Zip File Download Issue
• Corrupted ZIP Archive

I received the first complaints on corrupted zip files the day of the server upgrade. I figured it was likely related to the upgrade but I assumed that jag would fix the problem so I didn't need to worry about it. Unfortunately, it looks like this is not going to happen.

I think this should be addressed by support for shared server customers. I don't like having to address this in htaccess. It makes me nervous.

[Ron]

Shwn, thanks for sharing this. Did you PM Masood or Rizwan with this info?

Thanks again. It's nice that you're sharing with our community.

[Vin DSL]

If I add the following to my htaccess file it seems to resolve the issue for zip files:

<Files *.zip>
SetEnv no-gzip
</Files>

but not if the file is obtained using the following:

function provideFile($file) {
$path = $GLOBALS['PATH_TO_HIDDEN'] . 'downloads/';
$filename = 'dw_' . $file . '_lic.zip';
$url = $path . $filename;
$this->status = 'no_file';
if ( file_exists($url) ) {
$len = @filesize($url);
$type = "application/zip"; //$type="application/x-zip-compressed"; // mac ie chokes (old)

header("Cache-Control: must-revalidate, post-check=0, pre-check=0, max-age=0");
header("Cache-Control: public", false);
header("Pragma: public");
header("Expires: 0");
header('Last-Modified: '.gmdate('D, d M Y H:i:s') . ' GMT');
header("Content-Type: $type");
header("Content-Length: $len");
header("Content-Disposition: attachment; filename=$filename");
header("Content-Transfer-Encoding: binary");

set_time_limit(0);
$ok = @readfile($url);
$this->status = 'read_er';
if ( $ok ) {
$this->status = 'ok';
return true;
}
}
return false;
}

Any suggestions?

Hrm...

First thing I would do is move the content-type header to the top of the header list, e.g.

 function provideFile($file) {
        $path = $GLOBALS['PATH_TO_HIDDEN'] . 'downloads/';
        $filename = 'dw_' . $file . '_lic.zip';
        $url = $path . $filename;
        $this->status = 'no_file';
        if ( file_exists($url) ) {
            $len = @filesize($url);
            $type = "application/zip";
            
>>>>        header("Content-Type: $type");
            header("Cache-Control: must-revalidate, post-check=0, pre-check=0, max-age=0");
            header("Cache-Control: public", false);
            header("Pragma: public");
            header("Expires: 0");
            header('Last-Modified: '.gmdate('D, d M Y H:i:s') . ' GMT');
            header("Content-Length: $len");
            header("Content-Disposition: attachment; filename=$filename");
            header("Content-Transfer-Encoding: binary");
            
            set_time_limit(0);
            $ok = @readfile($url);
            $this->status = 'read_er';
            if ( $ok ) {
                $this->status = 'ok';
                return true;
            }
        }
        return false;
    } 

Second thing I would do is turn off the compression in my zip files.

Believe it or not, many winders users are still running IE6... LoL!

Regardless, IE has historically had problems decoding 'application/zip' type files that are encoded with gzip.


Lastly, I would use the real thing -- PKZIP.

When I was on regular servers here, I ran zip/unzip (pre-compiled) from my /home directory -- outside the web path, of course.


That's all I can think of, off the top of my head...

[Vin DSL]

Put the following in htaccess for the relevant directory for php header and readfile:

SetEnv no-gzip dont-vary

Both methods for zip download seem okay now. Just groping in the dark here

Eureka!

I should have kept reading. It would have saved me some typing.

Anyway, glad you got it sorted out. What I said still applies...

BTW, don't be afraid of .htaccess -- it's the glue that holds everything together!

[shwn]

... Did you PM Masood or Rizwan with this info? ...

I pointed support to my info links here and they are going to look into it more and pass it up to the next level.

... Lastly, I would use the real thing -- PKZIP...

Doesn't work for pkzip either. The experience is even more confusing for pkzip users since it seems the extraction is working until they view the folder, which only contains one file, with the zip's filename and .out extension.

...BTW, don't be afraid of .htaccess -- it's the glue that holds everything together!

Also the means of bringing everything crashing down

[JPC-NickO]

Hi shwn,

Until we can determine where the problem is we need to know exactly what is happening.

The first thing we need to determine is if the files are actually corrupted. The zip specification has changed over the years and Windows XP compressed folders cannot open the more recent format and you'll get the same error message.

Have a client that is able to reproduce this consistently download something like hashcalc and have them get a md5 for sha512 hash of the downloaded file. Give us the file name and the calculated hash and we can determine if the file is actually corrupted.

[Ron]

I don't understand what difference all that makes if the server shouldn't be trying to compress zip files. Aren't zip files already compressed? Isn't trying to compress them pretty much a waste of everybody's CPU time?

But maybe I'm missing something.

EDIT: One of those links had a good approach to allow gzip on all but zip and bin files, I'm sure the list could(should?) be expanded to include all sorts of file types. (mp3, video, etc. )

SetEnvIfNoCase Request_URI .(?:zip|bin)$ no-gzip dont-vary

I have no knowledge on how apache/ Mod_deflate is normally configured for this (nor do i know how apache 1.x was configured with mod_gzip before the upgrade) but it makes sense to me that certain filetypes wouldn't be compressed.

[shwn]

Hi shwn,

Until we can determine where the problem is we need to know exactly what is happening.

The first thing we need to determine is if the files are actually corrupted. The zip specification has changed over the years and Windows XP compressed folders cannot open the more recent format and you'll get the same error message.

Have a client that is able to reproduce this consistently download something like hashcalc and have them get a md5 for sha512 hash of the downloaded file. Give us the file name and the calculated hash and we can determine if the file is actually corrupted.

I can consistently reproduce the issue. I have installed winzip and pkzip. The problem occurs with those latest-version installs when any zip file on my site is downloaded using Internet Explorer. I have tested with IE7 and IE8. No other browser or decompression method has a problem with the files, including windows compressed folders on windows XP or windows 7. Mac also shows no problems. The files can also be decompressed using 7-zip without a problem.

See ticket # 13341137. My latest reply includes a file name, location and md5.

Note: the htaccess fixes are in place for zips in other parts of my site, so no more of my visitors think I'm a total nitwit.

Doesn't the fact that those solutions resolve the issue (I can consistently reproduce that too) verify that the problem is not corrupted files?

[Vin DSL]

I don't understand what difference all that makes if the server shouldn't be trying to compress zip files. Aren't zip files already compressed? Isn't trying to compress them pretty much a waste of everybody's CPU time?

But maybe I'm missing something. [...]

Maybe this will help (picked at random).

Examples like this abound, all across the web -- been a problem for years.

The ZIP files are used to compress the files and folders so as to encapsulate them into one single entity.

Using this phenomena, you can save enormous amount of disk space. One more advantage that these ZIP files provide that these are extremely handy and light when you have to transfer the data over the Internet. Also, these save you from the risk of virus infections as well. You can extract the data from the ZIP files using an extracting application such as WinZIP. However, when you are unable to unzip the ZIP file, it may be a possibility that the file is corrupt. If you are unable to repair ZIP file, then I would suggest you to use a professional ZIP repair tool.

Considering a ZIP corruption scenario in which you have downloaded a tutorial, which is in the form of a ZIP file from an e-learning website on Internet Explorer 8. After downloading the ZIP file, when you attempt to unzip it, you fail to do so. An error message is displayed in this respect, which is:

"The compressed (zipped) folder is invalid or corrupted."

Cause:
You would receive such error messages if the HTTP (Hyper Text Transfer Protocol) compression has been turned on for the ZIP files on the Web server from where you have downloaded the ZIP file.

What happens is that when HTTP compression is enabled on the Web server, then the ZIP file is encrypted, after which it is sent to Internet Explorer. After getting the HTTP-compressed file, the ZIP file is transferred and downloaded to your system's hard disk. However, Internet Explorer does not decode the HTTP compression from the ZIP file when the unzipping application tries to unzip the ZIP file.

Resolution:
To overcome this problem, you should check if this ZIP file is available on any other website as well. If yes, then you should download the file from that link. The other thing that you can do is to use any other ZIP extracting application to extract the data.

If you are unable to extract the data from the ZIP file, then there is a need to use third-party solutions to repair ZIP file. These ZIP recovery tools are extremely non-destructive that do not overwrite or modify the ZIP files while scanning them.

Read more: http://www.articlesbase.com/data-rec...#ixzz1LktOytrZ
Under Creative Commons License: Attribution

SetEnv no-gzip dont-vary

... in the .htaccess file fixed the problem.

I'm assuming, during the server upgrade, HTTP compression got enabled.

Really it's an IE problem. IE has always been a kludge...

[Vin DSL]

I can consistently reproduce the issue. I have installed winzip and pkzip. The problem occurs with those latest-version installs when any zip file on my site is downloaded using Internet Explorer. I have tested with IE7 and IE8. No other browser or decompression method has a problem with the files, including windows compressed folders on windows XP or windows 7. Mac also shows no problems. The files can also be decompressed using 7-zip without a problem. [...]

You might find this article interesting...

http://perishablepress.com/press/201...ile-downloads/

Download Headers that actually work

After trying hundreds of different headers and combinations, I hit upon a set that works great for ZIP downloads (and other file types as well) in all tested browsers. [...]

Worth a shot, maybe.

[gohighvoltage]

It's not a problem with the server. I had the same problem with my VPS, using Vbulletin. Zip's would download and open ok with firefox, but not with IE (always corrupted). The problem is that the server Gzips the files, and your script is gzipping them. So the double gzip causes the issue.

In my vbulletin, I just added this for my forum attachments (in htaccess) and no more problems:

SetEnvIfNoCase Request_URI attachment.php$ no-gzip dont-vary

[Ron]

If it wasn't happening before the upgrade, it is the server config that's changed that is causing the issue for this user. Apparently the previous configuration wasn't gzipping zip files which on the surface seems to make a lot of sense.