ZIP (file format)

From Free net encyclopedia

The ZIP file format is a popular data compression and archival format. A ZIP file contains one or more files that have been compressed or stored.

The format was originally designed by Phil Katz for PKZIP. However, many software utilities other than PKZIP itself are now available to create, modify or open ZIP files, notably WinZip, BOMArchiveHelper, PicoZip, Info-ZIP, WinRAR, IZArc and 7-Zip. Microsoft has also included minimal built-in ZIP support (under the name "compressed folders") in later versions of its Windows operating system.

ZIP files generally use the file extensions ".zip" or ".ZIP" and the MIME media type application/zip. Some software uses the ZIP file format as a wrapper for a large number of small items. Generally when this is done a different file extension is used. Examples of this usage are Java JAR files, id Software .pk3/.pk4 files, package files for StepMania and Winamp, XPInstall, and some OpenOffice.org document formats. The OpenDocument format usually uses the JAR file format internally, so it can be easily uncompressed and compressed using tools for ZIP files.

1 Technical information
2 History
- 2.1 Early history
- 2.2 Moving beyond the command line
3 See also
4 External links

[edit]

Technical information

ZIP is a fairly simple archive format that compresses every file separately. Compressing files separately allows for individual files to be retrieved without reading through other data; in theory, it may allow better compression by using different algorithms for different files. However a caveat to this is that archives containing a large number of small files end up significantly larger than if they were compressed as a single file (the classic example of the latter is the common tar.gz archive which consists of a TAR archive compressed using gzip).

The specification for ZIP indicates that files can be stored either uncompressed or using a variety of compression algorithms. However, in practice, ZIP is almost always used with Katz's DEFLATE algorithm.

ZIP supports a simple password based symmetric encryption system which is known to be seriously flawed; being susceptible to known-plaintext attack, dictionary attack and brute force attack. It also supports spreading archives across multiple removable disks (generally floppy disks, but it could also be used with other removable media).

New features including new compression and encryption methods have been added to ZIP in more recent times, but these are not supported by many tools and are not in wide use.

Compression methods

The size for comparison figures were made using the contents of ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.9.tar.bz2 and maximum compression.

Shrinking (method 1)

Shrinking is a variant of LZW with a few minor tweaks. As such it was affected by the LZW patent issue. It was never clear if the patent covered unshrinking but some open source projects (for example Info-ZIP) decided to play it safe and not include unshrinking support in the default builds.

Reducing (methods 2-5)

Reducing involves a combination of compressing repeated byte sequences then applying a probability based encoding to the result.

Imploding (method 6)

Imploding involves compressing repeated byte sequences with a sliding window then compressing the result using multiple Shannon-Fano trees.

Tokenizing (Method 7)

This method number is reserved. The PKWARE specification does not define an algorithm for it.

Deflate and enhanced deflate (methods 8 and 9)

These methods use the well-known deflate algorithm. Deflate allows a window up to 32 KiB. Enhanced deflate allows a window up to 64 KiB. The enhanced version performs slightly better but is not as widely supported.

size for comparison for deflate 52.1 MiB (tested with pkzip for Windows, version 8.00.0038)

size for comparison for enhanced deflate 51.8 MiB (tested with pkzip for Windows, version 8.00.0038)

PKWARE Data Compression Library Imploding (method 10)

The official ZIP format specification gives no further information on this.

size for comparison 61.6 MiB (tested with pkzip for windows Version 8.00.0038 with binary mode selected)

Method 11

This method number is reserved by PKWARE.

Bzip2 (method 12)

This method uses the well-known bzip2 algorithm. This algorithm performs better than deflate but is not widely supported by (Windows-based) tools.

size for comparison 50.6 MiB (tested with pkzip for Windows, version 8.00.0038)

[edit]

History

[edit]

Early history

The ZIP file format was originally created by Phil Katz, founder of PKWARE. Katz publicly released technical documentation on the ZIP file format, along with the first version of his PKZIP archiver, in January 1989.

An earlier compression and archival program, ARC, was distributed not only as the executable software, but also its C source code. Katz had copied ARC and converted the compression routines from C to optimized assembler code, which made it much faster. SEA initially tried to get Katz to license their code used in Katz's archiver, called PKARC, but Katz refused. SEA then sued Katz for copyright infringement and won.

During settlement, Katz still refused to pay license fees to SEA, instead agreeing to pay SEA's legal fees and stop selling PKARC. He then went on to create his own file format, and the ".ZIP" format he designed was a much more efficient compression format than .ARC. Once the PKZIP software was released, many users abandoned .ARC because of its slower performance and because Katz had successfully convinced them that he was the "good guy" who was being unfairly treated by an evil corporation.

The name zip (meaning speed) was suggested by Katz's friend Robert Mahoney. They wanted to imply that their product would be faster than ARC and other compression formats of the time. It is frequently named in all capitals as ZIP for historical reasons, because filenames in DOS were usually represented in all uppercase (due to the non-case-preserving nature of the FAT filesystem that DOS used).

[edit]

Moving beyond the command line

In the mid 1990s, as more new computers included graphical user interfaces, there were more users who were not comfortable with the command-line operation of PKZIP. Seeing an opportunity, shareware authors began pitching compression and archival programs with graphical user interfaces. Many of these used the ZIP format. WinZip was among the most popular. PKWare (Katz's company) also offered a graphical version of PKZip. These graphical compression programs were easier to learn to use than the older command-line equivalents, but they still required learning an additional program and an additional interface just for compression.

In the late 1990s, various file manager software products started integrating support for the ZIP format into the file manager user interface. Even before that, Norton Commander and clones like Volkov Commander in DOS started that trend, and that remains the norm for the "Commander-like" or Orthodox file managers like Midnight Commander (Linux and UNIX like systems) and Total Commander, previously Windows Commander (Windows). The KDE file manager (kfm) supported this very early, and support was also added to Windows Explorer first with Plus! for Windows 98 and later included with Windows Me and Windows XP, the Mac OS Finder (as of Mac OS X, via the BOMArchiveHelper utility), the Nautilus file manager used with GNOME, the Konqueror file manager used with newer versions of KDE, and others. By 2002, all major desktop environments included ZIP file support in their file managers. Typically, in any modern file manager, a ZIP file may be treated as a directory or folder, so that files are copied into and out of it in the same manner as any other folder; the compression is handled in a way that is largely transparent to the end user. This eliminates the need for the user to learn to use a program and an interface just for the purpose of compression and archival, since the same interface can be used as for regular file management.