Standard Compression Scheme for Unicode
From Free net encyclopedia
| Unicode |
|---|
| Encodings |
| UCS |
| Mapping |
| Bi-directional text |
| BOM |
| Han unification |
| Unicode and HTML |
| Unicode and e-mail |
The Standard Compression Scheme for Unicode (SCSU) is a Unicode Technical Standard for reducing the number of bytes needed to represent Unicode text, especially if that text uses mostly characters from one or a small number of per-language character blocks. It does so by dynamically mapping values in the range 128–255 to offsets within particular blocks of 128 characters. Since most alphabets do reside in blocks of contiguous Unicode codepoints, the scheme allows for 1 byte per character (plus overhead) encoding for many text files. SCSU can also switch to UTF-16 internally to handle non-alphabetic languages.
SCSU has not been a resounding success. Few applications need to compress so much Unicode text that it's worth using a special-purpose compression scheme which (so far) does not have widespread support. Treated purely as a compression algorithm, SCSU is inferior to most commonly-used general-purpose algorithms for texts of over a few kilobytes. It can be used as a text encoding, but it can be difficult to handle internally, and the percentage savings of SCSU versus UTF-16 or UTF-8 drops after external compression, dramatically in the case of bzip2 and other modern compression schemes. SCSU does have the advantage that it can usefully compress texts that are only a few characters long, whereas most full-scale compressors need a few kilobytes of data to break even against their own overhead.
Reuters, the organization that floated the first draft of SCSU, is believed to use SCSU internally.