J
Jarret W. Buse
Guest
Applications 34 – XZ Compression – Longer Options Part Two
The “xz” compression utility has longer options which are for more advanced users. Be aware that these parameters do exist. This is the second article on the Longer Options.
The syntax for “xz” usage:
xz [parameters] [files]
The various parameters follow with examples.
The “preset” option allows you to set the Dictionary Size, Compression Speed, Compressor Memory and Decompressor Memory. The default preset is “6”. The presets are as follows:
The “dict” option allows you set the Dictionary Size with a default of 8MB. For larger files to be compressed, larger dictionaries need to be used. The Dictionary stores the compression information needed to uncompress the file.
The “mode” option determines the speed at which the compressed file is created. The “fast” mode operates quickly to compress fast, but loses on a smaller file. The “normal” mode spends time to set up the compress to compress better. The “normal” option is the default.
Match Finder (mf) can effect encoder speed, memory usage, and compression ratio. Hash Chain (hc) match finders are faster than the Binary Tree (bt). The default depends on the preset: 0 uses hc3, 1-3 use hc4, and the rest use bt4. In most cases, “bt4” is the default. The match finders are as follows:
Specify the maximum search depth for the Match Finder (mf). The default is 0. Values can range from 4-100 and 16-1,000+ for Binary Trees (bt). High values slow down the compression, especially for values of 1,000 or more.
The above six options specify the architecture of the system. The option helps the program to align the data to the system instruction set. The alignment of each architecture is:
If any displayed message needs to be in a machine-parsable format, use the “--robot” option. If you wanted to have machine-parsable output for uncompressing the “database.db.xz” file, the command would be “xz -d --robot database.db.xz”.
Keep in mind that these options are less used for the “xz” command. You may not come across these options ever, except in extreme cases when something “special” is required for compressing or decompressing a file. It is best, as always, to be aware of their existence just in case they are needed.
The “xz” compression utility has longer options which are for more advanced users. Be aware that these parameters do exist. This is the second article on the Longer Options.
The syntax for “xz” usage:
xz [parameters] [files]
The various parameters follow with examples.
- --memlimit-compress=LIMIT
- -M (--memlimit=LIMIT) - set memory usage limit for compression, decompression, or both; LIMIT is in bytes, % of RAM or 0 for defaults.
- --no-adjust - if compression settings exceed the memory usage limit,give an error instead of adjusting the settings downwards
- --lzma1[=OPTS] – LZMA1, OPTS is a comma-separated list of presets
- --lzma2[=OPTS] - LZMA2, OPTS is a comma-separated list of presets
- preset=PRE - reset options to a preset (0-9[e])
The “preset” option allows you to set the Dictionary Size, Compression Speed, Compressor Memory and Decompressor Memory. The default preset is “6”. The presets are as follows:
Code:
Preset DictSize CompCPU CompMem DecMem
-0 256 KiB 0 3 MiB 1 MiB
-1 1 MiB 1 9 MiB 2 MiB
-2 2 MiB 2 17 MiB 3 MiB
-3 4 MiB 3 32 MiB 5 MiB
-4 4 MiB 4 48 MiB 5 MiB
-5 8 MiB 5 94 MiB 9 MiB
-6 8 MiB 6 94 MiB 9 MiB
-7 16 MiB 6 186 MiB 17 MiB
-8 32 MiB 6 370 MiB 33 MiB
-9 64 MiB 6 674 MiB 65 MiB
- dict=NUM - dictionary size (4KiB - 1536MiB; 8MiB)
The “dict” option allows you set the Dictionary Size with a default of 8MB. For larger files to be compressed, larger dictionaries need to be used. The Dictionary stores the compression information needed to uncompress the file.
- lc=NUM - number of literal context bits (0-4; 3)
- lc=NUM - number of literal context bits (0-4; 3)
- lp=NUM - number of literal position bits (0-4; 0)
- lp=NUM - number of literal position bits (0-4; 0)
- pb=NUM - number of position bits (0-4; 2)
- pb=NUM - number of position bits (0-4; 2)
- mode=MODE - compression mode (fast, normal; normal)
The “mode” option determines the speed at which the compressed file is created. The “fast” mode operates quickly to compress fast, but loses on a smaller file. The “normal” mode spends time to set up the compress to compress better. The “normal” option is the default.
- nice=NUM - nice length of a match (2-273; 64)
- nice=NUM - nice length of a match (2-273; 64)
- mf=NAME - match finder (hc3, hc4, bt2, bt3, bt4; bt4)
Match Finder (mf) can effect encoder speed, memory usage, and compression ratio. Hash Chain (hc) match finders are faster than the Binary Tree (bt). The default depends on the preset: 0 uses hc3, 1-3 use hc4, and the rest use bt4. In most cases, “bt4” is the default. The match finders are as follows:
- hc3 - Hash Chain with 2- and 3-byte hashing with a minimum value for nice of 3. The memory usage is dict * 7.5 (if dict <= 16 MiB) or dict * 5.5 + 64 MiB (if dict > 16 MiB)
- hc4 - Hash Chain with 2-, 3-, and 4-byte hashing with a minimum value for nice of 4. The memory usage is dict * 7.5 (if dict <= 32 MiB) or dict * 6.5 (if dict > 32 MiB)
- bt2 - Binary Tree with 2-byte hashing with a minimum value for nice of 2. The memory usage is dict * 9.5
- bt3 - Binary Tree with 2- and 3-byte hashing with a minimum value for nice of 3. The memory usage is dict * 11.5 (if dict <= 16 MiB) or dict * 9.5 + 64 MiB (if dict > 16 MiB)
- bt4 - Binary Tree with 2-, 3-, and 4-byte hashing with a minimum value for nice of 4. The memory usage is dict * 11.5 (if dict <= 32 MiB) or dict * 10.5 (if dict > 32 MiB)
- depth=NUM - maximum search depth; 0=automatic (default)
Specify the maximum search depth for the Match Finder (mf). The default is 0. Values can range from 4-100 and 16-1,000+ for Binary Trees (bt). High values slow down the compression, especially for values of 1,000 or more.
- --x86[=OPTS] - x86 BCJ filter (32-bit and 64-bit)
- --powerpc[=OPTS] - PowerPC BCJ filter (big endian only)
- --ia64[=OPTS] - IA-64 (Itanium) BCJ filter
- --arm[=OPTS] - ARM BCJ filter (little endian only)
- --armthumb[=OPTS] - ARM-Thumb BCJ filter (little endian only)
- --sparc[=OPTS] - SPARC BCJ filterValid OPTS for all BCJ filters: start=NUM start offset for conversions (default=0)
The above six options specify the architecture of the system. The option helps the program to align the data to the system instruction set. The alignment of each architecture is:
- x86
- PowerPC
- ARM
- ARM-Thumb
- IA-64
- SPARC
- x86
- --delta[=OPTS] - Delta filter; valid OPTS (valid values; default):dist=NUM where the distance between bytes being subtracted from each other (1-256; 1)
- -Q (--no-warn) – makes any warnings not affect the exit status
- --robot - use machine-parsable messages (useful for scripts)
If any displayed message needs to be in a machine-parsable format, use the “--robot” option. If you wanted to have machine-parsable output for uncompressing the “database.db.xz” file, the command would be “xz -d --robot database.db.xz”.
- --info-memory - display the total amount of RAM and the currently activememory usage limits, and exit
Keep in mind that these options are less used for the “xz” command. You may not come across these options ever, except in extreme cases when something “special” is required for compressing or decompressing a file. It is best, as always, to be aware of their existence just in case they are needed.