Tail Packing

Discussion in 'Filesystem' started by Jarret W. Buse, Jul 9, 2013.

  1. Jarret W. Buse

    Jarret W. Buse Well-Known Member Staff Writer

    Jul 3, 2013
    Likes Received:
    Tail Packing (Tail Merging)

    Tail Packing can sometimes be referred to as Tail Merging or Block Suballocation.

    In the article on Extents, there were details about how space is wasted because of whole blocks not filled with data. For example, if each block is 512 bytes and a small file is placed on the hard disk that is 100 bytes, then 412 bytes are wasted. It should be remembered the block is the smallest addressable unit on a hard disk. A block is comprised of sectors, but the sectors are not individually addressable.

    NOTE: Block sizes should always be a multiple of 512 bytes. Sectors are 512 bytes and each block is a specific number of sectors.

    To correct the problem of wasted space, some file systems support tail packing. Tail packing is an ability to take the “tail” of a file that does not fill a whole block and combine them so they do fill or nearly fill a whole block.

    NOTE: Another term, “clusters” may also come into discussions of hard disks. Usually, clusters and blocks are the same. They are the smallest unit addressable (as in having a disk address) by the file system.

    To find the block size of your hard disk, use the following command:

    sudo tune2fs -l /dev/sda1 | grep Block
    You should get an output similar to the following:

    Block count:              17919921
    Block size:              4096
    Blocks per group:        32768
    As you can see, the block size is 4kb (4,096 bytes). Use this information to compare to the examples which follow in the article.

    Files, when written to the hard disk, must start a new block. On a file system with no tail packing, the disk has a limited file number equal to the block count. Of course, this only occurs if the file size is equal to or less than the block size.

    When a file is too small to fill a whole block, the block can be allocated into sections of 512 byte portions. With a block being 4,096 bytes, the block size can be:

    • 2 – 2,048 byte portions
    • 4 – 1,024 byte portions
    • 8 – 512 byte portions
    In large hard disks, block sizes can be as large as 64kb (65,536 bytes) and can be divided into the following:
    • 2 – 32,768 byte portions
    • 4 – 16,384 byte portions
    • 8 – 8,192 byte portions
    • 16 – 4,096 byte portions
    • 32 – 2,048 byte portions
    • 64 – 1,024 byte portions
    • 128 – 512 byte portions
    It may be wondered what the difference is between the various block sizes. The hard disk reads one block at a time. If a file is on the hard disk that is 100 MB and the block size is 4 kb, then it would take 250,000 reads to get the data from the hard disk. If the block size is 64 kb, the read count would only be 15,625 reads. With larger files you can see the advantages of using a larger block size. With smaller files, you gain speed, but have wasted space.

    Each suballocation block, or sector, is set at 512 bytes (at a minimum). Technically the largest wasted space when using tail packing is 511 bytes. The worst case scenario is when a block is filled and only a byte remains. The single byte will be tail packed into a 512 byte sector. Since only one byte is written, it leaves 511 bytes as wasted.

    File systems can reserve sections of blocks to use for tail packing. Each file system reserves a specific amount depending on the hard disk size. An example is found by dividing the block size by 512 then subtracting one. So for a hard disk with 4kb blocks, the equation would be: (4096/512) - 1. In this case there would be seven reserved blocks. After the seven are filled, more can be created as needed, one at a time.

    To determine which files will be tail blocked is another issue. Since 511 is the most wasted space possible, files that are within 511 bytes of filling a block are not tail packed. Let’s use a hard disk with a block size of 4kb (4,096 bytes). The file size is divided by the block size and the remainder is what we are concerned with. If the remainder is less than 512, the tail is not packed. If the remainder is 512 or more, then the tail is packed.

    As files shrink and grow, tails can be removed from the tail packed blocks and placed into a standard block. When this occurs, the tail blocks can be repacked to remove “blank” areas left when a tail is removed.

    Similar to Extents, suballocated portions of files are only created when new files are written. When a file system is upgraded to a newer version that supports tail packing, only new files can take advantage of tail packing. It is usually best to perform a full backup before performing a file system upgrade. The disk should be formatted, wiping all data, and then restored. New files will be written with no fragmentation and file tails will be packed as needed.

    NOTE: The suballocation tables are stored in memory,so when using tail packing, more memory is required to track disk use for the file tails. If needed, add more memory if the system does not have enough. Be aware that memory will be utilized when tail packing is enabled.
    DevynCJohnson likes this.

Share This Page