How to format disks from the command line.

[dos2unix]

This is a basic skill. Anyone should be able to do this.

Note: When doing a new distro install, I recommend letting the installer handle partitioning and formatting the disk.

This is more for systems that already have an OS installed, but want to add or use another disk.

I like to use fdisk to see which disks the OS can see.

fdisk -l

That will return something like this.

Disk /dev/sda: 238.47 GiB, 256060514304 bytes, 500118192 sectors
Disk model: SQF-SM4V2-256G-S
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 34713260-E825-4896-A4D7-27A13153B7A4

Device       Start       End   Sectors   Size Type
/dev/sda1     2048    616447    614400   300M EFI System
/dev/sda2   616448   4810751   4194304     2G Linux filesystem
/dev/sda3  4810752 250068991 245258240 116.9G Linux LVM


Disk /dev/sdb: 489.05 GiB, 525112713216 bytes, 1025610768 sectors
Disk model: Crucial_CT525MX3
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 214CD305-8D57-49C3-A244-B232E66D2141

Device         Start        End   Sectors  Size Type
/dev/sdb1       2048     698367    696320  340M EFI System
/dev/sdb2     698368    1394687    696320  340M EFI System
/dev/sdb3    1394688    7686143   6291456    3G Linux extended boot
/dev/sdb4    7686144  102057983  94371840   45G Linux root (x86-64)
/dev/sdb5  102057984  175458303  73400320   35G Linux variable data
/dev/sdb6  175458304  200624127  25165824   12G Linux swap
/dev/sdb7  200624128 1008027647 807403520  385G Linux home


Disk /dev/nvme0n1: 1.82 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: WD_BLACK SN850X 2000GB
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 09A0425F-2853-45FA-AB17-9D096F9C58E1

Device         Start        End    Sectors  Size Type
/dev/nvme0n1p1  2048 3907028991 3907026944  1.8T Linux filesystem

I have three disks in this computer. /dev/sda, /dev/sdb, and /dev/nvme0n1
As you can see, I have 3 partitions on sda, 7 partitions on /dev/sdb, and 1 partition of /dev/nvme0n1
I can also see the size of each disk, and the manufacturer for each disk.

This makes the disks easy to identify, unless you have two of the exact same disks installed.
Hopefully they are partitioned differently so you have some way of telling them apart.

In most OS's, including Linux, you can format the entire disk, or you can format just a partition.
But even if you use the entire disk as one partition, it's still recommended to create at least one partition,
even if that partition takes up the entire disk.

For the purpose of this thread, I'm not going into more advanced file systems like ZFS, BRTFS, and LVM.
I'm just going to keep it simple with the more standard filesystems. There are multiple tools to do everything here,
and of course in Linux, there are always multiple ways to accomplish the same thing. You can use parted, gparted, gdisk,
and a couple of other tools to accomplish the same thing. But for this tutorial, we will use fdisk.

So now, I have identified /dev/sda as the disk I want to format. How do I do that?
Well first we will create a partition table.

 fdisk /dev/sda

g

w

There are two options for creating a partition table. o and g. o is the old style Microsoft MBR/DOS type. g is the newer gpt type.
(I recommend always using g. There may be a few rare cases where o may be required for backwards compatibility, but I really haven't
seen this. After you press w to "write" the partition tabel to this disk you will see a message similar to this. gpt supports larger disk, more partitions, and larget file systems. As far as I am concerned, there is really no reason not to use use it. You might think, well I only am formatting a 4GB USB drive, I don't need gpt. True, but is there any reason not to use it?

The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

Then it will kick you out of fdisk. So we can use the up arrow key to re-run the same command again.

Now we need to create a partition, this is different from the partition table. The partition table, and the actual partition are two separate things.

 fdisk /dev/sda

Welcome to fdisk (util-linux 2.40.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): n
Partition number (1-128, default 1):
First sector (2048-488397134, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-488397134, default 488396799):

Created a new partition 1 of type 'Linux filesystem' and of size 232.9 GiB.
Partition #1 contains a vfat signature.

Do you want to remove the signature? [Y]es/[N]o: y

The signature will be removed by a write command.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
[CODE]

In the example above I just created one partition to use the entire disk.  But what if I wanted to create two partitions or three or more?
Well, I have to know the size of the disk.  In this case, my disk is 232 GB.  So I will make 3 partitions of about 71GB each, hopefully.

[CODE]
 fdisk /dev/sda

Welcome to fdisk (util-linux 2.40.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): g
Created a new GPT disklabel (GUID: 9D341F0C-79B0-4583-81D3-960CB40A443B).

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

root@dellXPS:~# fdisk /dev/sda

Welcome to fdisk (util-linux 2.40.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): n
Partition number (1-128, default 1):
First sector (2048-488397134, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-488397134, default 488396799): +71G

Created a new partition 1 of type 'Linux filesystem' and of size 71 GiB.

Command (m for help): n
Partition number (2-128, default 2):
First sector (148899840-488397134, default 148899840):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (148899840-488397134, default 488396799): +71G

Created a new partition 2 of type 'Linux filesystem' and of size 71 GiB.

Command (m for help): n
Partition number (3-128, default 3):
First sector (297797632-488397134, default 297797632):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (297797632-488397134, default 488396799):

Created a new partition 3 of type 'Linux filesystem' and of size 90.9 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

Ooops, the last partition was 90GB. What happened here?

Notice I used +71G to determine the size of partition1, and partition2, but on partition3, I just pressed enter, and it automatically
assigned all the rest of the existing disk space to partition3. Which happened to be 90GB. I could go back and delete that partition if I wanted
to by pressing "d" for delete, it would then ask me which partition I want to delete, and I could press 3.

But I will leave it as it for now. Notice I didn't to type in the partition number each time, it automatically knew the next partition number.
By default it automatically created all of these three partitions as a "Linux file system" partition.

However you can over-ride this.

fdisk /dev/sda

Welcome to fdisk (util-linux 2.40.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): t
Partition number (1-3, default 3): 1
Partition type or alias (type L to list all): L
  1 EFI System                     C12A7328-F81F-11D2-BA4B-00A0C93EC93B
  2 MBR partition scheme           024DEE41-33E7-11D3-9D69-0008C781F39F
  3 Intel Fast Flash               D3BFE2DE-3DAF-11DF-BA40-E3A556D89593
  4 BIOS boot                      21686148-6449-6E6F-744E-656564454649
  5 Sony boot partition            F4019732-066E-4E12-8273-346C5641494F
  6 Lenovo boot partition          BFBFAFE7-A34F-448A-9A5B-6213EB736C22
  7 PowerPC PReP boot              9E1A2D38-C612-4316-AA26-8B49521E5A8B
  8 ONIE boot                      7412F7D5-A156-4B13-81DC-867174929325
  9 ONIE config                    D4E6E2CD-4469-46F3-B5CB-1BFF57AFC149
 10 Microsoft reserved             E3C9E316-0B5C-4DB8-817D-F92DF00215AE
 11 Microsoft basic data           EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
 12 Microsoft LDM metadata         5808C8AA-7E8F-42E0-85D2-E1E90434CFB3
 13 Microsoft LDM data             AF9B60A0-1431-4F62-BC68-3311714A69AD
 14 Windows recovery environment   DE94BBA4-06D1-4D40-A16A-BFD50179D6AC
 15 IBM General Parallel Fs        37AFFC90-EF7D-4E96-91C3-2D7AE055B174
 16 Microsoft Storage Spaces       E75CAF8F-F680-4CEE-AFA3-B001E56EFC2D
 17 HP-UX data                     75894C1E-3AEB-11D3-B7C1-7B03A0000000
 18 HP-UX service                  E2A1E728-32E3-11D6-A682-7B03A0000000
 19 Linux swap                     0657FD6D-A4AB-43C4-84E5-0933C84B4F4F
 20 Linux filesystem               0FC63DAF-8483-4772-8E79-3D69D8477DE4
 21 Linux server data              3B8F8425-20E0-4F3B-907F-1A25A76F98E8
 22 Linux root (x86)               44479540-F297-41B2-9AF7-D131D5F0458A
 23 Linux root (x86-64)            4F68BCE3-E8CD-4DB1-96E7-FBCAF984B709
 24 Linux root (Alpha)             6523F8AE-3EB1-4E2A-A05A-18B695AE656F
 25 Linux root (ARC)               D27F46ED-2919-4CB8-BD25-9531F3C16534
 26 Linux root (ARM)               69DAD710-2CE4-4E3C-B16C-21A1D49ABED3
 27 Linux root (ARM-64)            B921B045-1DF0-41C3-AF44-4C6F280D3FAE
 28 Linux root (IA-64)             993D8D3D-F80E-4225-855A-9DAF8ED7EA97
 29 Linux root (LoongArch-64)      77055800-792C-4F94-B39A-98C91B762BB6
 30 Linux root (MIPS-32 LE)        37C58C8A-D913-4156-A25F-48B1B64E07F0
 31 Linux root (MIPS-64 LE)        700BDA43-7A34-4507-B179-EEB93D7A7CA3
 32 Linux root (HPPA/PARISC)       1AACDB3B-5444-4138-BD9E-E5C2239B2346
 33 Linux root (PPC)               1DE3F1EF-FA98-47B5-8DCD-4A860A654D78
 34 Linux root (PPC64)             912ADE1D-A839-4913-8964-A10EEE08FBD2
 35 Linux root (PPC64LE)           C31C45E6-3F39-412E-80FB-4809C4980599
 36 Linux root (RISC-V-32)         60D5A7FE-8E7D-435C-B714-3DD8162144E1
 37 Linux root (RISC-V-64)         72EC70A6-CF74-40E6-BD49-4BDA08E8F224
 38 Linux root (S390)              08A7ACEA-624C-4A20-91E8-6E0FA67D23F9
 39 Linux root (S390X)             5EEAD9A9-FE09-4A1E-A1D7-520D00531306
 40 Linux root (TILE-Gx)           C50CDD70-3862-4CC3-90E1-809A8C93EE2C
 41 Linux reserved                 8DA63339-0007-60C0-C436-083AC8230908
 42 Linux home                     933AC7E1-2EB4-4F13-B844-0E14E2AEF915
 43 Linux RAID                     A19D880F-05FC-4D3B-A006-743F0F84911E
 44 Linux LVM                      E6D6D379-F507-44C2-A23C-238F2A3DF928
 45 Linux variable data            4D21B016-B534-45C2-A9FB-5C16E091FD2D
 46 Linux temporary data           7EC6F557-3BC5-4ACA-B293-16EF5DF639D1
 47 Linux /usr (x86)               75250D76-8CC6-458E-BD66-BD47CC81A812
 48 Linux /usr (x86-64)            8484680C-9521-48C6-9C11-B0720656F69E
 49 Linux /usr (Alpha)             E18CF08C-33EC-4C0D-8246-C6C6FB3DA024
 50 Linux /usr (ARC)               7978A683-6316-4922-BBEE-38BFF5A2FECC
 51 Linux /usr (ARM)               7D0359A3-02B3-4F0A-865C-654403E70625

You press t, to change the type. Then you press L to list all the filesystem types.

Partition type or alias (type L to list all): 11

Changed type of partition 'Linux filesystem' to 'Microsoft basic data'.

So I just changed partition 1 file system type is Microsoft basic data.

Command (m for help): t
Partition number (1-3, default 3): 2
Partition type or alias (type L to list all): 20

Changed type of partition 'Linux filesystem' to 'Linux filesystem'.

Command (m for help):

If I already know the number for the filesystem type in advance, I don't have to list them all, I can just type in the number.
In this case, I created a standard Linux file system.

 

[dos2unix]

But what about partition 3?

Command (m for help): t
Partition number (1-3, default 3):
Partition type or alias (type L to list all): 21

Changed type of partition 'Linux filesystem' to 'Linux server data'.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

There really isn't much difference between the Linux file system, and the Linux Server file system, but I purposely
wanted them to be different, just to show how to do it.

Notice everytime you press w, to write everything to the disk, fdisk automatically kicks you out of fdisk.

So now let us go back in and see what we have.

 fdisk /dev/sda

Welcome to fdisk (util-linux 2.40.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p
Disk /dev/sda: 232.89 GiB, 250059350016 bytes, 488397168 sectors
Disk model: RTL9210B
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 33553920 bytes
Disklabel type: gpt
Disk identifier: 9D341F0C-79B0-4583-81D3-960CB40A443B

Device         Start       End   Sectors  Size Type
/dev/sda1       2048 148899839 148897792   71G Microsoft basic data
/dev/sda2  148899840 297797631 148897792   71G Linux filesystem
/dev/sda3  297797632 488396799 190599168 90.9G Linux server data

So, as expected we have 3 partitions, two are 71GB, and 1 is 90GB.
Well that's great, we have some partitions, but we haven't actually formatted anything yet.
How do we do that?

Command (m for help): q

root@dellXPS:~#

Press q to quit fdisk. Usually the following mkfs command to format partition1 to exfat.

mkfs.exfat /dev/sda1
exfatprogs version : 1.2.5
Creating exFAT filesystem(/dev/sda1, cluster size=131072)

Writing volume boot record: done
Writing backup volume boot record: done
Fat table creation: done
Allocation bitmap creation: done
Upcase table creation: done
Writing root directory entry: done
Synchronizing...

exFAT format complete!

Now we will format partition2 with ext4.

mkfs.ext4 /dev/sda2
mke2fs 1.47.1 (20-May-2024)
Discarding device blocks: done
Creating filesystem with 18612224 4k blocks and 4653056 inodes
Filesystem UUID: ff67396f-0b29-45ec-aa46-ccce1dd3537b
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424

Allocating group tables: done
Writing inode tables: done
Creating journal (131072 blocks): done
Writing superblocks and filesystem accounting information: done

and finally...
partition3 as xfs.

 mkfs.xfs /dev/sda3
meta-data=/dev/sda3              isize=512    agcount=16, agsize=1489056 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=1
         =                       reflink=1    bigtime=1 inobtcount=1 nrext64=1
data     =                       bsize=4096   blocks=23824896, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=25650, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
Discarding blocks...Done.

 

[dos2unix]

What are the pros and cons of one filesystem over another?

I had coPilot write this out for me.


Sure! Here’s an overview of the pros and cons of using ext4, XFS, FAT32, and exFAT on Linux, focusing on aspects like partition sizes, number of partitions, file size limits, and more.

ext4​

Pros:

• Large File and Partition Support: ext4 supports volumes up to 1 exabyte and individual files up to 16 terabytes1.
• Backward Compatibility: It can mount ext3 and ext2 filesystems, making it versatile for upgrades2.
• Performance: Offers good performance for both small and large files, with features like delayed allocation and multiblock allocation1.
• Journaling: Provides journaling for improved reliability and faster recovery after crashes.
• Security: Supports extended attributes and access control lists (ACLs) for enhanced security.

Cons:

• Scalability: While suitable for most use cases, it may not scale as well as XFS for extremely large filesystems.
• No Native Data Deduplication: Unlike some newer filesystems, ext4 does not support data deduplication.

XFS​

Pros:

• High Performance: Optimized for parallel I/O operations, making it ideal for large files and high-performance environments1.
• Scalability: Supports very large filesystems (up to 8 exabytes) and files (up to 8 exabytes).
• Journaling: Provides robust journaling, which helps in quick recovery and data integrity.
• Advanced Features: Includes features like dynamic inode allocation and online defragmentation.

Cons:

• Complexity: Can be more complex to manage compared to ext4.
• Compatibility: Not as widely supported as ext4, especially on older systems.
• Overhead: May have higher overhead for small files and directories.

FAT32​

Pros:

• Compatibility: Highly compatible with many operating systems, including Windows, macOS, and Linux.
• Simplicity: Simple to use and manage, making it a good choice for USB drives and other removable media.

Cons:

• File Size Limit: Maximum file size is limited to 4 GB.
• Partition Size Limit: Maximum partition size is 2 TB.
• No Journaling: Lacks journaling, which can lead to data corruption in case of crashes.
• Security: Does not support advanced security features like ACLs.

exFAT​

Pros:

• Large File and Partition Support: Supports very large files (up to 16 exabytes) and partitions1.
• Compatibility: Widely supported across different operating systems, including Windows, macOS, and Linux2.
• No Overhead: Does not have the 4 GB file size limit of FAT321.

Cons:

• No Journaling: Like FAT32, exFAT does not support journaling2.
• Security: Lacks advanced security features such as ACLs and encryption1.
• Licensing: exFAT is patented by Microsoft, which can lead to licensing issues in some cases2.

Conclusion​

Choosing the right filesystem depends on your specific needs:

• ext4 is a solid all-around choice for most Linux systems, offering a good balance of performance, features, and compatibility.
• XFS is ideal for environments requiring high performance and scalability, especially with large files.
• FAT32 is best for compatibility with older systems and removable media, but its limitations make it less suitable for modern use.
• exFAT is a good choice for large files and cross-platform compatibility, though it lacks some advanced features and security.

I typically do not recommend using NTFS on Linux systems, but sometimes you have no choice.

there are several disadvantages to using NTFS disks in Linux:

1. Limited Write Support: While Linux can read NTFS partitions out of the box, write support is limited without additional drivers like NTFS-3G. Even with NTFS-3G, performance may not be as good as native Linux filesystems.
2. Performance Issues: NTFS is not optimized for Linux, which can lead to slower performance compared to native filesystems like ext4 or XFS. This is especially noticeable with small files and high I/O operations.
3. Compatibility and Features: NTFS lacks support for some Linux-specific features such as symbolic links, hard links, and certain file permissions. This can cause issues with software that relies on these features.
4. Complexity and Maintenance: Managing NTFS partitions on Linux can be more complex. Tools for checking and repairing NTFS filesystems are not as robust or integrated as those for ext4 or XFS.
5. Potential Data Corruption: There is a higher risk of data corruption, especially if the NTFS partition is accessed by both Windows and Linux without proper shutdowns or if Windows’ Fast Startup feature is enabled.
6. No Native Support for Advanced Features: NTFS does not support advanced Linux filesystem features like journaling, which can impact data integrity and recovery.

For these reasons, while NTFS can be useful for dual-boot setups or sharing data between Windows and Linux, it is generally recommended to use native Linux filesystems like ext4 or XFS for better performance and reliability.

 

[dos2unix]

Using a GPT (GUID Partition Table) over the older DOS/MBR (Master Boot Record) partitioning scheme offers several advantages:

1. Support for Larger Drives: GPT can handle drives larger than 2TB, whereas MBR is limited to 2TB.
2. More Partitions: GPT allows for a nearly unlimited number of partitions, limited only by the operating system. In contrast, MBR supports up to four primary partitions.
3. Data Integrity: GPT includes a cyclic redundancy check (CRC) to detect and recover from data corruption, making it more robust.
4. Backup and Recovery: GPT stores multiple copies of the partitioning and boot data across the disk, which helps in recovering data if the primary copy is corrupted.
5. Compatibility with Modern Systems: GPT is designed to work with UEFI (Unified Extensible Firmware Interface), which offers faster boot times and better system management compared to the traditional BIOS used by MBR.

These features make GPT a more reliable and flexible choice for modern computing needs. Do you have a specific use case in mind for switching to GPT?