Solved Safe remove drive vs sync command?

[CaffeineAddict]

When I copy some data to my external HDD I always use "Safely remove" which changes the text on screenshot below to:
"Please wait, data is still in transfer" and this takes a few seconds until the drive is removed, that is, unmounted.

But my question is, can I instead run sync command and expect this will be equivalent to safe removal?
I know sync command won't unmount the drive, but other than this is that the same?
If I simply unmount the drive manually after running sync, is that the same as "Safely remove"?

 

[atanere]

"Please wait, data is still in transfer" and this takes a few seconds until the drive is removed

Sync will complete that data transfer, and it can take many minutes after a big transfer, like after burning a Linux ISO to a flash drive. The terminal will not release the cursor back to you until the sync is complete. Then you can simply unmount or remove, and it will happen instantly because there is no more data to transfer. Completing the data transfer is what causes the delay with the safely remove command.

If one simply unmounts and pulls out their USB stick too quickly... they may lose data. This is one of the key points that I teach to friends/family when they first begin using Linux.

 

[CaffeineAddict]

Thanks, therefore sync followed by unmount is same as Safe remove, I'll mark this as solved then.

Subquestions about sync:
I just did man sync and there is option -f or --file-system, how is that option different than running sync without any options?

When you run sync, I suppose it synces all drives?

 

[atanere]

When you run sync, I suppose it synces all drives?

I suppose that to be true also. Though I'd guess it likely there is usually only one that needs it done most times you would run it. Sync may be a good command to add to the end of some bash scripts... I want to look into that with a rsync backup script I use.

I just did man sync and there is option -f or --file-system, how is that option different than running sync without any options?

A good question, and I don't know the answer. There are only two options for sync (-d and -f)... and the man page also does not say which of these is the default action. It only hints that -f will include metadata not included with -d.

 

[CaffeineAddict]

I suppose that to be true also. Though I'd guess it likely there is usually only one that needs it done most times you would run it.

I just googled out this and found out this:

Linux sync Command Explained with 7 Real Life Examples

Synchronize your cache data to the permanent memory with sync. Read this tutorial to learn how to use the Linux sync command with examples.

phoenixnap.com

sync run as regular user synces "current" filesystem, but it's not clear which one is current, while sudo sync synces all mounted filesystems.

The -f option is also not well explained there, and some other blogs say -f "forces" sync which is silly lol, sadly not many resources about this.

 

[new_vintage]

Sync may be a good command to add to the end of some bash scripts... I want to look into that with a rsync backup script I use.

I always add a couple lines at the end of any rsync backup script:

sync
echo Please wait for sync...
echo Finished.

Required for USB but useful, too, for platter spinners. Backup is pretty much all I use spinning media for these days.

 

[atanere]

I always add a couple lines at the end of any rsync backup script:

sync
echo Please wait for sync...
echo Finished.

Required for USB but useful, too, for platter spinners. Backup is pretty much all I use spinning media for these days.

Cool.

I only just added sync, so my "finish text" was much more verbose (and written for other people besides myself). I need to update it since sync works very well.

sync

echo ""
tput smso
echo "If no errors are shown above, the backup should be successful.        "
echo "Or there was nothing new added or updated that needed to be backed up."
echo "UNMOUNT, or EJECT, or SAFELY REMOVE DRIVE before unplugging the USB.  "
echo ""
echo "  WARNING!  "
echo "It may take a VERY LONG TIME to eject the drive after a backup. WAIT FOR IT!"
echo "It will eventually finish writing data to the USB and be safe to remove.    "
echo "You will corrupt your backup if you don't allow the write process to finish."
tput rmso
read -n 1 -r -s -p $'\nPress any key to exit...\n'

 

[new_vintage]

Cool.

Yeah, except the order of those lines should be:

echo Please wait for sync....
sync
echo Finished.

of course.

 

[Trenix25]

Sync is not the same. Unmounting a file system writes data to the administrative blocks. Sync only flushes data from the system buffer. You should sync and umount before removing a drive. Doing otherwise can lead to file system corruption. That "Safely Remove" option might be doing that background work for you. This is why I prefer the command line. Then I know for sure what's happening.

Signed,

Matthew Campbell

 

[bob466]

I always use either...Eject or Safely Remove Drive and never have problems.

 

[osprey]

From the command "info sync"

‘sync’ writes any data buffered in memory out to the storage device.
This can include (but is not limited to) modified superblocks, modified
inodes, and delayed reads and writes. This must be implemented by the
kernel; The ‘sync’ program does nothing but exercise the ‘sync’,
‘syncfs’, ‘fsync’, and ‘fdatasync’ system calls.

The kernel keeps data in memory to avoid doing (relatively slow)
device reads and writes. This improves performance, but if the computer
crashes, data may be lost or the file system corrupted as a result. The
‘sync’ command instructs the kernel to write data in memory to
persistent storage.

In other words, sync initiates writes to the "persistent storage" and amends the inodes and other relevant files. When working on the command line, that is all that is needed to ensure a write has occurred.

 

[CaffeineAddict]

Sync is not the same. Unmounting a file system writes data to the administrative blocks.

I'm not able confirm this from man umount all it says is that it may fail if there are processes still using a filesystem.

 

[Trenix25]

I'm not able confirm this from man umount all it says is that it may fail if there are processes still using a filesystem.

There is still some stuff, like journaling, that will need to be updated when unmounting. Otherwise you should see "Recovering journal" when checking the drive while it is unmounted. You will not be allowed to unmount if the something is still using it. You will get a Device or resource busy error. The kernel needs to keep track of which devices are mounted anyway.

Signed,

Matthew Campbell