dd failure

[BHarts]

Hello. I have used the Linux dd command many times. Recently, on a USB-connected 4TB drive, I went to use it to copy one small partition to another. After about 15GB of copying, it failed with a message "Input/Output error". Literally worded, that was all; there was no clue as to the reason for the failure. It is a rather new drive, and running the fsck command on its files turns up no errors. (Yes, I know that the dd command does not need to pay attention to file structures; I ran the fsck to check for obvious errors). I got the block size on the partition and used the 'bs' option to specify block sizes. Nothing so far has made the error message change. Eventually, I was able to use the Clonezilla utility to get the partition copied. That was 100% successful; no errors at any point. Still, I would like to know what could cause the dd command to fail. Perhaps it cannot handle drives larger than a certain size, e.g., 2TB, even for small partitions such as the 32GB size. I have searched for articles on the limitations of "dd", and so far, no luck. btw: the host system is running the Devuan variant of Debian. I even tried booting up with an up-to-date version of System Rescue; its dd operations had the same results.

Comment is invited.

 

[forester]

please enter the dd command(s) given before the IO Error

 

[forester]

Was Clonezilla Live used interactively so errors could be noted?
Clonezilla uses dd

 

[BHarts]

please enter the dd command(s) given before the IO Error

dd if=/dev/sdd1 of=/dev/sdd2 status=progress

 

[BHarts]

Was Clonezilla Live used interactively so errors could be noted?
Clonezilla uses dd

I am new to Clonezilla; I basically followed the most direct instructions it had. I compared the results of the copy, and they appear to be 100% good.

 

[shebang]

dd if=/dev/sdd1 of=/dev/sdd2 status=progress

I used to use dd on my mac but that's been years ago. dd uses 512k bs by default. I believe for a partition copy they suggest 64 and use noerror

I would try,
dd if=dev/sdd1 of=dev/sdd2 bs=64M conv=noerror

 

[forester]

Still, I would like to know what could cause the dd command to fail.

it's probably the external 4TB hard drive but cannot say for sure. Just use Clonezilla or try

dd if=/dev/sdd1 of=/dev/sdd2 bs=4096 conv=sync status=progress

 

[dos2unix]

Make sure that sdd2 is created in advance. Also make sure it's at least as big sdd1. (I usually go a couple of GB larger).
The bs usually isn't needed, dd will typically load as much as it can into memory between reads and writes anyway.
I don't usually recommend using dd on the same input/target device. In fact If you have another drive, I've had better
luck dd'ing to an external drive image files, and then back to the target disk. But it should work on the same device.
nvme m.2 drive are full duplex, so they can read and write same time. Sata SSD/HDD drives are not so you get slower
i/O, because they can't do both at same time.

If it's a external USB drive, it'll be even slower.

 

[BHarts]

I used to use dd on my mac but that's been years ago. dd uses 512k bs by default. I believe for a partition copy they suggest 64 and use noerror

I would try,
dd if=dev/sdd1 of=dev/sdd2 bs=64M conv=noerror

Hello! Thanks for the "64" suggestion. I did a "man dd" on the Linux system. For the noerror option, it says to continue after a read error is encountered. I find this puzzling ...

 

[BHarts]

Make sure that sdd2 is created in advance. Also make sure it's at least as big sdd1. (I usually go a couple of GB larger).
The bs usually isn't needed, dd will typically load as much as it can into memory between reads and writes anyway.
I don't usually recommend using dd on the same input/target device. In fact If you have another drive, I've had better
luck dd'ing to an external drive image files, and then back to the target disk. But it should work on the same device.
nvme m.2 drive are full duplex, so they can read and write same time. Sata SSD/HDD drives are not so you get slower
i/O, because they can't do both at same time.

If it's a external USB drive, it'll be even slower.

I hear you. I had used another drive as the destination partition and got the same results. The source drive in this case is an external USB connection; I do not know about the "full duplex" factor for USB; that looks interesting!

 

[forester]

i believe input from 4TB ext drive may be too slow. Hence, sync. if not, config file editing somewhere. Try.

 

[BHarts]

i believe input from 4TB ext drive may be too slow. Hence, sync. if not, config file editing somewhere. Try.

Thanks!

 

[dos2unix]

dd uses 512k bs by default.

For me personally I've found that ibs and obs make the copy faster than bs.
Maybe if you only have 2 or 4GB of RAM it won't make much difference but you have 32 or 64GB.
I've had it do a complete copy in one pass (50GB).
ibs=32G obs=32G if you can. But even 4G, 8G or 16G makes a difference.

 

[BHarts]

For me personally I've found that ibs and obs make the copy faster than bs.
Maybe if you only have 2 or 4GB of RAM it won't make much difference but you have 32 or 64GB.
I've had it do a complete copy in one pass (50GB).
ibs=32G obs=32G if you can. But even 4G, 8G or 16G makes a difference.

Thanks!