Solved Find and copy files script doesn't work

[banderas20]

Hi.

I have the following script which searches for a list of files (included in a list.txt file) and then copies them into a new directory. The source files are all under the same folder, but they are scattered among several subfolders.

#!/bin/bash
SOURCE="/opt/source_folder/"
DEST="/home/destfolder"

while read -r line
do
  find $SOURCE -type f -name "$line" -exec cp {} $DEST \;

done < list.txt

However, it always gets stuck in the first file. It's like the find command gave timeout.
I have made sure that the files do exist.

Am I missing something?

Thanks in advance!

PS: Using RedHat

 

[CaffeineAddict]

while read -r line

What's the purpose of that command? you don't need to read file contents to copy them.
And even if you do that would apply to which file?

 

[banderas20]

What's the purpose of that command? you don't need to read file contents to copy them.
And even if you do that would apply to which file?

The purpose is to read every filename stored in list.txt. For each filename, I find the file under SOURCE directory (or subdirectories) and copy every file in DEST.

 

[dos2unix]

Maybe try this.

#!/bin/bash
SOURCE="/opt/source_folder/"
DEST="/home/destfolder"

while IFS= read -r line
do
find "$SOURCE" -type f -name "$line" -exec cp {} "$DEST" \;
done < list.txt


IFS This prevents leading/trailing whitespace from being trimmed.
Quoting Variables Ensures that paths with spaces are handled correctly.

 

[banderas20]

Maybe try this.

#!/bin/bash
SOURCE="/opt/source_folder/"
DEST="/home/destfolder"

while IFS= read -r line
do
find "$SOURCE" -type f -name "$line" -exec cp {} "$DEST" \;
done < list.txt


IFS This prevents leading/trailing whitespace from being trimmed.
Quoting Variables Ensures that paths with spaces are handled correctly.

Still nothing...

Thanks anyway for pointing that out.

 

[osprey]

The purpose is to read every filename stored in list.txt. For each filename, I find the file under SOURCE directory (or subdirectories) and copy every file in DEST.

This following is a proof of concept which generalises the script by avoiding hard coding of directories, but rather using the command line. It has no error correction and a few other things it might need, but it works.

The file with the list of files is named: list.txt.

Here is the filesystem it's working on:

$ ls -RF
.:
dir2/  dir3/  dir4/  fifi*  list.txt

./dir2:
file1  file2  file3  file4  file5  setPL/

./dir2/setPL:
LICENSE  readme.md  setPL.sh*

./dir3:
newfile1  newfile2  newfile3  newfile4  newfile5

./dir4:

This is the contents of list.txt:

$ cat list.txt
file1
file2
newfile1

The script contents is:

$ cat fifi
#!/bin/bash
for i in $(cat $1); do
find "$2" -type f -name "$i" -exec cp {} $3 2>/dev/null \;
done;

# first arg $1 is the file with the list
# second arg $2 is directory to search
# third arg $3 is the directory to cp files to

Running the script thus:

$ ./fifi list.txt . dir4

Rechecking the locations of the files listed in list.txt shows them now to reside in the directory specified on the command line, dir4

$ ls -RF
.:
dir2/  dir3/  dir4/  fifi*  list.txt

./dir2:
file1  file2  file3  file4  file5  setPL/

./dir2/setPL:
LICENSE  readme.md  setPL.sh*

./dir3:
newfile1  newfile2  newfile3  newfile4  newfile5

./dir4:
file1  file2  newfile1

 

[truckerDave]

I'm no guru. But a thought popped into my head.

Could this be a permissions issue?

I wrote a php script to select mp3 files at random and rename them with a number at the beginning of the name. (My player has the habit of selecting the same 20 or so songs, out of 800+ when set to random).

I had to change permissions on the mp3 files to www-data for it to work. However, if I recall correctly, I was shown an error regarding permissions when I first tried my script.

 

[CaffeineAddict]

The purpose is to read every filename stored in list.txt. For each filename, I find the file under SOURCE directory (or subdirectories) and copy every file in DEST.

Oh I see, I think the problem with your loop is that it tries to read a line without accessing the lists.txt first, therefore the loop immediately terminates.

I suggest for loop instead or modify your loop, see example:

Is there a do-while loop in bash?

Is there a do-while loop in bash? I know how to program a while loop in bash: while [[ condition ]]; do body done Is there a similar construct, but for a do-while loop, where the body is exe...

stackoverflow.com

Basically problem is that your condition is in the wrong place.
done < list.txt there should be nothing after done keyword.
done mean end of the loop so put your condition where it belongs, and that's after while or inside the loop body.

 

[banderas20]

I fixed it. It was the encoding of mi txt file. It was generated with Windows and every line had a '\r' at the end.

I removed them with dos2unix and it worked.

Thanks everyone!

 

[Diputs]

I fixed it. It was the encoding of mi txt file. It was generated with Windows and every line had a '\r' at the end.

I removed them with dos2unix and it worked.

Thanks everyone!

When reviewing any code, the question to ask is : what could go wrong ?
Even if you have 5 lines of code, the answer may be a lot.

Any file, meaning any user manipulated file, may be missing. It may be non-readable. It may exist as a directory.
That's just a start ..
Then, it can exist but be empty ...
... and, it can also be Windows formatted. If you have any environment with FTP involved, you know that is the price to pay.
A file can also be binary, or partly binary.
Files can also be huge, even text files.
The file may exist but not be in the location you think it is, in other words, the path is wrongly used.
File can be a broken symbolic link.
File can be a sparse file.
File can have other case, or just spelled incorrectly.
Disk can be unmounted

I'm sure I can go on