July 1, 2003

Shell For-Loops Argument Problem

Argh. This is driving me mad with frustration.

I'm trying to write a shell script with a loop over a set of file names. The file names are not arguments to the script. Normally, it would be sufficient to do: 

for i in `ls *.jpg`
do
   echo $i
done

However, some of the target files have spaces in their names. This causes the 'for i in' mechanism to parse the filenames into multiple pieces, and therefore break. E.g., suppose the only file is "My Trip.jpg". The output of the script above is: 

   My
   Trip.jpg

There are arguments I can throw onto the 'ls' which single-quote or double-quote the filenames, or slash-escape the spaces in the filenames. However, the 'for i in' mechanism ignores the quotes & escapes in the 'ls' output, continuing to parse the filenames into multiple pieces.

Now, I am aware that the "$@" variable protects (at least) single quotes in the script argument list, but I can't arrange to pass the filenames in as an argument list. I'm looping over several directories in an outer loop, so I'd have to do something like: 

for i in $dirlist
do 
   cd $i
   do_things_with_files.bat `ls --quoted-style=shell-always *.jpg` #puts filenames in single quotes
   cd ..
done

And generating the list of files in that fashion still seems to destroy the integrity of the quotes.

I'm at a loss. I'd rather not have to remove all spaces from file names! Any suggestions?

Posted by Greg at July 1, 2003 1:50 PM

Comments
#1 ::: Greg Morrow ::: July 1, 2003 2:54 PM ::: link

I've discovered a way. Because it requires "$@" to work, it requires calling a second script, which decreases the clarity by splitting off some functionality.

for i in $dirlist
do
   cd $i
   eval do_things_with_files.bat `ls -Q *.jpg`
done

The eval basically causes two shell-interpretation passes; the first turns the "ls -Q *.jpg" into a list of files with double-quotes, while the second turns the list of double-quoted files into the correct arguments.

Good enough, I suppose. Good enough to get me to a beta-test.

#2 ::: Ginger ::: July 1, 2003 2:58 PM ::: link

You need to use underscores in the filenames anyway. Some browsers don't interpret spaces correctly. (Learned this with a different kind of file.)

#3 ::: Michael Croft ::: July 2, 2003 10:46 AM ::: link

I've seen worse looking maps. I wonder what the real map looks like?

#4 ::: Michael Croft ::: July 2, 2003 10:46 AM ::: link

Hey! You're not the comment window for Chuck's blog!

#5 ::: Phillip J. Birmingham ::: July 2, 2003 2:56 PM ::: link

I usually do this sort of thing in Perl, and have never had any trouble like this. Must investigate further -- I also don't use lots of spaces in filenames, so this may be lurking to bite me on the butt.