Printer-friendly version

Sed, that most useful of pattern replacement tools, can be used for much more than simple string replacements in text files. This article will examine the uses of Sed for creating commands and feeding them to the shell. This can be extremely useful, especially when working on file manipulations for large batches of files at a time. Let's say for example that you've got a directory full of files, and you'd like to move them into subdirectories according to the first three characters of the filenames. This is a one-liner with sed.

ls | sed "s/\(...\).*/test -d '\1' || mkdir -p '\1'; mv '&' '\1\/'/" | bash 

This will execute a new shell, and then feed each 'test || mkdir; mv' to it. It will run reasonably quickly because it's only executing [em]one[/em] subshell, and is quicker to write and implement than a loop. The magic here is in two little-exercised features in sed, namely holdspaces and the special symbol for the input string. Using the \(..\) construct takes the regular expression between the '\(' and '\)' and puts it into a hold space. The holdspaces are numbered, and there are only nine of them, but that's usually enough. If it's not, there's always multi-line sed. Anyway, the holdspaces are numbered, and as you might imagine, are assigned to these patterns sequentially. To recall them, in the replacement section you just use \n (like \1, \2 etc). This is replaced with the contents of that numbered holdspace. Second is the '&' character, which is replaced in this second section with the entire contents of the pattern space (the whole line of input) AS IT WAS (before any substitutions). In this case, it's just a filename.

A few more examples then...

• changing capitalized (DOS-style) file extensions to lower case without changing the rest of the filename:
  

  ls | sed -n "/\.[A-Z]..$/s/\(.*\)\.\([A-Z]..\)/mv '&' \"\1.\`echo '\2' | tr [:upper:] [:lower:]\`\"/p" | sh

  To do the same with bash:

  while read file; do mv $file "${file%%.[A-Z]??}.`echo ${file##*.} | tr [:upper:] [:lower:]`"; done < <(ls *.JPG)

  It might seem like they're pretty much the same, but with sed, you can leave off the '| sh' and keep running the line until it's right, then just finish by adding the pipe to the shell.

Hopefully some of these examples will help you get things done faster, and perhaps learn a little bit more about sed, the wonderful streaming editor.

Brian