Getting started with Parameter expansion and pattern substitution for files

I have recently been working on a project to transfer all of my legacy backups that include quite a lot of cd's and dvd's with tracks on that I have backed up over the years. When backing these up, a lot of the file names were prefixed with numbers underscores and hyphens etc. When organising these thousands of files, I was looking for a way to name them in a consistent manner and where necessary, remove any duplicates that I had.

After some investigation, I discovered I could use Parameter Expansion to help me. For those new to bash parameters:

Parameters are a sort of named space in memory you can use to retrieve or store information. Generally speaking, they will store string data, but can also be used to store integers, indexed and associative arrays.

http://mywiki.wooledge.org/BashGuide/Parameters, and 

http://mywiki.wooledge.org/BashGuide/Parameters#Parameter_Expansion
 

Parameter Expansion substitutes a variable or special parameter for its value. It is the primary way of dereferencing (referring to) variables in Bourne-like shells such as Bash.

http://mywiki.wooledge.org/BashFAQ/073

Ok so lets have a look at some examples:

parameter     result
-----------   ------------------------------
$name         polish.ostrich.racing.champion
${name#*.}           ostrich.racing.champion
${name##*.}                         champion
${name%%.*}   polish
${name%.*}    polish.ostrich.racing

Parameter expansion can be used in combination with a for loop.  Here is an example can be used to rename all JPEG files with a .JPG or a .jpeg extension to have a normal .jpg extension. The expression ${file%.*} cuts off everything from the end starting with the last period (.). Then, in the same quotes, a new extension is appended to the expansion result.

$ for file in *.JPG *.jpeg
do mv -- "$file" "${file%.*}.jpg"
done

Ok so lets have a look at some practical examples in my situation

Rename files starting with a number

for name in *; do mv -v "$name" "${name#[0-9]* }"; done

so 05 _27 Suspicious Minds.mp3 becomes _27 Suspicious Minds.mp3

here the square brackets allows us to specify a range of characters. Also the verbose flag outputs any files changes in the console.

Rename files starting with either a number or an underscore 

for name in *; do mv -v "$name" "${name#[0-9_]* }"; done

e.g. _01 Heartbreak Hotel.mp3 becomes Heartbreak Hotel.mp3 

SO we have simply added the underscore character to the reg expression

Rename files that start with a hyphen or -

for name in *; do mv -v -- "$name" "${name#-* }"; done

so - 26 In The Ghetto.mp3 becomes In The Ghetto.mp3

Notice the requirement for the -- which tells the mv command not to expect anymore flags! Also as we no longer require the number range, we can dispense with the square brackets

Remove the first 2 characters from the filename

for name in *; do mv -v -- "$name" "${name:2}"; done

Again, I have specified the -- flag here incase a filename starts with a hyphen. Other useful mv flags include:

    -f      Do not prompt for confirmation before overwriting the destination path.  (The -f option overrides any previous -i or -n options.)

    -h      If the target operand is a symbolic link to a directory, do not follow it.  This causes the mv utility to rename the file source to the
            destination path target rather than moving source into the directory referenced by target.

    -i      Cause mv to write a prompt to standard error before moving a file that would overwrite an existing file.  If the response from the
            standard input begins with the character ‘y’ or ‘Y’, the move is attempted.  (The -i option overrides any previous -f or -n options.)

    -n      Do not overwrite an existing file.  (The -n option overrides any previous -f or -i options.)

    -v      Cause mv to be verbose, showing files after they are moved.

for more info man mv