Linux: Copy files with dates in filename between two dates

Question

I have a folder full of files where I from time to time need to copy a subset of these to somewhere else for processing.

The files have the date they were created in the filename, i.e. filename_20131028.txt.

How do I go about selecting and copying the files between two dates. Optimally I'd like to make a script like 'get_my_foo_files.sh 20131010 20131025' and get the files between 10th october and 25th october.

Thanks in advance.

Edit: I'm open for solutions in Python/Perl as well.

Answer 1

Using GNU date you can loop through a range of YYYYMMDD format dates easily:

#!/usr/bin/bash
first=$1
last=$2

while [[ $first -le $last ]]; do
    match=$(date +%Y%m%d  -d $first)
    echo "filename_${match}.txt"
    first=$(date +%Y%m%d -d "$first 1 day")
done 

Add your processing instead of echo .... This uses date arithmetic, where "n day" is addition of days to the given date, "-n day" also works for subtraction (though watch out for this beartrap if HH:MM is present too).

(In case you ever encounter it, ancient date (sh-utils era) versions allowed an mktime() normalisation feature, you could simply keep incrementing the day-of-month up to 99, and it would silently fix it for you:

 $ date -d 20160832
 Thu Sep  1 00:00:00 IST 2016

)

Something very similar (but arguably not identical in behaviour) can also be done with filesystem timestamps using GNU find and its modification time related options (-mtime, -daystart and more).

A possibly useful variation is to use bash's extended globbing (via @(pattern)) so you can deal with all the files in one go:

#!/usr/bin/bash
first=$1
last=$2

globlist=$1
while [[ $first -le $last ]]; do
    first=$(date +%Y%m%d -d "$first 1 day")
    globlist="${globlist}|${first}"
done 

shopt -s nullglob extglob
filelist="$(echo filename_@($globlist).txt)"  # NB whitespace in filenames
echo "filelist is: $filelist"

filearr=( filename_@($globlist).txt )  # safer for troublesome names
declare -p filearr

This also sets nullglob so that zero matches results in an empty string rather than an error. This is useful when it's better to perform a single operation on multiple files (e.g. tar or rsync).