Linux command to filter date string from file and compare it against a date

Question

I have a file that looks like:

ASDF,100090,D84,2007-12-29T01:07:35Z
ASDF,101090,F84,2008-01-03T01:09:36Z
ASDF,101190,h84,2008-01-04T01:07:31Z
ASDF,178915,r54,2008-01-15T01:09:21Z
ASDF,144290,k74,2008-02-05T01:03:31Z

I want to find the count of lines older than date 2008-01-10. I am able to list the dates with the command cut -d "," -f 4 2008A.csv | cut -d "T" -f 1:

2007-12-29
2008-01-03
2008-01-04
2008-01-15
2008-02-05

But how do I find the number of lines older than 2008-01-10?

Answer 1

The procedure:

1. inject a marker with a semi-date that is numerically slightly lower than your boundary date;
2. apply numeric sort;
3. stop processing at the marker, without passing the marker line itself;
4. count lines.

The code (it starts with your code):

{ cut -d "," -f 4 2008A.csv | cut -d "T" -f 1; \
printf "2008-01-09.9 marker\n"; } |
sort -n |
sed '/marker/Q' |
wc -l

Notes:

• Q in sed is not POSIX, your sed may not understand it. If so, use sed '/marker/q'; but this will pass the marker line, you need to filter it out before wc -l xor decrease the final result by one.
• Beware of empty lines (if any); sort will place them at the very beginning, this will alter the result. My answer assumes there are none. A newline just before EOF (end of file) is OK, it doesn't create an empty line.

Answer 2

Just use grep:

grep -v '2008-01-\(0[0-9]\|10\)' 2008A.csv

or

grep -v -E '2008-01-(0[0-9]|10)' 2008A.csv

(in other words, remove all the lines with 2008-01-0something or 2008-01-10).

Answer 3

Use dategrep from dateutils:

$ dategrep '<2008-01-10' <<EOF
ASDF,100090,D84,2007-12-29T01:07:35Z
ASDF,101090,F84,2008-01-03T01:09:36Z
ASDF,101190,h84,2008-01-04T01:07:31Z
ASDF,178915,r54,2008-01-15T01:09:21Z
ASDF,144290,k74,2008-02-05T01:03:31Z
EOF
=>
  ASDF,100090,D84,2007-12-29T01:07:35Z
  ASDF,101090,F84,2008-01-03T01:09:36Z
  ASDF,101190,h84,2008-01-04T01:07:31Z