Monitor SSH-traffic per user

Question

We have a backup application that runs once every 12 hours.

Multiple servers, desktops, and laptops connect to an EC2-instance and push a backup via SFTP using their own log-in credentials.

If they need to recover a file, they can browse files via a simple SFTP-Client and restore their files. This has been running very well for the last 8 months.

I would like to know how many data is transferred on a monthly basis, per user via SSH. I don't need the logs for the past 8 months, but something that would record it starting now would be great.

Is there anything that allows me to do this?

OS: Ubuntu 10.10

Answer 1

It took some awk-magic, this is what my colleague and I where able to put together.

#!/bin/bash

main() {
  if [ -e $1 ] ; then
    MONTH=$(date | awk '{ print $2 }')
  elif [ $1 -ge 1 -a $1 -le 12 ] ; then
    month $1
  else
    exit 1
  fi

  echo
  echo "Usage statistics for month $MONTH"
  echo

  USERS=(`awk '/^'$MONTH'.*session opened for local user.*$/ { print $(NF-2) } ' /var/log/auth.log* | sort | uniq`)
  for i in "${USERS[@]}"
  do :
    echo "################################"
    echo "Usage for user: $i"
    READ=0
    WRITTEN=0
    #processes for this user  
    PROCS=(`awk '/^'$MONTH'.*session opened for local user '$i'.*$/ { gsub("\\[|]|sftp-server|:","", $(NF-8)); print $(NF-8) } ' /var/log/auth.log* | sort | uniq`)
    for j in "${PROCS[@]}"
    do :

      TEMP_READ=$(awk '/^'$MONTH'.*\['$j'\].*\ read\ [0-9]+\ written\ [0-9]+$/ { sum+=$(NF-2)}END{ print sum}' /var/log/auth.log*)
      READ=$(($TEMP_READ+$READ))
      TEMP_WRITTEN=$(awk '/^'$MONTH'.*\['$j'\].*\ read\ [0-9]+\ written\ [0-9]+$/ { sum+=$(NF)}END{ print sum}' /var/log/auth.log*)
      WRITTEN=$(($TEMP_WRITTEN+$WRITTEN))
    done
    echo "Read     $(($READ/(1024*1024))) MiB"
    echo "Written  $(($WRITTEN/(1024*1024))) MiB"
    echo "################################"
    echo
  done
}

month() {
case "$1" in
  1)  MONTH='Jan'
    ;;
  2)  MONTH='Feb'
    ;;
  3)  MONTH='Mar'
    ;;
  4)  MONTH='Apr'
    ;;
  5)  MONTH='May'
    ;;
  6)  MONTH='Jun'
    ;;
  7)  MONTH='Jul'
    ;;
  8)  MONTH='Aug'
    ;;
  9)  MONTH='Sep'
    ;;
  10)  MONTH='Oct'
    ;;
  11)  MONTH='Nov'
    ;;
  12)  MONTH='Dec'
    ;;
  *) echo 'Crash and Burn!'
     exit 1
   ;;
esac
}

main $1
exit 0

In sshd_config I put this:

 Subsystem sftp /usr/lib/openssh/sftp-server -l VERBOSE

Warning: This script hogs memory! If you have large logfiles, it could take up to 10 min for the script to finish (tested on EC2 Micro).

Answer 2

You could track user logons by IP (available in the log files), and then use nearly any traffic monitoring solution (Netflow and IPtraf come to mind) to keep track of the port 22 traffic by that IP.

Unfortunately, historical usage for the previous few months is not available unless you already had something to this effect set up.

Answer 3

You might want to check out this post on sftp logging. I think it will get you what you want, with a bit of additional work to parse the logs.

Answer 4

auditd is a good utility to know what have been done to every file. It might not be what you need tough.