How to determine the socket connection up time on Linux

Question

I can verify that the connection is up:

$ netstat -tn | grep "192.168.2.110"
tcp  0  0 192.168.2.100:10444  192.168.2.110:52639  ESTABLISHED

is there a way to check how long this tcp port connection was up (connected)?

(No, I don't have access to app logs)

Answer 1

You can try the following:

1. get the PID (say $pid) of the program by adding the -p option to netstat.

2. identify the proper line in the /proc/net/tcp file by looking at the local_address and/or rem_address fields (note that they are in hex format, specifically the IP address is expressed in little-endian byte order), also make sure that the st is 01 (for ESTABLISHED);

3. note the associated inode field (say $inode);

4. search for that inode among the file descriptors in /proc/$pid/fd and finally query the file access time of the symbolic link:

   find /proc/$pid/fd -lname "socket:\[$inode\]" -printf %t
   

---

That is a grunt work... here's a script (stub) to automatize the above points, it requires the remote address and it prints the socket uptime in seconds:

function suptime() {
    local addr=${1:?Specify the remote IPv4 address}
    local port=${2:?Specify the remote port number}
    # convert the provided address to hex format
    local hex_addr=$(python -c "import socket, struct; print(hex(struct.unpack('<L', socket.inet_aton('$addr'))[0])[2:10].upper().zfill(8))")
    local hex_port=$(python -c "print(hex($port)[2:].upper().zfill(4))")
    # get the PID of the owner process
    local pid=$(netstat -ntp 2>/dev/null | awk '$6 == "ESTABLISHED" && $5 == "'$addr:$port'"{sub("/.*", "", $7); print $7}')
    [ -z "$pid" ] && { echo 'Address does not match' 2>&1; return 1; }
    # get the inode of the socket
    local inode=$(awk '$4 == "01" && $3 == "'$hex_addr:$hex_port'" {print $10}' /proc/net/tcp)
    [ -z "$inode" ] && { echo 'Cannot lookup the socket' 2>&1; return 1; }
    # query the inode status change time
    local timestamp=$(find /proc/$pid/fd -lname "socket:\[$inode\]" -printf %T@)
    [ -z "$timestamp" ] && { echo 'Cannot fetch the timestamp' 2>&1; return 1; }
    # compute the time difference
    LANG=C printf '%s (%.2fs ago)\n' "$(date -d @$timestamp)" $(bc <<<"$(date +%s.%N) - $timestamp")
}

(Edit thanks to Alex for the fixes)

Example:

$ suptime 93.184.216.34 80
Thu Dec 24 16:22:58 CET 2015 (46.12s ago)

Answer 2

This questions was helpful to me, but I found using lsof instead of netstat let me avoid all the HEX stuff:

For a process ${APP} run by user ${USER}, the following returns all the open sockets to the IP address ${IP}:

PEEID=$(sudo pgrep -u ${USER} ${APP}) && for i in `sudo lsof -anP -i -u logstash | grep ${IP} | awk '{print $6}'` ; do echo "${device} time" ; sudo find /proc/${PEEID}/fd -lname "socket:\[${device}\]" -printf %t 2> /dev/null  ; echo  ;  done

The lsof contains the PID too, but I am not sure how to get it and the device number.

This was tested on Amazon Linux.

Answer 3

The script by cYrus worked for me but i had to fix it a bit (to get rid of a "L" in the hex address and to make port a 4 digit hex):

--- suptime.orig    2015-08-20 15:46:12.896652464 +0200
+++ suptime 2015-08-20 15:47:48.560074728 +0200
@@ -7,8 +7,8 @@
     hex_addr=$(python -c "
 import socket, struct;
 print hex(struct.unpack('<L',
-socket.inet_aton('$addr'))[0])[2:].upper().zfill(8)")
-    hex_port=$(python -c "print hex($port)[2:].upper()")
+socket.inet_aton('$addr'))[0])[2:10].upper().zfill(8)")
+    hex_port=$(python -c "print hex($port)[2:].upper().zfill(4)")
     inode=$(awk '$3 == "'$hex_addr:$hex_port'" {print $10}' /proc/net/tcp)
     time=$(find /proc/$pid/fd -lname "socket:\[$inode\]" -printf %A@)
     LANG=C printf '%.2fs' $(bc <<<"$(date +%s.%N) - $time")

Answer 4

How about:

lsof -t -i @192.168.2.110 | xargs ps -fp

You can also tailor the "ps" command to just get pid and start time with -o such as:

lsof -t -i @192.168.2.110 | xargs ps --no-headers -o'pid,start' -p

Of course this assumes the socket was started when the process was.

Answer 5

Thanks for the script maintained in cYrus' answer. I had issues with it printing duplicates, probably because there may be many connections from different PIDs to the provided address, so here's my improved version that also prints the PID on each output line:

function suptime() {
    local addr=${1:?Specify the remote IPv4 address}
    local port=${2:?Specify the remote port number}

    # convert the provided address to hex format
    local hex_addr=$(python -c "import socket, struct; print(hex(struct.unpack('<L', socket.inet_aton('$addr'))[0])[2:10].upper().zfill(8))")
    local hex_port=$(python -c "print(hex($port)[2:].upper().zfill(4))")

    # get the inode of the socket
    local inodes=$(awk '$4 == "01" && $3 == "'$hex_addr:$hex_port'" {print $10}' /proc/net/tcp)
    [ -z "$inodes" ] && { echo 'Cannot lookup the socket(s)' 2>&1; return 1; }

    # get file descriptors
    for inode in $inodes; do
        # get inode's file descriptor details
        local fdinfo=( $(find /proc/[0-9]*/fd -lname "socket:\[$inode\]" -printf "%p %T@") )
        [ -z "$fdinfo" ] && { echo 'Cannot find file descriptor' 2>&1; return 1; }

        # extract pid
        local fdpath=${fdinfo[0]}
        local pid=${fdpath#/proc/}
        pid=${pid%%/*}

        # extract timestamp
        local timestamp=${fdinfo[1]}

        # compute the time difference
        LANG=C printf 'PID: %s; Age: %s (%.2fs ago)\n' "$pid" "$(date -d @$timestamp)" $(bc <<<"$(date +%s.%N) - $timestamp")
    done
}

Notes:

• needs bc, netstat (provided by net-tools on rhel >= 7 and similar systems)
• needs to be run as root