On a centos box, I like to dump tcp connections - I would like to see if a server tries to send requests to a certain IP. Usually tcpdump would do the trick - but tcpdump is not installed, and installing software is not an option (because of company policy). I am afraid netstat will not show me a single request.
So I was wondering what other options I have. I do have root access on the server.
I'd really try to get tcpdump. That being said, some alternatives to see if a certain connection exists for an IP are:
strace:
[kbrandt@ny-kbrandt01: ~] strace -e trace=network nc 1.2.3.4 1234
...
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(1234), sin_addr=inet_addr("1.2.3.4")}, 16) = -1 EINPROGRESS (Operation now in progress)
lsof:
[kbrandt@ny-kbrandt01: ~] nc 1.2.3.4 1234 &
[1] 11434
[kbrandt@ny-kbrandt01: ~] lsof -p 11434
....
nc 11434 kbrandt 3u IPv4 4543149 0t0 TCP 10.7.0.78:58886->1.2.3.4:search-agent (SYN_SENT)
netstat:
[kbrandt@ny-kbrandt01: ~] nc 1.2.3.4 1234 &
[1] 11486
[kbrandt@ny-kbrandt01: ~] sudo netstat -a -p | grep 11486
tcp 0 1 10.7.0.78:58891 1.2.3.4:search-agent SYN_SENT 11486/nc
Surely you have python?
from socket import *
from struct import unpack
import sys
INTERFACE = "eth0"
TARGET = "8.8.8.8"
if __name__ == "__main__":
sock = socket(AF_PACKET, SOCK_DGRAM, 0x0800)
sock.bind((INTERFACE, 0x0800))
while True:
data = sock.recvfrom(1500, 0)[0]
ip = inet_ntop(AF_INET, data[12:16])
if ip == TARGET:
print "GOT TARGET"
sys.exit(1)
This will exit with "GOT TARGET" providing the IP address coming back matches. Since TCP has to send something back during a handshake, this should catch anything from a specific target address. It doesn't care if the protocol is TCP or UDP though (nor do I check).
Dont forget to change TARGET and INTERFACE.
Iptables has a debug capability and that can be used for traffic analysis too.
The solution is described on the URL below.
It's also worth reading the following URL to set up the logging of trace output to a file of your choice.
http://backreference.org/2010/06/11/iptables-debugging/
I would not consider this solution equal to tcpdump, but it can be done using a minimal install of Centos. You need to be careful not to fill up the disk with the logs, because tcpdump is much more efficient in disk use. Turn off the logging when it is not required.
You can use the following as a basic template in your script.
# Logging
log(){
SOURCE=a.b.c.d (IP address)
$IPT -A INPUT -s $SOURCE -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "In: "
$IPT -A OUTPUT -s $SOURCE -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "Out: "
$IPT -A FORWARD -s $SOURCE -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "Fw: "
$IPT -t nat -A POSTROUTING -m limit --limit 50/minute -j LOG --log-level 7 --log-prefix "Nat: "
}
#log (remove comment to enable)
trace(){
iptables -t raw -A PREROUTING -p tcp -j TRACE
iptables -t raw -A OUTPUT -p tcp -j TRACE
}
#trace (remove comment to enable)
If you need specific software to do your job, and aren't allowed to, you're either not making a good business case or selling your ideas to the right people... or you're not in control of this system...
If I were tasked to do something and needed the type of debugging/troubleshooting information that you require in this case, I'd use the right tool. That's likely tcpdump or tshark. Yes, those are pieces of software, but I'd deem them more essential utilities. In fact, they are utilities that could be installed or loaded onto the system temporarily and removed without incident (is removable media an option?...hint)
But the point is that a janky workaround to company policy probably takes more effort than getting approval for this use case.
Kyle offered some great options. One more would be to use iptables:
[james@server ~]$ sudo iptables -I OUTPUT -d 1.2.3.4/32
...
[james@server ~]$ sudo iptables -L OUTPUT -n -v
Chain OUTPUT (policy ACCEPT 105 packets, 35602 bytes)
pkts bytes target prot opt in out source destination
87 33484 LOG all -- * * 0.0.0.0/0 1.2.3.4 LOG flags 0 level 4
This is essentially an accounting rule. It doesn't explicitly permit or deny traffic, so the default policy for the OUTPUT chain is used (which defaults to ACCEPT). However, any matching packet will increment the counters for the rule.
You can optionally log details about the packet as well with the -j LOG option:
[james@server ~]$ sudo iptables -I OUTPUT -d 1.2.3.4/32 -j LOG
...
[james@server ~]@ dmesg | grep 1.2.3.4 | tail -1
IN= OUT=eth0 SRC=192.168.1.1 DST=1.2.3.4 LEN=100 TOS=0x10 PREC=0x00 TTL=64 ...
Logs will go to the kernel logging facility, so it should show up in /var/log/messages on Red Hat derivatives and /var/log/kern.log on Debian derivatives. It would also be visible in the output of dmesg, as shown. Unlike tcpdump, however, it will not log the complete contents of the packet, only the contents of the packet header.
Since your server is connecting to a certain IP, I presume it will be to a port that you also have knowledge of?
In any case, netstat or ss are designed to do what you want. You can do the same with either command:
netstat -n -t | awk '{print $5}' | grep A.B.C.D:n
ss -n -t | awk '{print $5}' | grep A.B.C.D:n
where A.B.C.D represents an IPv4 address, and n represents a port number that your server is connecting to on the remote side. For example:
ss -n -t | awk '{print $5}' | grep 10.137.54.22:3389
Or, if you'd just like to know that the connection is made:
ss -n -t | awk '{print $5}' | grep -q 10.137.54.22:3389 && echo "CONNECTION MADE"
If you don't know the port number that you are attempting to connect to, then the job would be more difficult as TCP will open a port on both sides of the conversation for the data and ACK packets. In that case you could simply grep for the IP address to show that any connection has been made, whether that's to or from.
Finally, you can loop this to your heart's content to use as your monitoring tool:
while true; do
ss -n -t | awk '{print $5}' | grep -q A.B.C.D:n && \
echo "CONNECTION MADE" && \
exit 0
sleep 1
done
