21

We have a java server running in linux at a specific port that accepts persistent connections for thousands and thousands of users. Recently our clients are not able to connect with a time out error. We suspect the traffic is getting too high but our java log actually shows that not many are connected in per second.

We suspect that could it be that too many are trying at the same time and they are basically dropped at the OS level and therefore the java program never really gets a chance to accept the connection? Is there some sort of log in linux that can show someone trying to hit a socket?

erotsppa
  • 2,033
  • 6
  • 23
  • 24

5 Answers5

13

iptables -I INPUT -p tcp --dport some_port -j LOG then
tail -f /var/log/messages
Afterwards, to see how much data has been hit by that rule: iptables -L -n -v
Or you could run tcpdump and grep out the ports.

James L
  • 5,915
  • 1
  • 19
  • 24
  • 3
    +1. A slight modification might work better to catch just the new connection attempts: `iptables -I INPUT -p tcp --dport some_port -m state --state NEW`. Note that unless you're interested in the details of each connection attempt, omitting `-j LOG` avoids spamming the system log file with lots of unneeded data. – Steven Monday Oct 21 '10 at 21:17
  • Upon setting this up, prepare to see a stomach-churningly high number of supposedly non-malicious security crawlers connecting to your server, like `implant-scanner-victims-will-be-notified.threatsinkhole.com`, `stretchoid.com`, `jobqueue-listener.jobqueue.netcraft.com-digitalocean`, ... good to know. – ijoseph Sep 06 '20 at 03:21
3

When I have really nasty network problems, I tend to fire up wireshark. For me, there's no better network diagnostic tool when I have to get down to the nitty-gritty details. And don't worry if you can't install it on either source or destination box; you can run tcpdump -w to write packet data to a file on start and/or endpoint, and feed file to wireshark on another box at your convenience.

MadHatter
  • 78,442
  • 20
  • 178
  • 229
1

by the way, you may also want to delete it:

iptables -L INPUT --line-numbers
iptables -D INPUT <line num>
LIU YUE
  • 121
  • 2
1
watch -n1 -d "netstat -an | grep ESTABLISHED | wc -l"

shows the number of currently established connections.

Compare this with your active ulimit settings, and of course with the maximum number of connections your java app can handle.

Michael Hampton
  • 237,123
  • 42
  • 477
  • 940
sjas
  • 305
  • 1
  • 4
  • 12
0

Would be good to see precisely what your Java threads are getting at a socket level. At the same time you would want to correlate that with OS network info. Take a look at AppFirst. They can do this kind of thing.

IAPaddler
  • 161
  • 4