1

I'm using Apache Spark, a Java application, to create a cluster of machines. The processes that are launched try to communicate with each other across randomized ports. Is there a way to script the opening of a random port in the cluster?

This is a similar answer, but I want to open a single randomized port, not a range. I don't know what the range is, but I guess I could try and figure that out. https://serverfault.com/a/540517/398062

activedecay
  • 205
  • 1
  • 2
  • 6

1 Answers1

1

You could scan the listening ports on the server and run a periodic bash script via crontab to open ports when a port is being detected. This would obviously need more validation but its a working base. If you need help coding the validation port (is the port already open?) just tell me i'll hook you up.

#!/bin/sh

ports=`netstat -pat | grep LISTEN | awk '{ print $4 }' | cut -f2- -d:`


for port in $ports; do
iptables -A INPUT -p tcp --dport $port -j ACCEPT
done
Dexirian
  • 430
  • 2
  • 11
  • This may work, but I think I'll have to scrape through the logs and do the same thing. By the time the Java processes are running, the process on the other machines already expect the port to be available. There is some timeout and waiting, but a cron job may not cut it. – activedecay Jan 25 '19 at 18:19
  • Not sure how your process are being run but you could always call the script or a similar one directly on the building process, example if you're using chef, you could call a EOH code execution after the web server is built – Dexirian Jan 25 '19 at 18:48