10GbE Win8.1 file sharing between 3 PCs without a switch

2

I have 3 windows 8.1 PCs, and they each have two one-port Mellanox 10GbE NICs. I'm using 3 SFP+ Twinax cables to connect them all as per below diagram. Only one of those PCs is also connected to internet via 1GbE port on the Mobo (and is the only PC that needs internet).

Network topology: http://i61.tinypic.com/2ymhl00.jpg

I want to be able to access files from all computers from any of the 3 computers using Windows sharing. If any one of the 3 PCs is shut down at any time, I still want to be able to transfer files between the two that are turned on.

What network configuration do I have to perform in order to enable stable file sharing between all of the computers? I read about static routing being the answer, but I have no idea what IP/subnetmask/gateway values I need to use on each machine for proper configuration.

I'm not using a switch/router because no affordable switch/router with 3+ 10GbE ports even exists. I need 10GbE for moving huge raw 4K video files at speeds up to 1GB/s between raid0 arrays. 1GbE speed of 111MB/s is not acceptable.

ToASTER BLU

Posted 2015-08-10T22:49:20.020

Reputation: 23

Setup three networks, with each node participating in two of them. Then use HOSTS file to map names to directly connected IPs. – ssnobody – 2015-08-10T22:58:02.280

@ssnobody, thanks for your input. How exactly do I set node participation within a network in Win8.1? Sorry if it's a dumb question, but I don't have any network config experience. Thanks – ToASTER BLU – 2015-08-10T23:40:57.410

You set node participation in networks by setting their IP address to be part of a particular network. The answer by Gene seems like a workable example. – ssnobody – 2015-08-10T23:58:59.133

That is not a good design. You'll have to use different IP^s to connect to the target PC, multiple router, etc. The proper way to do it is with one (or two if youw ant failover) 10 GBe switches. You can find pretty decent Cisco things in eBay for 150 bucks, however, it may be too advanced for you to configure them. – DGoiko – 2017-11-18T14:05:20.663

Answers

3

Here is one way to do it:

Example diagram

  • System A would have the IPs 10.11.11.10 & 10.22.22.10.
  • System B would have the IPs 10.22.22.20 & 10.33.33.20.
  • System C would have the IPs 10.33.33.30 & 10.11.11.30.

The netmask for all these interfaces would be set to 255.255.255.0 (/24).

None of those network interfaces would have a gateway defined (Windows can only have one gateway defined). I'm assuming all three PCs will have some kind of general internet access for updates and whatnot.

Additionally, to make life easier:

Gene

Posted 2015-08-10T22:49:20.020

Reputation: 210

Thanks Gene, what a great answer (and a beautiful diagram)! :-) This makes things very clear, except I'm not sure what you mean by permanent mounting of network shares? What exactly does that involve? – ToASTER BLU – 2015-08-10T23:58:18.677

As indicated in the original post, only System A would have general internet access, no internet connection sharing required as systems B and C are intended to be offline systems (apart from file sharing). – ToASTER BLU – 2015-08-11T00:00:25.067

Network drive mapping. It's when you assign a network share a drive letter or directory on your system. So instead of navigating to the network share manually every time (e.g. \10.11.11.10\media) you can just go to a folder that already exists on your system. – Gene – 2015-08-11T00:02:09.060

1Ah, I see. Like a "favourite folder" shortcut in the Windows Explorer, right? Thanks so much for your input. :-D – ToASTER BLU – 2015-08-11T00:06:25.357

You're welcome. :) I updated the answer with a link to how to perform drive mapping in Windows 8. – Gene – 2015-08-11T00:07:04.010

Perfect. Thanks. Can I just ask one last thing - what is the significance of the hosts file, and what values should I put in? The link in your answer just deals with how to run notepad++ as administrator. :) – ToASTER BLU – 2015-08-11T00:14:49.597

1HOSTS file maps names (e.g. "SystemA" or "ToasterServer") to IP addresses. That mapping allows you to do something like ping ToasterServer or access shared files using \\ToasterServer\ and have it actually talk to the right computer. Usually this "name resolution" requires a DNS server, which I don't think is necessary in your scenario, and you can use the HOSTS file to avoid setting up a DNS server. – ssnobody – 2015-08-11T00:37:09.017

I get it. :-) This means that I can refer to System A as ToasterServer from both Systems B and C, even though ToasterServer will have different IPs for each of those systems. :-) Thanks again for clear explanations. – ToASTER BLU – 2015-08-11T01:53:01.343

You need them to route, if not the failover means nothing: one network cable between PCs down and communication is lost. – DGoiko – 2017-11-18T14:06:41.090

@DGoiko, the question wasn't regarding access to the internet, it isn't a question about high availability. The original poster wants each computer to be able to access the data on the other two computers utilising their 10GbE links. It would appear these are video editing stations. – Gene – 2017-11-28T08:15:08.047

@Gene I know, but if you do it it your way, when the link between computers is down (cable or NIC broken) they wont be able to see each other or the internet, depending which cable it is. An HA scenario can still work if 1 link is down. Its the proper way to do so unless you explicitly dont want any data to pass through System A to go from B to C. If system A is there JUST to communicate them with the internet and nver has to send local data to System C/D it would be enough for it to have 3 Gb NICs, and the other 2 would only need 1 Gb card and another 10 Gb which is a much cheaper deployment. – DGoiko – 2017-11-28T13:14:31.870

@Gene by the way, I read his post again and he clearly said: " If any one of the 3 PCs is shut down at any time, I still want to be able to transfer files between the two that are turned on." So this is HA scenario if you expand the request to "PC or Link" – DGoiko – 2017-11-28T13:19:03.633

@DGoiko Read the original question again. Only one computer requires internet access. Each computer just needs to be able to access data on the other two systems. Since it's a triangle topology that is already covered. Nothing more is needed. Adding switching or routing creates more complexity and cost for features the original poster doesn't need. Twinax cables are inexpensive and they already had the necessary hardware. – Gene – 2017-11-28T17:45:57.483