2

Recent versions of docker (or any version of nvidia-docker) allow direct(?) access to the host GPU from within docker containers, with full access to CUDA APIs. This is very convenient when deploying complex machine learning inference servers.

However, currently only Linux hosts are supported as far as I can tell.

Why can't Microsoft and Apple step up their game and provide the same level of support? Which is to say, what trick is being used on Linux, which is apparently hard to imitate in other OSes?

Will
  • 219
  • 2
  • 7

3 Answers3

3

Docker on linux uses the hosting kernel directly, but Docker on Windows actually runs in a VM (hosted by Hyper-V in Windows) because Windows doesn't have a linux kernel for Docker to directly interact with. So reaching the host GPU from a docker container on Windows then requires reaching out from the docker container in the guest VM into the hosting Windows OS to communicate with the GPU drivers. A lot of special development from Microsoft and the GPU manufacturer is required to support all that properly.

Raulinbonn
  • 54
  • 5
  • 1
    Doh! That makes perfect sense, thanks. For some inexplicable reason I completely forgot that Docker uses the host Linux kernel natively, isolated just by cgroups and kernel namespaces. (Embarrassing, but it was late at night when I asked the question.) – Will Feb 14 '20 at 22:53
1

As of Feb 2020, access to GPU via Windows container is possible for Dx and compute based on Dx. You can read more at https://github.com/MicrosoftDocs/Virtualization-Documentation/tree/master/windows-container-samples/directx, in particular the below section:

demonstrates containerizing and running a DirectX workload with GPU acceleration. Specifically, we use the WinMLRunner machine learning inference app

Windows containers with the accompanying Docker update as mentioned in [https://techcommunity.microsoft.com/t5/containers/bringing-gpu-acceleration-to-windows-containers/ba-p/393939], add a "process isolation" mode, that does not require or use a VM.

As for CUDA access, wait for future updates on what additional APIs are supported beyond Dx.

Update 29Jun2020 - CUDA is now accessible from WSL2, on Windows. https://docs.microsoft.com/en-us/windows/wsl/tutorials/gpu-compute https://developer.nvidia.com/cuda/wsl

Prabindh
  • 111
  • 3
0

It would require GPU passthrough (i.e DDA) which is only available in Windows Server 2016

Bert
  • 2,733
  • 11
  • 12