Deploying Docker with GPU support on Windows Subsystem for Linux

At Aotu.ai we develop BrainFrame, a deep learning video analysis platform designed to make smart AI video inference accessible to everyone. BrainFrame makes heavy use of tools such as Docker, docker-compose, and CUDA. These tools allow us to accelerate inference on the GPU, and make it faster and easier to make deterministic deployments.

Even though BrainFrame is primarily deployed on Linux machines, some users like having the option to run and/or deploy on Windows. Unfortunately, getting this set up is a little tricky. In this blog you’ll learn to use WSL with GPU support and also how to deploy docker-compose services on Windows.

Optional Software

During this blog post we will be working in the windows command line. We will also be using a linux shell once WSL is installed on the system. The new (and fancy!) Windows Terminal, which can be downloaded here, greatly simplifies this process. It’s a breeze to use, looks good, and you can open multiple shells within its tab system even if they are from different linux distros or different windows command line tools. It’s also open source and can be found on GitHub!

System Requirements

Let’s start with some system requirements. WSL2 can be installed on most Windows 10 systems. If you are on a personal device, you probably have Windows 10 Home; this also works on Windows 10 Pro and Server editions.
WSL2 is available for both x64 and ARM64 architecture Windows computers. For x64 machines, you must have Version 1903 or higher, along with Build 18362 or higher. For ARM64 machines, you must have Version 2004 or higher, along with Build 19041 or higher. If for some reason you don’t know what architecture your computer uses you can use PowerShell to check.

systeminfo | find “System Type”

PowerShell can be run straight from your Windows Start menu.

To check your Version and Build, you can click the Settings icon in the Start menu, and navigate to System > About. Your windows specs will be displayed at the bottom of the window. You can also press Windows + R, type winver, and hit OK.

Your specs will look similar to the below image. If the version/build is not high enough, try updating Windows to the latest version.

Another requirement is having hardware virtualization enabled in your computer’s BIOS/UEFI. Most computers usually have this enabled by default. You can check this by opening the Task Manager (right click on taskbar and click Task Manager, or press Shift + Ctrl + Esc). Under the Performance tab, it should show a “Virtualization” area with either Enabled or Disabled. This should say Enabled. If it does not, you will have to restart your computer and modify the BIOS/UEFI settings. Look for Virtualization Technology, VTx, or something similar and ensure it is enabled.

Enabling Windows Subsystem for Linux

Windows Subsystem for Linux is an optional feature on Windows machines. We’ll need to enable it along with the Virtual Machine Platform feature for WSL2 to work. To enable it, open an elevated (with admin) PowerShell and run these commands:

dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart

dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart

This will enable WSL and the Virtual Machine platform.

Installing Linux Kernel Update

Running WSL2 requires an update to the Linux kernel package. Be sure to install the package that is associated with your system. It is an installer so just double-click to run it. The packages can be found here:

Windows 10 x64 package.

Windows 10 ARM64 package.

Setting WSL2 as the default version

Once your kernel is updated we need to switch the default WSL version from 1 to 2, you may run this command to set WSL2 as default.

wsl –set-default-version 2

It may notify you that you don’t have any distributions installed, but this is okay as we will install some later and should not matter for now. This simply ensures that all future distros will default to using WSL2 instead of WSL1.

Install your flavor of Linux

Once the above has been completed, you may install any version of Linux you like from the Microsoft Store. Open the Microsoft Store and type Linux to get a list of available distributions available to you. Popular distros are Ubuntu, Debian, and Kali. For brevity, we will be using Ubuntu 20.04 in this blog post.

Once installed, you can launch it from the store or the Start menu, where it will take you through the initial user setup for the machine. (This should only happen the first time you start each distro.)

Enabling your GPU on WSL2

GPU usage in WSL2 is (unfortunately) only available through the Windows Insider Program right now. You will first have to register for the program, which is thankfully fairly simple.

Once registered, follow the instructions on the Windows Insider website to get started. Make sure you choose the Dev Channel when selecting Insider settings during setup. Also make sure you check for updates in the Windows Update panel, and install any Insider updates.

After installing the Windows Insider features, and subscribing to the latest dev channel, you may install the GPU drivers found here to get started using GPU. Note that these drivers only work on distros that use glibc, like Ubuntu or Debian.

If you skipped some of the above steps because you already had WSL2 installed, make sure that your Linux kernel version is greater than or equal to 4.19.121. If not, be sure to update it as the insider preview only works with that version and up.

Setting up CUDA in WSL2

We will be using Ubuntu 20.04 for this installation, but you may use any glibc-based distro you please. Just follow these instructions but with your distro’s tools. For more info refer to NVIDIA’s guide here.We will set up the CUDA Network Repository, and then install CUDA itself on the system. First run these commands in your linux shell:

sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub

sudo sh -c 'echo "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64 /" > /etc/apt/sources.list.d/cuda.list'

sudo apt-get update

Once set up we can now install CUDA. Use this package:

sudo apt-get install cuda-toolkit-11-0

This will take a while to download and install, so go grab a snack

Using Docker with GPU in WSL2

With CUDA now installed on the system, our next step is to set up our workflow for Docker containers.

There is a Docker desktop app for Windows, which is a fabulous tool for running Docker containers. While it provides a really good user experience, it unfortunately does not have GPU support, so we won’t be able to use it. We will have to install Docker using an install script within our Linux shell like this:

curl https://get.docker.com | sh

Next we need to install the NVIDIA Container Toolkit. I am installing this under Ubuntu 20.04 but you may use whatever you like. We must set up the experimental nvidia-docker2 repository as this is what supports WSL2 at this time. To do so, run these commands:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -

curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

curl -s -L https://nvidia.github.io/libnvidia-container/experimental/$distribution/libnvidia-container-experimental.list | sudo tee /etc/apt/sources.list.d/libnvidia-container-experimental.list

Once that is set up, we can now install the package:

sudo apt-get update

sudo apt-get install nvidia-docker2

Once the installation is finished, you may open another WSL2 window and start the docker daemon to complete your install.

sudo service docker stop

sudo service docker start

Running Docker CUDA containers in WSL2

Great! Now that your install is complete, you can now run GPU containers in WSL2!

You can try running “nvidia-smi” to see if WSL2 is able to see your GPU. That should look a little like this:

Here are a few examples from the NVIDIA docs:

docker run --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark

Your output should look like this:

Or we can run a Jupyter notebook!

docker run -it --gpus all -p 8888:8888 tensorflow/tensorflow:latest-gpu-py3-jupyter

Which should look like this:

Congratulations! You now have GPU support in WSL2!

Troubleshooting

Permission Denied errors when trying to run docker containers

If you seem to be getting a permission denied error when running docker containers, that means that you group may have not been added to the container. If this is the case, run this command in your linux shell: