Aws Virtual Gpu Device Plugin

Virtual GPU device plugin for Kubernetes

The virtual device plugin for Kubernetes is a Daemonset that allows you to automatically:

• Expose arbitrary number of virtual GPUs on GPU nodes of your cluster.
• Run ML serving containers backed by Accelerator with low latency and low cost in your Kubernetes cluster.

This repository contains AWS virtual GPU implementation of the [Kubernetes device plugin](https://github.com/kubernetes/design-proposals-archive/blob/main/resource-management/device-plugin.md).

Prerequisites

The list of prerequisites for running the virtual device plugin is described below:

• NVIDIA drivers ~= 361.93
• nvidia-docker version > 2.0 (see how to install and it's prerequisites)
• docker configured with nvidia as the default runtime.
• Kubernetes version >= 1.10

Limitations

• This solution is build on top of Volta Multi-Process Service(MPS). You can only use it on instances types with Tesla-V100 or newer. (Only Amazon EC2 P3 Instances and Amazon EC2 G4 Instances now)
• Virtual GPU device plugin by default set GPU compute mode to EXCLUSIVE_PROCESS which means GPU is assigned to MPS process, individual process threads can submit work to GPU concurrently via MPS server. This GPU can not be used for other purpose.
• Virtual GPU device plugin only on single physical GPU instance like P3.2xlarge if you request k8s.amazonaws.com/vgpu more than 1 in the workloads.
• Virtual GPU device plugin can not work with Nvidia device plugin together. You can label nodes and use selector to install Virtual GPU device plugin