Pyxis

Pyxis

Pyxis is a [SPANK](https://slurm.schedmd.com/spank.html) plugin for the Slurm Workload Manager. It allows unprivileged cluster users to run containerized tasks through the `srun` command.

A pyxis is an [ancient small box or container](https://en.wikipedia.org/wiki/Pyxis_(vessel)).

Benefits

• Seamlessly execute the user's task in an unprivileged container.
• Simple command-line interface.
• Fast Docker image download with support for layers caching and layers sharing across users.
• Supports multi-node MPI jobs through PMI2 or PMIx (requires Slurm support).
• Allows users to install packages inside the container.
• Works with shared filesystems.
• Does not require cluster-wide management of subordinate user/group ids.

Installation

Pyxis requires the [enroot](https://github.com/nvidia/enroot) container utility (version `3.1.0`) to be installed.

Since [Slurm 21.08](https://github.com/SchedMD/slurm/blob/slurm-21-08-8-2/RELEASE_NOTES#L119-L121), pyxis must be compiled against the release of Slurm that is going to be deployed on the cluster. Compiling against `spank.h` from a different Slurm release will cause Slurm to prevent pyxis from loading with error `Incompatible plugin version`.

With `make install`

$ sudo make install
$ sudo ln -s /usr/local/share/pyxis/pyxis.conf /etc/slurm/plugstack.conf.d/pyxis.conf
$ sudo systemctl restart slurmd

With a deb package

$ make orig
$ make deb
$ sudo dpkg -i ../nvslurm-plugin-pyxis_*_amd64.deb
$ sudo ln -s /usr/share/pyxis/pyxis.conf /etc/slurm/plugstack.conf.d/pyxis.conf
$ sudo systemctl restart slurmd

With a rpm package

$ make rpm
$ sudo rpm -i rpm/RPMS/x86_64/nvslurm-plugin-pyxis-*.x86_64.rpm
$ sudo ln -s /usr/share/pyxis/pyxis.conf /etc/slurm/plugstack.conf.d/pyxis.conf
$ sudo systemctl restart slurmd

Usage

Pyxis being a SPANK plugin, the new command-line arguments it introduces