Noload Driver

Noload driver is a driver for the Linux and FreeBSD kernels which supports Noload hardware which have substantial similarities with NVMe but are for acceleration tasks, not block devices.

Work is submitted with ioctls on a character device. The packages install the header that defines them, kmod/inc/noload_ioctl.h, as /usr/include/noload_ioctl.h.

Documentation

• Devices and discovery - the device nodes and what they report.
• Submitting work - the command model, the synchronous ioctl, return values and errors.
• Buffers and data transfer - flat buffers and io vectors, copying against zero-copy, alignment and size limits.
• Asynchronous submission - batched submission and completion, queue depth and waiting with poll.
• Queues, depth and tuning - how work reaches the card's queues, dedicated queues and the module parameters.
• FreeBSD notes - what differs on FreeBSD, and its tunables.

Every ioctl, structure and inline helper is documented in noload_ioctl.h.

Supported kernels

The Linux driver currently supports kernel versions from 5.14 through 6.18. The following operating system kernels are supported:

• Ubuntu 22.04 (Jammy)
• Ubuntu 24.04 (Nobel)
• Debian 12 (Bookworm)
• Debian 13 (Trixie)
• RHEL 9.2
• RHEL 9.4
• RHEL 9.6
• RHEL 9.8
• RHEL 10.0
• RHEL 10.1

A DKMS package is available for easier distribution.

Installation

The steps to install the module are as follows:

1. Install dependencies (Note the libssl library is required to build the example applications in the tools directory.)

   sudo apt install linux-headers-generic libssl-dev

2. Obtain a copy of the source and extract it into a local directory:

   git clone git@github.com:Eideticom/noload-drv.git
   cd noload-drv

3. Build the project

   mkdir build
   cd build
   cmake ..
   make -j$(nproc)

4a. Install the driver and probe the module (from the build directory):

sudo make noload_modules_install
sudo depmod
sudo modprobe noload

4b. Alternatively, use the module without installing (from the build directory):

sudo insmod kmod/build/noload.ko

If compiling the module for a non-standard kernel build, pass the kernel path via -DKDIR= during the cmake step:

cmake -DKDIR=~/path/to/linux/build ..

To let non-root users reach the hardware, install the udev rule and add them to the noload group:

sudo cp tools/71-noload-drv.rules /etc/udev/rules.d/
sudo groupadd -f noload
sudo usermod -aG noload $USER

Module Parameters

The following parameters can be passed when inserting the module:

• max_tags - The number of commands each of the card's io queues holds. The default is 64 and the maximum is 128.
• dedicated_queues - The number of queues per card set aside for applications that ask for one with NOLOAD_IOCTL_SET_QUEUE(). The default is 8.
• file_depth - The number of commands that can be in flight at once on one file descriptor. The default is 8.
• allow_unsupported_controllers - Probe Eideticom devices whose device ID the driver does not recognise, creating a controller device and no namespace devices. Off by default.

These parameters can be set when inserting the module:

sudo insmod noload.ko max_tags=32 dedicated_queues=0 file_depth=16
sudo modprobe noload max_tags=32 dedicated_queues=0 file_depth=16

Queues, depth and tuning covers when to change them. FreeBSD has the same knobs as sysctl hw.noload tunables, listed in FreeBSD notes.

FreeBSD

On FreeBSD the noload driver can either build against a patched in-tree nvme driver or use its own copy. Two build modes are selected with -DNOLOAD_BSD_KERNEL_TARGET:

• patched - build against a patched in-tree nvme driver. This requires patching and rebuilding the kernel, as described below.
• standalone - bundle a renamed private copy of the nvme driver into noload.ko. No kernel changes are required, but the bundled copy must be ABI-compatible with the running kernel.

When -DNOLOAD_BSD_KERNEL_TARGET is not given, cmake probes the kernel source tree for the nvme patch and defaults to patched when it is found and standalone otherwise.

Patching the kernel

kmod/freebsd/freebsd-16-nvme-patch.patch adds the Eideticom support to the in-tree nvme driver. It is generated against FreeBSD 16.0-CURRENT and applies to a matching /usr/src tree.

1. Apply the patch to the kernel source tree:

   git -C /usr/src apply /path/to/noload-drv/kmod/freebsd/freebsd-16-nvme-patch.patch

2. Build and install the patched kernel, then reboot into it:

   make -C /usr/src -j$(nproc) buildkernel KERNCONF=GENERIC
   # may need to run: sudo pkg unregister -y FreeBSD-kernel-generic
   sudo make -C /usr/src installkernel KERNCONF=GENERIC
   sudo shutdown -r now

3. Build and load the module against the patched kernel:

   cmake -B build -DNOLOAD_BSD_KERNEL_TARGET=patched
   make -C build -j$(nproc)
   sudo kldload ./build/noload.ko

To build without touching the kernel, use the standalone target instead (cmake -B build -DNOLOAD_BSD_KERNEL_TARGET=standalone); no patch is needed in that case.

Examples

The following code examples are included to demonstrate methods for integrating noload-drv:

• noload-gzip - A gzip-like CLI tool capable of compression and decompression with either sync or async methods.
• noload-testdrive-blaster - A performance tool for inline-testdrive cores. Demonstrates noload-drv access using a simple accelerator. This tool may be used with inline-testdrive hardware for infrastructure performance bounds validation.

The packages install the example sources under /usr/share/doc/noload-drv/examples/, with a Makefile for them.