My flow named its encoder, and the Intel node paid for it

This is my homelab: one Tdarr server, an AV1 flow, and GPU nodes from two vendors pulling from the same queue. Tdarr 2.50.01, FFmpeg 7.1.2-Jellyfin, and the Set Video Encoder plugin at 1.0.0 with a helper file that is upstream plus one line I patched in July. Everything below is about that installed revision, not every release.

My NVIDIA nodes had been running the flow for weeks. When the Intel node joined and picked up the same flow, every job on it died with Cannot load libcuda: an NVIDIA runtime error on a machine with no NVIDIA card. Card, driver and registration were all fine, and I spent the first hour on the layer that was working.

The flow's Set Video Encoder step stored nvenc. I had set it. The installed plugin's default for that field is auto, so creating a flow does not capture the first card; I had pinned it and forgotten.

What a pinned value does in this revision

Reading the installed helper: it holds a fixed candidate list, filters it to the target codec, and when hardwareType is anything but auto it returns the entry whose name contains that vendor string without probing it. The only error it raises is when the vendor is not in the list at all. So my job was handed an NVIDIA encoder on a node that could not load the NVIDIA runtime, and it failed inside ffmpeg, which is why the message looked like a driver problem.

# Read-only, inside the server's cluster: what each flow's encoder step actually stores.
TDARR="http://tdarr.media.svc:8265"
curl --fail --silent --max-time 10 -X POST "$TDARR/api/v2/cruddb" \
  -H 'Content-Type: application/json' \
  -d '{"data":{"collection":"FlowsJSONDB","mode":"getAll"}}' \
  | jq -r '.[] | .flowPlugins[] | select(.pluginName == "ffmpegCommandSetVideoEncoder")
           | [.name, .inputsDB.hardwareType, .inputsDB.hardwareEncoding, .inputsDB.hardwareDecoding] | @tsv'

What auto does in this revision

With auto, and only when the worker is a GPU worker, hardware encoding is enabled and the codec is one it supports, the helper probes each AV1 candidate in list order: av1_nvenc, av1_amf, av1_qsv, av1_vaapi. Each probe is a one-second 512 by 512 black synthetic source encoded to a null output. The first candidate that passes wins. That is list order, not a quality or speed comparison. An NVENC winner then goes through a device-selection step that reads nvidia-smi. If nothing passes, or the conditions above are not met, the AV1 fallback is the libsvtav1 software encoder. Hardware decoding is a separate switch and is not decided by this probe.

On 2026-09-06 the Intel node's job report showed both av1_qsv and av1_vaapi passing, and Quick Sync chosen because it comes first. I had written that node's manifest for the VAAPI path and assumed that was what ran. A one-second black frame passing says the encoder initialises on that device; it says nothing about every resolution, bit depth, filter chain or concurrency the real files will bring. The VAAPI candidates are given the first renderD device the helper finds under /dev/dri, so on a multi-GPU node that is one card considered for VAAPI; the Quick Sync entries carry no device path and leave selection to the runtime.

The local patch is in that same helper, hardwareUtils.js, at line 246, where the VAAPI probe's filter is built. Upstream has -vf format=nv12,hwupload; mine reads -vf format=nv12|vaapi,hwupload, so a frame already on the VAAPI device passes the upload step instead of failing it. Against upstream master as fetched on 2026-09-06 that is the only difference. The installed file's SHA-256 begins 7ec4f33a, upstream's 0bc799b8; the plugin directory is versioned 1.0.0 and the file carries no finer revision, so that pair of digests is the identification.

Verify from the job report

The server keeps one report per job. It names the node and worker, logs each probe result, and records the exact spawnArgs handed to ffmpeg. A report line Encoder av1_qsv is enabled followed by spawnArgs containing -c:0 av1_qsv is the requested configuration; the report's own result line says whether it completed. This selects the newest transcode report for one node and prints the argument record with media paths masked, an excerpt of the report rather than all of it:

# Read-only, inside the server pod.
NODE="elfastc-b580"
R="/app/server/Tdarr/DB2/JobReports"
[ -d "$R" ] || { echo "no report directory at $R"; exit 1; }
hit=""
for d in $(ls -t "$R" | head -400); do
  f=$(grep -lsE "Node\[$NODE\]:Worker\[[^]]+\]:\{\"workerType\":\"transcodegpu\"" "$R/$d"/* | head -1)
  [ -n "$f" ] && { hit="$f"; break; }
done
[ -n "$hit" ] || { echo "no transcode report for $NODE in the newest 400"; exit 1; }
grep -oE 'Encoder av1_[a-z]+ is (enabled|disabled)' "$hit"
grep -oE '"spawnArgs":\[[^]]*\]' "$hit" | head -1 | sed -E 's#/data/[^"]+#/data/...#g; s#/temp/[^"]+#/temp/...#g'
grep -oE 'Transcode (success|error)' "$hit" | tail -1

My first diagnostic for this was a grep of the process list for av1_ names. It could not show libsvtav1, and it could not tell a probe from an encode. If you still want a live view of what is running, this is a heuristic, and labelled as one: you name the pod, it prints a role guess, every codec flag on the line with copy omitted, and the full command so you can read what it guessed from.

# Read-only. Heuristic: the role is guessed from the arguments, and the codec column lists every
# -c flag on the line, audio included. Read the full command before trusting either.
CTX="admin@elfastc"
POD="tdarr-node-dd58bd88b-s5bd4"   # kubectl -n media get pod -l app.kubernetes.io/name=tdarr-node
kubectl --context "$CTX" -n media exec "$POD" -c tdarr-node -- sh -c '
  ps -eo args 2>/dev/null | grep -E "(^|/)(tdarr-)?ffmpeg( |$)" | grep -v grep > /tmp/ff.txt || true
  n=$(wc -l < /tmp/ff.txt)
  [ "$n" -eq 0 ] && { echo "no ffmpeg process found (or ps failed)"; exit 0; }
  while IFS= read -r a; do
    role=unclassified
    case "$a" in *" -f lavfi "*|*" -f null "*) role=probe ;; esac
    [ "$role" = unclassified ] && case "$a" in *" -i /data/"*) role=encode ;; esac
    encs=$(printf "%s" "$a" | grep -oE -- "(-c:v(:[0-9]+)?|-c:[0-9]+|-vcodec|-codec:v) [A-Za-z0-9_-]+" | awk "{print \$2}" | grep -vE "^copy$" | paste -sd, -)
    printf "%s\t%s\t%s\n" "$role" "${encs:-none}" "$a"
  done < /tmp/ff.txt'

I tested that classifier against synthetic lines covering an NVENC encode, a Quick Sync encode, a libsvtav1 encode, two probes, a multi-stream command and a non-media input. It classified all of them, and on the multi-stream line it listed an audio codec beside the video ones, which is why the column is labelled every flag rather than the video encoder. A command line shows what was requested. Completion is the report's result line, and the output's stream metadata says which encoder wrote the file, though it cannot tell two AV1 encoders apart. A software encode is not automatically a fault: hardware encoding switched off, a CPU worker, or a deliberate fallback all produce one.

The plumbing below the flow, as it is here

My NVIDIA nodes set runtimeClassName: nvidia and claim an nvidia.com/gpu resource from the device plugin. The RuntimeClass selects the runtime handler that injects the driver libraries; the reservation is the resource claim. The Intel node claims a gpu.intel.com/xe resource from Intel's plugin, which injects the render device. When I once mounted /dev/dri by hand instead, the calls that report card state failed uninitialised on that node. I did not chase whether permissions or the missing injection caused it, so that is one observation, not a rule against manual device access.

Quality flags in this revision: -qp for GPU encoders, -global_quality for hevc_qsv, -crf on the CPU path, and no preset argument at all for AV1, which my job reports confirm with -qp 30 and nothing else. Equal numbers across vendors do not mean equal quality.

For one shared flow across mixed nodes, auto worked for me because each node can answer for itself. Pinning is the right choice when you want one encoder for quality or repeatability and every node has it. What broke mine was a pinned value I had forgotten I set, running where it could not.

Observed in this lab: one AV1 flow draining one queue across NVIDIA and Intel nodes, Tdarr 2.50.01, checked 2026-09-06. Revised 2026-09-06.