v3.1 development notes

A working log of non-obvious things that came up while landing v3.1 (DVAI Hub + library finalization). These are operational and design discoveries that didn't fit cleanly into the user-facing migration guide or the distributed-inference guide but are worth preserving for anyone digging into the v3.0 → v3.1 transition.

Two pairing stores: how core + Hub share keys

The v3.1 Hub introduces MultiTenantPairing — a per-app pairing store at ~/.dvai-hub/apps/<appId>/pairings.json. The core library ships its own PairingStore (InMemoryPairingStore in browsers, NodeFsPairingStore at %LOCALAPPDATA%/dvai-bridge/ / ~/.cache/dvai-bridge/ in Node).

Both stores hold a pairingKey per peer. If they generate keys independently, every HMAC-signed request fails verification because the requester signs with the key core's PairingPolicy echoes (in the handshake response) but the verification path at the Hub looks up the key in MultiTenantPairing (which generated a different one).

The fix in b783537 changes the PairingPolicy.onPairingRequest return type from Promise<boolean> to a tagged union:

Promise<
  | boolean                                 // backwards compat
  | { approved: true; pairingKey: string }  // host has its own key
  | { approved: false }                     // denied
>

When a host (the Hub) returns { approved: true, pairingKey }, PairingPolicy uses that key directly instead of generating a fresh one. Both stores end up agreeing.

The same widening is mirrored on OffloadConfig.onPairingRequest so application-level callbacks can opt in too.

Tauri 2 dev: .env doesn't propagate to spawned sidecars

pnpm tauri dev reads .env files for its own environment, but the env vars don't automatically reach a child process the Rust shell spawns via tokio::process::Command. Particularly painful on Windows where PowerShell's DVAI_HUB_PREFER_BETTER_QUANT=1 pnpm tauri dev (bash syntax) silently does nothing — PowerShell needs $env:DVAI_HUB_PREFER_BETTER_QUANT="1"; pnpm tauri dev.

The fix in e981040 is to load .env ourselves at sidecar startup:

// Top of hub/peer-mode/server.ts — runs before any other imports
function loadDotenv(): void {
  const candidates = [
    process.env.DVAI_HUB_ENV_FILE,           // explicit override
    path.join(__dirname, "..", "..", ".env"),
    path.join(__dirname, "..", "..", "src-tauri", ".env"),
  ].filter(Boolean);
  for (const candidate of candidates) {
    try {
      const raw = readFileSync(candidate, "utf8");
      for (const line of raw.split(/\r?\n/)) {
        // parse KEY=VALUE, skip comments + empty lines, set process.env
        // …  (existing process.env values WIN — no override)
      }
      return;  // first hit wins
    } catch { continue; }
  }
}
loadDotenv();

Existing process.env values take precedence over the file, so explicit env vars on the command line still override.

Tauri 2: app.exit(0) vs prevent_exit

The Hub keeps itself alive when the dashboard window closes (tray icon stays, peer-mode keeps serving). The intuitive way to do that is to handle RunEvent::ExitRequested and call api.prevent_exit() unconditionally — but that breaks the tray's "Quit DVAI Hub" menu because app.exit(0) also fires ExitRequested.

RunEvent::ExitRequested carries code: Option<i32> that distinguishes the two:

code	Source	What we want
None	OS close signal (window X click, last-window-closed)	Prevent exit; keep Hub in tray
Some(n)	app.exit(n) was called	Honor the request

The fix in 66c190d:

.run(|_app_handle, event| {
    if let tauri::RunEvent::ExitRequested { code, api, .. } = &event {
        if code.is_none() {
            api.prevent_exit();
        }
    }
})

Hop-by-hop headers when proxying responses

The Hub's chatCompletionInterceptor forwards responses from external engines (Ollama, LM Studio) back to the requesting peer. Naively forwarding the engine's response headers verbatim breaks the connection: engines emit transfer-encoding: chunked and connection: keep-alive, but the Hub re-emits the body via Response.json() (NOT chunked). Mismatched framing causes Node fetch on the client side to fail mid-stream with terminated.

RFC 2616 §13.5.1 lists the hop-by-hop headers that describe the immediate connection rather than the message:

Connection
Keep-Alive
Proxy-Authenticate
Proxy-Authorization
TE
Trailers
Transfer-Encoding
Upgrade

Plus Content-Length (the runtime re-derives this; forwarding a stale value is wrong). Fix in e981040 filters all of those out before forwarding.

TransformersBackend in Node: cpu is not wasm

A v3.0 surprise. The library's TransformersBackend mapped device: "cpu" to ONNX runtime device "wasm" "for Transformers.js v3/v4 compat". That works in browsers (onnxruntime-web accepts wasm) but throws in Node (onnxruntime-node 1.24.x rejects wasm with Unsupported device: "wasm". Should be one of: dml, webgpu, cpu).

Fix in 5ed6007 detects the runtime and routes appropriately:

const isNode =
  typeof window === "undefined" &&
  typeof process !== "undefined" &&
  process.versions?.node !== undefined;

if (this.device === "auto") {
  const hasWebGPU = await detectWebGPU();
  this.resolvedDevice = hasWebGPU ? "webgpu" : isNode ? "cpu" : "wasm";
} else if (this.device === "cpu") {
  this.resolvedDevice = isNode ? "cpu" : "wasm";
} else {
  this.resolvedDevice = this.device;
}

Browser consumers see no change. Node consumers (the Hub, the node-langchain example, anyone running DVAI on Node 22+) needed this for the Transformers backend to initialize at all.

/v1/dvai/* routes weren't dispatched in v3.0

The v3.0 codebase exported buildDvaiRoutes() from handlers/dvai/ index.ts — a complete map of GET /v1/dvai/health, POST /v1/dvai/handshake, etc. — but no transport ever called it. Every /v1/dvai/* request returned {"error":"not found"}. Pairing handshakes couldn't actually pair.

Fix in 22afa5a:

1. Add dvaiRoutes?: Record<string, DvaiHandler> to HandlerContext as a late-bound getter (offload state initializes AFTER the transport starts in the current lifecycle, so the routes can't be set eagerly).
2. HTTP transport's route() reads ctx.dvaiRoutes per request and dispatches by ${method} ${path} lookup.
3. DVAI.initializeOffload() builds the map from the Hub's discovery / pairing / capability / backend collaborators.

If you wrote a v3.0 consumer that depended on the /v1/dvai/* routes returning JSON, this is the fix that makes it work. v3.0 behaviour was always silently broken on this surface.

Substitution policy: family: "unknown" is a sentinel

Surfaced via live E2E. The model-name parser uses "unknown" as a sentinel for "couldn't classify this model name." A v3.1 Hub user's custom Ollama-cached model links:v7B parses to family: "unknown". A phone request for completely-fictional-model ALSO parses to family: "unknown". Without a guard, the substitution policy treated "unknown" === "unknown" as a real same-shape match and routed completely-fictional-model to links:v7B.

Fix in 0553e56 short-circuits at the top of SubstitutionPolicy.pick():

if (request.family === "unknown") {
  return {
    kind: "refuse",
    reason: "family_mismatch",
    detail: "Request model could not be classified (family=unknown). Use a recognized model id.",
  };
}

This only checks the request side. A backend with family: "unknown" is still serviceable when a request explicitly names its engineModelId (via the exact-match string-equality check).

Other unknown sentinels (size: "unknown", type: "unknown") stay permissive — those have legitimate "field absent in the model name" uses (e.g. gemma3:latest).

The Aallam openai-kotlin Ktor-version foot-gun

When porting the Android example to call the Hub, the obvious choice was com.aallam.openai:openai-client (the idiomatic OpenAI Kotlin SDK). Two failure modes surfaced:

1. Silent no-op. openai-client:4.0.1 paired with ktor-client-okhttp:2.3.13 causes the engine to fail to install silently — coroutines swallow the exception and the request never fires. No error log. No exception. Just nothing in the audit log.
2. SLF4J no-op logger. When Ktor logs to SLF4J without a provider on the classpath, useful diagnostic output disappears.

The pragmatic fix for the example app: bypass the OpenAI client and use raw OkHttp + org.json. Two-thirds the code, none of the version mismatch surface area:

val payload = JSONObject().apply {
    put("model", modelId)
    put("messages", JSONArray().apply { … })
    put("stream", false)
}.toString()

val request = Request.Builder()
    .url("$HUB_BASE_URL/v1/chat/completions")
    .post(payload.toRequestBody("application/json".toMediaType()))
    .build()

val res = httpClient.newCall(request).execute()

If you want the higher-level OpenAI Kotlin surface, pin openai-client to a version compatible with whatever ktor-client-* your shared-core dep brings in transitively, OR upgrade the engine to match. Mixing 2.3.x and 4.x doesn't work and doesn't tell you why.

Single-instance lock vs. duplicate tray icons

Tauri 2's tauri-plugin-single-instance prevents a second dvai- hub.exe from binding the named pipe / Unix socket — that's how re-launching the app surfaces the existing window instead of spawning a parallel process. But it doesn't prevent declaring a tray icon TWICE at app startup.

The Hub's tauri.conf.json originally had a trayIcon block AND tray.rs called TrayIconBuilder::with_id(...). Tauri 2 doesn't dedupe across the two paths, so every launch ended up with two tray icons in the system tray — one from the config (no menu — Tauri only attaches menus via the Rust API, not declaratively) and one from the Rust code (with the proper menu). Both forwarded clicks to the same window-toggle handler.

Fix in c08340f drops the config-side registration. Either path works on its own; mixing them does not.

See also

• Migration: v3.0 → v3.1
• Distributed inference
• Distributed inference testing
• DVAI Hub user guide