wfzyx/von: The open-source System One decision model. Sub-15ms, non-autoregressive, local drop-in alternative to TypeSafe Jev.

The open-source System One decision model. Sub-15ms, non-autoregressive, local drop-in alternative to TypeSafe Jev. - wfzyx/von

Summary

Von is an open-source decision model that classifies, judges, and scores text or JSON states in one forward pass through a 395M ModernBERT encoder, instead of generating tokens. It implements the TypeSafe /v1/systemone wire protocol, acting as a local drop-in for the closed-source TypeSafe Jev. Apache-2.0 licensed, it ships Python and TypeScript SDKs and a Docker image that run on CPU (OpenVINO), CUDA, ROCm, and Apple MPS.

What it is useful for

Von fits routing, triage, and severity work where you need fast, calibrated outputs without autoregressive generation. The README shows it classifying an incident's root cause from a log line, judging whether a connection-pool error blocks customers, and rating memory pressure on a Nominal-to-Critical scale. Because each call is one encoder pass, it enables low-latency local serving — the README reports a remeasured p50 of 0.096 s on a 4-vCPU Xeon under OpenVINO, and 0.023 s on an A10G. The confidence_gate helper splits answers into automatic and escalate at a measured default threshold of 0.80, useful for cascading: a Von-first pass can defer low-confidence items to a larger model rather than replacing one.

How engineers can use it

Install the SDK (pip install von-sdk, uv add von-sdk, or bun add von-sdk) or run the container (docker run --rm -p 8000:8000 -v von-hf:/data/huggingface ghcr.io/wfzyx/von:cpu). The roughly 3 GB weights are pulled from the Hugging Face Hub into HF_HOME on first use. From Python, call von.decide(state=..., choices={...}, instructions=...) for a Choice, von.judge(...) for a Noul, or von.rate(...) for a Score; von.system_one(...) answers several questions over one state in a single forward pass. The server is started with von serve --host 0.0.0.0 --port 8000 and exposes /v1/systemone, so any client already talking to Jev works by swapping the base URL, or VON_BASE_URL for the SDKs. von calibrate labels.jsonl refits the confidence map on your own labels. Read the documented caveats before relying on outputs: Von is English-only, judge without criteria is the weakest path, and the README warns that out-of-domain ranking cannot be rescued by a confidence threshold — only labels and a refit can. The model card and docs/finetune.md cover fine-tuning and the full benchmark record.