Contrastive-LM/CLM

Contribute to Contrastive-LM/CLM development by creating an account on GitHub.

Summary

Contrastive-LM/CLM is a Python package serving the CLM-8B decision model, which pairs states with actions using two encoders trained with a contrastive objective. It exposes a typed-question API (Noul/Choice/Score) and a ranking endpoint, intended for routing, tool-calling, or verifier roles inside agent loops. The reference implementation runs a FastAPI server (clm-serve) backed by a vLLM Qwen3-8B embedding endpoint with cached state and action vectors.

What it is useful for

CLM scores candidates against a state using a frozen LLM encoder plus a small trainable projection head, returning a softmax over options. Uses the README targets:

• Routing requests via Choice questions, e.g. picking a department from descriptions.
• A boolean Noul filter (probability the statement is true).
• An ordinal Score, e.g. customer frustration on a 0..2 scale.
• A best-of-N verifier for code trajectories; the README reports 87.6% on Terminal-Bench 2.1 and 81.6% on DeepSWE after fine-tuning.
• A real-time game/UI loop, demonstrated on the Chrome T-Rex runner in examples/t_rex.

State and action embeddings are cached independently in a reserved device-memory arena, so repeated candidates cost a dot product rather than a re-encode. Revisited states drop to ~0.6 ms on an RTX 4090 per the README's measurements.

How engineers can use it

Install with pip install contrastive-lm, or pip install -e . from a clone. The serve path the README documents is two processes: a vLLM pooling endpoint for Qwen3-8B embeddings on port 8090, then clm-serve on port 8700, which downloads a ~75 MB reference head on first run.

A documented client call:

from clm import CLMClient, Choice
client = CLMClient()
r = client.system_one(
    state="Customer: my invoice was charged twice...",
    questions={"department": Choice(instructions="Which team should handle this?",
        criteria={"billing": "Charges, invoices, refunds"})})
print(r.answers["department"].choice, r.answers["department"].probabilities)

For in-process use without a server, Engine exposes .answer(...) and .rank(...) against an embeddings endpoint. A playground at http://localhost:8700/ (or --no-ui for API-only) lets you iterate and copy out curl/Python snippets; remote boxes can be tunneled with ssh -L 8700:localhost:8700 <host>. Authentication uses CLM_API_KEY; CORS is off by default.

Documented limitations to plan around:

• Default context is 2048 tokens; longer states are truncated unless you raise vllm serve --max-model-len and clm-serve --max-tokens together, costing more GPU memory.
• A head is tied to the encoder and pooling mode it was trained with — the README warns swapping encoders breaks a checkpoint.
• The reference head uses Qwen3-8B with last-token pooling; --device cpu|cuda controls placement.

For fine-tuning on a custom task, docs/FINETUNING.md and the README provide commands to reproduce the DeepSWE held-out result and to train a typed-decisions head on the 20M-parameter projection heads with the encoder kept frozen.