Skip to content

API reference

Cross-platform inference

OptionScorer accepts backend="auto" and device="auto". Automatic backend selection uses MLX on Apple silicon and PyTorch elsewhere. For PyTorch, automatic device selection prefers CUDA, then MPS, then CPU.

from openjev import OptionScorer

scorer = OptionScorer("google/gemma-3-4b-it", backend="torch", device="cuda", batch_size=2)
results = scorer.score("The capital of France is", [" Paris", " Berlin"])

Use device="cpu" for CPU inference. The server uses the same options: uv run openjev serve --backend torch --device auto. Both /score and /v1/systemone retain the same request and response contracts across backends. See Getting started for GPU installation instructions.

The openjev package. OptionScorer is re-exported from the top level:

from openjev import OptionScorer

openjev

Local option scoring and trainable candidate-decision heads.

Model backends are loaded only when used, so importing openjev does not initialize an accelerator or download weights.

OptionScorer

Scores pre-written options as continuations of a context with a causal LM.

Parameters:

Name Type Description Default
model_path str

local directory or Hugging Face repo id.

DEFAULT_MODEL
backend str

auto selects MLX on Apple silicon, PyTorch elsewhere.

'auto'
device str

PyTorch device (auto, cpu, cuda, cuda:N, mps).

'auto'
batch_size int

maximum number of options scored in one forward pass.

8
chat bool

wrap the context in Gemma's chat template (user turn, generation prompt appended) so options are scored as the start of the reply.

False
sep str

literal string placed between context and each option (ignored when chat is set, since the template already ends the turn).

''

score_naive(context, options)

Reference: re-encode context + option from scratch per option, no cache.

Returns the sum of option-token log-probs. Used to verify the prefix-shared batched path and to benchmark against it.

openjev.scorer

Zero-shot option scoring with Gemma 3 via MLX or PyTorch (Route B in docs/design/one-pass-option-scoring.md).

The context is prefilled once. Its KV cache is then expanded across the batch dimension so every option is scored in one padded forward pass that shares the context prefix. No decoding happens anywhere; the score is the log-probability the language model assigns to the option tokens given the context.

OptionScorer

Scores pre-written options as continuations of a context with a causal LM.

Parameters:

Name Type Description Default
model_path str

local directory or Hugging Face repo id.

DEFAULT_MODEL
backend str

auto selects MLX on Apple silicon, PyTorch elsewhere.

'auto'
device str

PyTorch device (auto, cpu, cuda, cuda:N, mps).

'auto'
batch_size int

maximum number of options scored in one forward pass.

8
chat bool

wrap the context in Gemma's chat template (user turn, generation prompt appended) so options are scored as the start of the reply.

False
sep str

literal string placed between context and each option (ignored when chat is set, since the template already ends the turn).

''

score_naive(context, options)

Reference: re-encode context + option from scratch per option, no cache.

Returns the sum of option-token log-probs. Used to verify the prefix-shared batched path and to benchmark against it.

openjev.systemone

TypeSafe-compatible System One API (https://docs.typesafe.ai) on top of the local scorer.

POST /v1/systemone {"state": , "model": "...", "questions": {id: Choice|Score|Noul}} -> {"model": "...", "answers": {id: answer}, "usage": {"input_tokens": n, "output_tokens": n}}

Every question is rendered into a plain-text prompt ending in "Answer:\n" and the candidate labels are scored as continuations with one prefix-shared batched forward pass per question. Nothing is generated. Confidence is 1 - normalised entropy of the probability distribution, an approximation of TypeSafe's "how spread out is the distribution" definition.

openjev.server

HTTP server: one loaded model, many scoring requests.

openjev serve --host 0.0.0.0 --port 8000 curl -s localhost:8000/score -H 'content-type: application/json' \ -d '{"context": "The capital of France is", "options": [" Paris", " Berlin"]}'

openjev.features

Frozen-Gemma feature extraction for the trainable head (Route A).

Stops Gemma before the LM head and returns final-layer hidden states. The context is encoded once (all token vectors kept). Options are encoded on their own, batched, and masked-mean pooled to one vector each, exactly as jevlike does: the head must do the matching, and the shuffled-context control stays meaningful. With contextual=True the options are instead run as continuations of the context via the prefix-shared cache; the pooled vectors then already contain the match, which makes the head trivial and the control useless. Features are cached to an .npz so head training never touches Gemma again.

Runs on MLX or, through the scorer's PyTorch engine, on CUDA/ROCm/MPS/CPU (independent options only).

FeatureExtractor

extract(context, options, chat=None, sep=None)

Return (context_h (Lc, H) float16, options_h (N, H) float16).

FeatureSet

In-memory view of a cached .npz: lists of (Lc, H) contexts, (N, H) options, labels.

batch(idx, shuffle_context=False)

Pad a batch as numpy arrays: ctx (B, Lc, H), ctx_mask (B, Lc), opt (B, N, H), opt_mask (B, N), labels (B,).

extract_dataset(scorer, data, out, limit=0, chat=False, sep='', contextual=False)

Cache features for a jevlike JSONL {context, options, label} file into one .npz.

openjev.head

jevlike's AttentionHead, ported to MLX: one cross-attention layer that scores each option against the context tokens. Options are queries, context tokens are keys/values, and the per-option logit is the dot product of the option query with its attended context vector.

openjev.head_torch is the same head in PyTorch with the same interface and weight names. Both take and return numpy arrays at the boundary (forward, Optimizer.step).

AttentionHead

Bases: Module

__call__(ctx, ctx_mask, opt, opt_mask)

ctx (B, Lc, H), ctx_mask (B, Lc), opt (B, N, H), opt_mask (B, N) -> logits (B, N).

Optimizer

AdamW on one head. step(terms) takes [(batch, weights or None, scale, brier_weight)], where batch is numpy (ctx, cm, opt, om, labels), and returns the loss.

forward(head, ctx, cm, opt, om)

Logits (B, N) as float64 numpy, from numpy inputs.

openjev.train

Train and evaluate the head on cached Gemma features (jevlike's train.py / eval.py), on MLX or PyTorch.

evaluate(head, fs, batch_size=64, shuffle_context=False)

top1, top3, 10-bin ECE (on the max probability), n.

openjev.cli

Command line entry points: score, eval, bench, check, serve, features, train, eval-head.

cmd_check(args)

Verify the prefix-shared batched path against naive full re-encoding.