Providers

provider id	what	strategy (auto)	key / endpoint	images
hf (transformers)	Hugging Face model in-process	logits (exact)	local GPU/CPU, pip install "polyjev[hf]"	not yet
vllm	vLLM server	logprobs	VLLM_BASE_URL (default http://localhost:8000/v1)	vision models
sglang	SGLang server	logprobs	SGLANG_BASE_URL (:30000)	vision models
llamacpp	llama.cpp llama-server	logprobs	LLAMACPP_BASE_URL (:8080)	multimodal builds
ollama	Ollama (0.12.11+ returns logprobs)	logprobs	OLLAMA_BASE_URL (:11434)	vision models
lmstudio	LM Studio	probed	LMSTUDIO_BASE_URL (:1234)	vision models
openai	OpenAI	probed: GPT-5.x reasoning models → verbalized; models with logprobs → logprobs	OPENAI_API_KEY	yes
anthropic (claude)	Anthropic Claude	verbalized (no logprobs)	ANTHROPIC_API_KEY	yes
gemini (google)	Google Gemini (API key or Vertex AI)	probed	GEMINI_API_KEY / options.vertexai	yes
openrouter	OpenRouter	probed per model	OPENROUTER_API_KEY	per model
together, fireworks, groq, deepseek, mistral, xai	hosted OpenAI-compatible APIs	probed	<PROVIDER>_API_KEY	per model
openai-compatible	anything else speaking Chat Completions	probed	base_url in config	per model
fake	deterministic stand-in for tests and demos	logprobs (fake/verbalized, fake/vote for the others)	none	yes

"Probed" means polyjev sends one tiny logprobs request on first use and remembers the answer. polyjev probe <model> shows the result.

Per-model settings

Everything provider-specific goes in the model's config entry:

models:
  qwen:
    provider: vllm
    model: local
    base_url: http://gpu-node:8000/v1
    extra_body: {chat_template_kwargs: {enable_thinking: false}}   # sent as-is in the request body
    max_concurrency: 64                                             # in-flight requests to this model
  gpt:
    provider: openai
    model: gpt-5.6-luna
    params: {reasoning_effort: low}                                 # extra top-level request params
    max_tokens: 4096                                                # budget for verbalized reads
  claude:
    provider: anthropic
    model: claude-opus-5
    params: {output_config: {effort: low}}
    options: {fallbacks: true, cache: true}
  gemini-vertex:
    provider: gemini
    model: gemini-3.8-flash
    options: {vertexai: true, project: my-project, location: us-central1}
  local-hf:
    provider: hf
    model: Qwen/Qwen3-4B-Instruct-2507
    options: {device_map: cuda, dtype: bfloat16, chat_template_kwargs: {enable_thinking: false}}

Keys can be given inline (api_key), by variable name (api_key_env: MY_KEY), or through the provider's usual environment variable.

Notes per provider

Thinking models on vLLM / SGLang. For logprob reads the first generated token must be the answer label. Turn thinking off with extra_body: {chat_template_kwargs: {enable_thinking: false}} (Qwen3 and friends), or use an instruct model such as Qwen/Qwen3-4B-Instruct-2507. If a model still thinks first, polyjev notices (no label in the top logprobs) and reads that question verbalized.

Claude. Claude has no logprobs, so reads are verbalized through structured output. The state block is cached, so a many-question decision pays for the state roughly once. For claude-opus-5 and claude-fable-5-1, refusal fallbacks are on by default (fallbacks: "default", beta server-side-fallback-2026-07-01); set options: {fallbacks: false} to turn them off. A refusal that survives raises RefusalError.

OpenAI. GPT-5.x reasoning models do not accept logprobs or temperature. polyjev drops rejected parameters, remembers that, and reads verbalized.

Gemini. Logprob support differs by model, so it is probed. Thinking tokens count against the output budget. When a logprob read finds no label, polyjev falls back to verbalized on its own.

Your own provider

import polyjev as pj
from polyjev.backends import Backend, Capabilities, Completion, CompletionRequest

class MyBackend(Backend):
    provider = "mine"

    def __init__(self, model: str):
        super().__init__(model)
        self.capabilities = Capabilities(logprobs=False, json_schema=True)

    async def complete(self, req: CompletionRequest) -> Completion:
        text = await call_my_api(req.system, req.conversation, max_tokens=req.max_tokens, schema=req.json_schema)
        return Completion(text=text)

pj.register_backend("mine", lambda cfg: MyBackend(cfg.model))
pj.Polyjev("mine/some-model").decide(...)

Packages can also register a provider through the polyjev.backends entry-point group.