Sovereign AI

mh-sovereign-small

Qwen3.5-4B Q4_K_M (unsloth GGUF). Context 16,384 tokens. The fast model for classification, extraction and short answers. Measured 2026-10-12: 38–40 tokens per second on the GPU.

mh-sovereign-reason

Qwen3.5-9B UD-Q4_K_XL (unsloth GGUF). Context 32,768 tokens. The default model (alias mh-sovereign): instruction following, reasoning with a thinking budget, drafting. Measured 2026-10-12: 24.6 tokens per second single-stream with thinking off, time to first token 0.3–0.9 s, 14 of 14 task classes passed.

mh-sovereign-lingua

Gemma-4-12B QAT Q4_K_XL (unsloth GGUF). Context 20,480 tokens. The model for multilingual drafting in the major European languages. Measured 2026-10-12: 17.2 tokens per second, 14 of 14 task classes passed.

Chat completions

POST /v1/chat/completions in the OpenAI-compatible format. stream, tools and response_format are passed to the llama.cpp runtime unchanged. Exercised by the three-hour soak of 2026-10-12: 1,617 requests, zero server errors.

Streaming

Server-Sent Events relay; with stream_options.include_usage the final chunk carries the token counts of the request.

Model discovery

GET /v1/models lists the served models and the default alias.

Health and metrics

/healthz, /health/full and a Prometheus /metrics endpoint on the gateway. The public health route of the platform is GET /health on the control plane.

Audit — metadata only

One line per request: request id, model, status, prompt tokens, completion tokens, milliseconds. Never the prompt, never the completion.

Chat and instruction following

The 9B and the 12B model passed all 14 task classes of the acceptance set on 2026-10-12: summaries, rewrites, structured answers, instructions with constraints.

Reasoning with a thinking budget

The 9B model reasons before it answers when thinking is enabled for the request (budget 1,024 tokens). Measured: tasks involving money, prices or quantities need it; without it, arithmetic errors are more frequent.

Multilingual generation

Major European languages are handled by the 9B and the 12B model. Measured limits: no served model writes publishable Estonian; Latvian is usable only from the 9B model and with editing.

Structured output (JSON)

response_format with json_object or json_schema is passed to the runtime unchanged. Works; not separately measured for conformance rates.

Tool calling

tools and tool_choice are passed to the runtime unchanged. Works; not separately measured for call accuracy.

Embeddings

Not served. A text-embedding model is on the plan for the second phase.

Image and audio input

Not served. The platform generates text from text.

Document drafting and summarisation inside the EU

Contracts, reports, correspondence: summarised, rewritten or drafted without the text leaving the European Union or being stored anywhere.

Support assistants over your own knowledge

A first-line assistant that answers from the documents you pass in the prompt, with structured output for your ticketing system.

Classification and extraction pipelines

Routing, tagging, field extraction into JSON at the speed of the 4B model; the 9B model where judgement is needed.

Multilingual drafts

Drafts in the major European languages from one source text; Estonian and Latvian need a human editor — measured, and stated.

Code assistance

Explanations, reviews and small changes on code you control, with the request content never retained.

Rate and size

120 requests per minute globally, a queue of 16, request body up to 4 MiB, timeout 600 seconds.

One model resident

One model is loaded at a time and is unloaded after 120 seconds idle; switching between models costs a load.

Shared machine, governed

The GPU is shared with media production under a governor (GPU norm 34 %); while a media job holds the lock the gateway refuses with an honest busy state instead of degrading silently.

Per-tenant limits and budgets

Limits and budgets per organisation and project, enforced by the control plane before a request reaches the inference plane.