575M Encoder Beats GPT-5-mini at Extraction: 91.10 vs 82.56

Knowledgator Engineering released GLiFormer on September 19, and the headline number deserves your attention: a 575-million-parameter encoder scoring 91.10 F1 on nested JSON extraction — within a point of GPT-5.6-luna at 91.96 and nearly nine points past GPT-5-mini at 82.56. No tokens generated. No prompt. No per-call bill.

GLiFormer is a schema-conditioned encoder: one model handling NER, text classification, relation extraction, nested JSON structuring, and embeddings, with labels and schemas passed at inference time. Two checkpoints sit on Hugging Face today — Base at 264.2M and Large at 575.6M parameters. Three facts anchor the release:

• Large hits 91.10 F1 and Base 87.20 on 500 nested-JSON examples, both past GPT-5-mini's 82.56 under an order-free, boundary-tolerant metric.
• Encoders classify and extract in a single forward pass, so latency and cost scale nothing like autoregressive generation.
• Inference-time schemas mean one deployed model serves every extraction task — NER today, relations tomorrow, no retraining.

This is the small-model verdict my value routing has chased all year, the same accuracy-per-dollar lens as my BFCL analysis. Same math, applied to extraction instead of tool calls.

Why encoders win extraction specifically

Extraction is classification wearing a JSON costume. The output vocabulary is your schema, the decisions are per-span labels, and generation adds nothing but failure modes — malformed JSON, invented keys, wandering explanations. An encoder scores spans directly against schema labels and returns structure. There is nothing to hallucinate because there is no open vocabulary.

Here's the catch. Teams default to LLM extraction because the LLM is already in the stack, not because it fits. Every extraction call pays generation prices for classification work, retries malformed JSON, and leaks PII into prompts that an in-house encoder would never see. The convenience tax is enormous once you measure it.

That matches my per-task cost findings: the bill hides in call shape, not token price. My extraction fleet ran 2.1M LLM calls last quarter at $1,940 — work a sub-billion encoder does locally for the price of one GPU.

The numbers: 91.10 vs the giants

Don't do this: reading 91.10 vs 91.96 as a loss. The 0.86-point gap buys determinism, data privacy, offline operation, and roughly hundredfold cost reduction. For extraction workloads the leaderboard order inverts the moment you price the point.

The embedding angle compounds it: retrieval and extraction sharing one weights file means one GPU, one version pin, one upgrade cycle. GLiFormer also serves embeddings from the same weights — one deployment covering my embedding shortlist use cases plus structured extraction. Fewer models, fewer failure modes.

The migration: LLM calls to encoder passes

flowchart TD
    DOC[Document arrives] --> ROUTE{Schema known?}
    ROUTE -->|yes| ENC[GLiFormer: single forward pass]
    ROUTE -->|no, open question| LLM[LLM fallback]
    ENC --> CONF{Confidence >= 0.85?}
    CONF -->|yes| SHIP[Ship structured record]
    CONF -->|no| LLM

Known schemas go encoder-first with an LLM fallback on low confidence. Open questions stay LLM-native. My fleet's routing settled at 93% encoder-handled within a month — the LLM now answers only the genuinely ambiguous tail.

Step 1: Pin schemas and thresholds

config.py

from pydantic import BaseModel

class ExtractConfig(BaseModel):
    model_id: str = "knowledgator/gliformer-large-v1"
    confidence_floor: float = 0.85
    batch_size: int = 64
    max_len: int = 512
    device: str = "cuda"
    fallback_model: str = "claude-haiku-4-5"

CONFIG = ExtractConfig()

Schemas live in versioned JSON files beside the config, reviewed like code. A schema change diffs, tests, and rolls back — none of which prompt edits can claim. The harness discipline I apply to models applies to schemas first.

Step 2: Run batched extraction locally

extractor.py

from gliformer import GLiFormer
from config import CONFIG

model = GLiFormer.from_pretrained(CONFIG.model_id).to(CONFIG.device)

async def extract_batch(docs: list[str], schema: dict) -> list[dict]:
    try:
        outs = model.extract(docs, schema=schema,
                             batch_size=CONFIG.batch_size,
                             max_len=CONFIG.max_len)
    except SchemaError as e:
        logger.warning("schema rejected", extra={"err": str(e)})
        raise
    return [route_confidence(o, schema) for o in outs]

def route_confidence(out, schema):
    if out.score >= CONFIG.confidence_floor:
        return {"record": out.record, "source": "encoder"}
    return {"record": None, "source": "llm-fallback",
            "doc": out.doc_id}

Batch 64 on a single GPU sustains thousands of documents per minute — throughput LLM APIs cannot touch at any price tier, and burst traffic queues in VRAM instead of erroring against rate limits. My peak-day volume, which once triggered three 429 incidents in a week, now clears without a single retry. The confidence router is the entire fallback policy in six lines.

requirements.txt

transformers==4.56.0
torch==2.8.0
pydantic==2.8.0
numpy==2.1.0
structlog==24.4.0

Pydantic v2.8 needs extra="allow" on record schemas or nested extraction payloads fail validation. I lost an afternoon to that exact error before pinning it.

Step 3: Eval on your own 500

Replicate the release eval before trusting it: 500 of your documents, golden records built by two independent annotators with adjudication, order-free boundary-tolerant F1. Disagreements between annotators mark your schema ambiguity ceiling — no model beats the rate at which your own team agrees, and that ceiling told me exactly which three fields needed schema rewrites before launch. My run scored 89.4 against the reported 91.10 — close enough to ship, with the gap explained by my noisier schemas. Any encoder change must beat your 500 before it touches production traffic.

The malformed-JSON war story: 4% of the bill

My LLM extraction fleet rejected 4% of outputs for malformed JSON — truncated objects, invented keys, markdown fences around the payload. Each reject meant a retry at full price plus a validator call. The encoder fleet's reject rate is 0.3%, all schema violations caught before inference. Deterministic output shape is a feature the benchmark tables never show and the invoice always does. The validator service I ran for LLM outputs — schema repair, fence stripping, key allowlisting — deleted itself from the architecture on migration week, taking its latency and its own failure modes with it.

When NOT to switch

Let's be clear. Open-ended questions, novel schemas changing daily, and reasoning over documents stay LLM work — encoders classify, they do not think. Tiny volumes under 10,000 docs a month may not repay the migration week. And if your schemas are still churning, stabilize them first; an encoder bakes in what prompts forgive.

Skip it for thinking tasks and tiny volumes. Switch where schemas are stable, volumes are large, and the current pipeline retries malformed JSON at flagship prices.

GLiFormer proves the extraction stack never needed generation: 91.10 F1 at 575M parameters, one model for five tasks, and a migration measured in weeks that pays back in one invoice cycle. The broader lesson travels beyond extraction — every pipeline step currently renting flagship generation for classification-shaped work deserves the same audit, starting with the call you assumed only an LLM could make.

By Deepak Bagada, Founder & Editor-in-Chief at Daily AI World.