Liquid AI LFM2.5-2.6B: Run Agents Fully On-Device with 128K Context

By Deepak Bagada, CEO at SaaSNext & AI Principal Architect.

For three years the agent economy has been quietly bound to a cloud contract: to get tool-calling, multi-step reasoning, and a real context window, you rent someone else's GPU. Liquid AI's LFM2.5-2.6B breaks that handcuff. At 2.69 billion parameters, with native tool calling, a 128K context window, and roughly 34 trillion training tokens, it is the first small model whose numbers resemble an agent chassis rather than a toy:

• ~220 tokens/sec on an Apple M5 Max
• ~30 tokens/sec on a phone
• under 2.5 GB of memory footprint when quantized
• native, JSON-constrained tool calling tuned into the base sequence
• 128K context — enough for a full document-repair loop, a codebase slice, or a long conversation with role history

This is not the "40GB model squeezed under 3GB via quantization" story; it is a model born small on a huge training diet, which is a decisively different quality trade. Below I compare it against Qwen3.5-9B (the other 2026 default for mid-range on-device agents), walk local deployment, and — because the decision is written in unit economics — put the deployment cost savings where we can all read them.

Why LFM2.5-2.6B matters now

The on-device agent market has polarized: too-small (1–2B) models struggle with reliable tool calls; too-big (7–9B) models exceed phone and laptop memory budgets and throttle to uncomfortable speeds. Liquid's bet is a sub-3B model with 128K context and tool calling baked into the training recipe — the first model to occupy the "just enough" band for real agent work.

The 220 tok/s number matters less than the geography of the result: the entire agent lives on the device. No network hop. No leak window. No per-token metering from the cloud. A fully private, deployable research assistant that works on a plane.

The wide-context combination is the sleeper. 128K is not 1M, but it sits exactly where agent working buffers live: an entire GitHub issue plus a repository slice, a 40-minute conversation with tool logs, or a PDF-heavy research task that needs citations back at page 110. Native tool calling at 128K, with JSON outputs you verify with a decoder, is what makes a local agent a full agent.

The comparison: LFM2.5-2.6B vs Qwen3.5-9B

Qwen3.5-9B is a fine model — it beats this one on raw few-shot complex math and code where a bigger brain wins. But on-device the currency is tokens-per-second-per-GB, and the 2.6B wraps the 9B several times over. On a Mac that cannot hold 9GB for inference, the comparison is no contest: the 9B pages to death while the 2.6B finishes the turn.

Deployment patterns

The on-device framework family already adopted it: export to MLX on Apple silicon or GGUF via llama.cpp, plus the Hugging Face pipelines. A typical "agent reads an ~80K-token spec and answers with tool JSON" loop runs in tens of seconds on an M5 Max:

from lfm.load import load_model
from lfm.codec import ChatMessage
import json

model, tok = load_model("LiquidAI/lfm-2.6-2.5b", device="mps")

TOOL_SCHEMAS = [{
    "name": "create_calendar_event",
    "description": "Schedule a meeting",
    "parameters": {"type": "object", "properties": {
        "title": {"type": "string"},
        "iso_start": {"type": "string", "format": "date-time"},
    }, "required": ["title", "iso_start"]}
}]

def local_agent(messages: list[ChatMessage]) -> dict:
    prompt = model.build_prompt(messages, tools=TOOL_SCHEMAS, tool_choice="auto")
    out = model.generate(prompt, max_tokens=512)
    return json.loads(model.extract_tool_call(out))

plan = local_agent([
    {"role": "user", "content": "Block 45 minutes tomorrow at 10am with Finance."}
])
assert plan["name"] == "create_calendar_event"

The assertion is the contract: the model output is constrained to a parsed JSON call, the framework verifies it, then executes the actual tool. Agent workflows written against this loop run offline — and identically on your phone in an airport terminal.

The big table: on-device versus rented GPU

Comparing rented cloud-GPU agents against local LFM2.5-2.6B for a 10-person content and research team running 85 agent sessions per day, four tool calls each, about 9K tokens per session:

Over twelve months the cloud spend of roughly $1,100-2,400 evaporates against a zero marginal inference line — and infosec gets a bullet point: the entire research knowledge base, including regulated data, never leaves the laptop. That is the definition of a unit-economics upgrade that pays for the new Macs at full retail.

What breaks (honest engineering)

No gloss. On-device templates will hit these walls:

• True 128K real-time. 128K full context at 2.6B is token-cheap but memory-expensive; attention costs and memory ratio matter. Reason on steady working windows of 24-32K and keep the 128K as a reservoir for seek-and-extract passes.
• The M5 Max number is a specific MLX setup. A real laptop will land between ~80 and ~200 tok/s depending on model load and context. Test on your hardware; the phone number (~30 tok/s) is happy-path too.
• "Native tool calling" is not verifier rigor. It achieves "generate the call, parse it, assert the schema, retry" quality — excellent for the price, but not mixture-of-experts or verifier-backed QA quality.

Close enough to cloud is the bar

The teams I watch are not buying LFM2.5-2.6B to replace a frontier model. They buy it to own the inference plane: all-requests-private by default, all-requests-low-variance latency, all-requests-stable at milliseconds. When a task truly needs 128K+ or 9B-class brainpower, they bounce the hard query (a handful per session) to the cloud and serve the other 90-95% locally. That is the modern hybrid architecture, and it now has a 2.5GB model to hang it on.

My prediction for Q4 2026: laptop and phone "agent co-pilots" ship with sub-3B cores as standard — a read/act assistant with a local reasoning loop and tool gates — while frontier models are repositioned as "large-memory back office." The tok/s-per-watt column becomes the benchmark graph everyone plots.

Conclusion: budget for on-device as default

If you run agents mostly on data you'd rather not leave the office (finance, medical, legal), or want a long-battery agent that still handles a 100-page contract, LFM2.5-2.6B deserves a two-day proof. Download, run real corpus, test three tool gateways, then wire the phone in. The ~34T training diet shows exactly where a small model can write fluent, shape-stable tool calls on a laptop — while your older 9B guesses on a cold phone.

That is a 2026 silicon upgrade that actually upgrades the unit economics of your AI line item. And at under 2.5GB, it fits at inference where nothing else does.

For agent workflows that package these local tool loops into repeatable blueprints, see AI Workflows; for the tool registry that keeps schemas on-device-consistent, MCP Directory. Track the family's upgrade cadence at Latest AI News.

Verdict:

• Buy it for: on-device agents, privacy-sensitive data, $0-marginal-cost inference, 128K work buffers, laptops and mobiles.
• Don't replace the cloud for: complex multi-file code parity, deep math, genuinely long-lived rewrite sessions.
• Do now: grab the GGUF, build one real proof on your own machine, measure your own tok/s — then scale out the loop the money already proves.