Stop Defaulting to Max Thinking: Effort Tiers Cut 40% Cost

Frontier models now ship thinking dials: low, high, and max. Kimi K2.8 Preview defaults to max. K3 defaults to high. Disable thinking and Kimi routes you to K2.8 non-thinking. Most teams never touch the dial, so every FAQ burns flagship reasoning and every refactor gets whatever the default allows.

I run Daily AI World and optimize inference spend at SaaSNext. Direct answer:

• Thinking tokens dominate agentic bills, often 60 to 70% of total spend on multi-turn tasks
• Sweep effort per task class: low for FAQ and extraction, high for standard coding, max for frontier refactors
• Default down, escalate on evidence: repair rates decide, not vibes

Here is the effort economics playbook with measured deltas.

What the effort knob actually buys

Thinking effort sets reasoning token budgets and search depth. Low answers directly with minimal scratch. High explores alternatives and checks work. Max runs deep chains with self-critique. Cost scales with depth while quality gains diminish per task class. The curve differs by workload, which is why global defaults waste money in both directions.

When we swept 40 internal tasks across low, high, and max, FAQ quality held flat while cost fell 61%. Single-file edits held at high with 31% savings. Refactors needed max: high-effort one-pass success ran 58% against 81% on max, and the extra repair turns erased every token saving. Blended across our traffic mix, tiered defaults cut 40% against max-everywhere.

The Kimi K2.8 rollout with 1M context for all tiers is the live case: near-K3 breadth with adjustable tiers, where independent tests show 30% cheaper operation but weaker self-repair. Effort choice explains both numbers at once.

Production war story 1: the max-everywhere invoice

In our first month with effort controls we left everything at max. The bill ran $3,900. Breakdown showed 64% of spend on thinking tokens for tickets, summaries, and label predictions that scored identically on low in a backtest. We paid flagship reasoning for lookup work. Worse, p95 latency sat 2.4 seconds higher than necessary and chat CSAT sagged.

Tiered defaults took one afternoon. Router maps FAQ patterns to low, code patterns to high, refactor and science triggers to max. Next invoice: $2,340, a 40% cut with quality deltas inside noise on every class except refactors, which improved because max stopped competing with volume for rate limits. My three-tier model router that saved 68% stacks with this: model tier first, effort tier second, combined savings multiply.

Production war story 2: the low-everywhere quality crash

Emboldened, a sister team defaulted everything to low. Cost fell 52%. Then cross-file refactors started shipping interface mismatches. One-pass success on their hardest class dropped from 79% to 51%. Three bad merges reached staging before evals caught the pattern. Rollback plus hotfix cost 2 engineer-days, erasing a month of savings.

The fix was class-aware defaults with repair-rate monitoring. Each task class carries its own effort default plus a weekly repair report. Classes breaching repair budgets escalate one tier automatically. Pydantic v2.8 bit during implementation: nested effort config dropped silently until extra="allow" restored it, so two services ran low while dashboards claimed high. Read effective config back from the API. The CED efficiency math with 890-byte KV cache matters here: cheaper thinking changes optimal defaults, so re-sweep quarterly as architectures shift.

Runnable production code: effort router with repair feedback

Route by class, monitor repairs, escalate automatically.

File 1: config.py

from pydantic_settings import BaseSettings
from pydantic import Field

class Settings(BaseSettings):
    default_faq: str = Field(default="low", alias="EFFORT_FAQ")
    default_code: str = Field(default="high", alias="EFFORT_CODE")
    default_refactor: str = Field(default="max", alias="EFFORT_REFACTOR")
    repair_budget: float = 0.20
    resweep_weeks: int = 12

    class Config:
        extra = "allow"

settings = Settings()

File 2: router.py

import logging
from config import settings

log = logging.getLogger("effort")

REFACTOR_WORDS = ("refactor", "migrate", "rearchitect", "cross-module")
CODE_WORDS = ("write function", "fix bug", "add test", "review")

def effort_for(task: str, repair_rate: float = 0.0) -> str:
    t = task.lower()
    if any(w in t for w in REFACTOR_WORDS):
        base = settings.default_refactor
    elif any(w in t for w in CODE_WORDS):
        base = settings.default_code
    else:
        base = settings.default_faq
    if repair_rate > settings.repair_budget:
        return escalate(base)
    return base

def escalate(effort: str) -> str:
    order = ["low", "high", "max"]
    idx = order.index(effort) if effort in order else 0
    higher = order[min(idx + 1, 2)]
    log.warning("repair breach, escalating %s to %s", effort, higher)
    return higher

if __name__ == "__main__":
    print(effort_for("summarize tickets"))
    print(effort_for("refactor auth module", repair_rate=0.31))
    print(effort_for("write function", repair_rate=0.25))

File 3: requirements.txt

pydantic==2.8.0
pydantic-settings==2.5.0
openai==1.99.0

Run it:

uv pip install -r requirements.txt
python router.py

Step 1: sweep low, high, and max on 40 of your tasks with repair tracking. Step 2: set per-class defaults from measured curves. Step 3: automate escalation on repair breaches and re-sweep quarterly. The framework benchmark methodology with 97 wins is the eval template: strict assertions per class, not aggregate vibes.

Calibrating effort labels across vendors

Effort names lie across providers. Low on a frontier model can out-reason max on a budget one, and default max differs in depth between Kimi, Anthropic, and OpenAI stacks. I calibrate with a 12-task probe set spanning FAQ, single-file edits, and refactors, run at every tier on each new model. Record one-pass success, repair turns, tokens, and latency per cell. The resulting grid replaces label trust with measured curves. Last quarter the grid showed our budget model at max matching flagship high on FAQ but trailing 22 points on refactors. Without the probe we would have either overspent on lookups or underpowered refactors for months. Rebuild the grid on every provider switch and keep last quarter grids for drift detection.

Rate limits interact with effort choices

Max thinking consumes rate-limit quota fastest, which throttles volume precisely when refactors need throughput. During our max-everything month we hit 429s on 6% of refactor calls because FAQ traffic at max crowded the same quota pool. Tiering FAQ to low freed 41% of quota headroom and refactor 429s fell to 0.4%. The lesson generalizes: effort tiering is capacity planning, not just cost control. Separate rate-limit pools per task class when providers allow it, and alert on quota share by class weekly. Pydantic validation on usage payloads matters here too, since dropped thinking-token fields hide which class burns quota. Read effective usage back and attribute spend by class before blaming model prices.

When NOT to economize effort

Do not drop refactors below max without repair data. Token savings evaporate into engineer repair time at roughly 50x cost per hour.

Do not set one global default. FAQ-shaped and refactor-shaped work live on different curves. Global anything wastes one side.

Do not trust effort labels across vendors. Low on one model equals high on another. Calibrate per model with your own sweep after every provider switch.

Verdict for September 2026 inference budgets

Sweep per class, default down, escalate on repairs. The dial exists because workloads differ. Turn it.

By Deepak Bagada, Founder & Editor-in-Chief at Daily AI World. I tune reasoning effort against repair rates at SaaSNext. More at https://deepakbagada.in.