Build a Research-Agent Citation-Integrity Workflow with AI-Paper Forensics

By Deepak Bagada, CEO at SaaSNext & Principal AI Architect.

In August 2026 the research community got its clearest picture yet of the AI-integrity crisis: one AI system produced roughly 30 research papers in a single month, and Google DeepMind cited one of them before the provenance questions were settled. The volume is the problem — a single agent can flood a literature faster than any human reviewer can check. The response has to be a workflow, not an appeal to ethics. This dispatch builds cite-guard, a LangGraph pipeline that verifies every citation against source databases, runs AI-generation forensics on style and temperature fingerprints, scores citation-integrity risk, and gates publication and QA pipelines behind a human reviewer. Track the fallout in the latest AI news hub, then build the gate.

Why citation integrity is a production gate now

The DeepMind incident is not a scandal — it is a system failure. When a paper's citations can be fabricated, its references can be wrong, and its provenance can be machine-generated, then "it passed review" no longer means much. Peer review assumes citations point at real, findable work and that the author is an accountable party. AI volume breaks both assumptions. cite-guard rebuilds them mechanically: verify every citation against authoritative sources, fingerprint the text for machine generation, and score the result before a QA or publication pipeline proceeds.

Architecture

flowchart TD
    A[Paper draft] --> B[Verify citations]
    B --> C[Resolve DOIs / Semantic Scholar]
    C --> D[AI-generation forensics]
    D --> E[Entropy + burstiness + temperature fingerprint]
    E --> F{Integrity score}
    F -- pass --> G[Release to QA gate]
    F -- review --> H[Human reviewer]
    H -- approved --> G
    H -- rejected --> I[Reject + flag]
    F -- reject --> I
    G --> J[Append-only audit log]
    I --> J

Project setup

mkdir cite-guard && cd cite-guard
python -m venv .venv && source .venv/bin/activate
pip install langgraph langchain-openai pydantic httpx

# .env
OPENAI_API_KEY=sk-...
CROSSREF_API=https://api.crossref.org
SEMANTIC_SCHOLAR_API=https://api.semanticscholar.org/graph/v1
RISK_THRESHOLD_PASS=0.3
RISK_THRESHOLD_REJECT=0.6
AUDIT_LOG_PATH=./audit/cite-guard.log
QA_GATE_URL=http://localhost:3000/api/qa-gate
APPROVAL_CHANNEL=slack

schemas.py

from pydantic import BaseModel, Field
from typing import Optional, Literal

class Citation(BaseModel):
    key: str = Field(..., description="citation key, e.g. li2026")
    raw_text: str = Field(..., description="the rendered citation string")
    claimed_doi: Optional[str] = None
    claimed_title: Optional[str] = None

class PaperDraft(BaseModel):
    paper_id: str
    title: str
    abstract: str
    body: str
    citations: list[Citation] = Field(default_factory=list)
    authorship: Literal["human", "mixed", "ai-claimed"] = "human"

class ForensicsReport(BaseModel):
    paper_id: str
    ai_likelihood: float = Field(..., ge=0.0, le=1.0)
    entropy: Optional[float] = None
    burstiness: Optional[float] = None
    temperature_fingerprint: Optional[float] = None
    confidence: Literal["low", "medium", "high"] = "medium"

class IntegrityAssessment(BaseModel):
    paper_id: str
    total_citations: int
    resolved_citations: int
    fabricated_citations: int
    ai_likelihood: float
    integrity_score: float = Field(..., ge=0.0, le=1.0)
    reasons: list[str] = Field(default_factory=list)
    verdict: Literal["pass", "review", "reject"] = "review"

tools.py

import os, math, re
from collections import Counter
import httpx
from schemas import Citation, PaperDraft, ForensicsReport

def tokenize(text: str) -> list[str]:
    return re.findall(r"[a-z]+", text.lower())

def shannon_entropy(tokens: list[str]) -> float:
    if not tokens:
        return 0.0
    n = len(tokens)
    counts = Counter(tokens)
    return -sum((c / n) * math.log2(c / n) for c in counts.values())

def burstiness(tokens: list[str]) -> float:
    if len(tokens) < 2:
        return 0.0
    s = [len(t) for t in tokens]
    mean = sum(s) / len(s)
    var = sum((x - mean) ** 2 for x in s) / len(s)
    return var / (mean * mean) if mean else 0.0

async def resolve_citation(c: Citation) -> dict:
    if c.claimed_doi:
        async with httpx.AsyncClient(timeout=15) as h:
            r = await h.get(f"{os.getenv('CROSSREF_API')}/works/{c.claimed_doi}")
        if r.status_code == 200:
            title = r.json()["message"].get("title", [""])[0]
            return {"resolved": True, "source": "crossref", "title": title}
        return {"resolved": False, "source": "crossref"}
    async with httpx.AsyncClient(timeout=15) as h:
        r = await h.get(f"{os.getenv('SEMANTIC_SCHOLAR_API')}/paper/search",
                        params={"query": c.raw_text[:200], "fields": "title,externalIds"})
    if r.status_code == 200 and r.json().get("data"):
        return {"resolved": True, "source": "semanticscholar"}
    return {"resolved": False, "source": "semanticscholar"}

def run_forensics(draft: PaperDraft) -> ForensicsReport:
    text = draft.abstract + " " + draft.body
    tokens = tokenize(text)
    entropy = shannon_entropy(tokens)
    burst = burstiness(tokens)
    # Low burstiness + moderate entropy + uniform token flow = machine fingerprint
    ai = 1.0 - min(1.0, burst * 2.0) if burst < 0.5 else 0.2
    temp = 1.0 - entropy / math.log2(26)  # uniform text looks 'low temperature'
    return ForensicsReport(paper_id=draft.paper_id,
        ai_likelihood=round(ai, 2), entropy=round(entropy, 3),
        burstiness=round(burst, 3), temperature_fingerprint=round(temp, 3),
        confidence="high" if ai > 0.8 else "medium" if ai > 0.5 else "low")

graph.py

import os, asyncio
from typing import TypedDict, Literal
from langgraph.graph import StateGraph, END
from schemas import PaperDraft, IntegrityAssessment, ForensicsReport
from tools import resolve_citation, run_forensics

class GuardState(TypedDict):
    draft: PaperDraft
    citation_results: list[dict]
    forensics: ForensicsReport
    assessment: IntegrityAssessment
    verdict: str

def verify_citations_node(state: GuardState) -> GuardState:
    results = [asyncio.run(resolve_citation(c)) for c in state["draft"].citations]
    return {**state, "citation_results": results}

def forensics_node(state: GuardState) -> GuardState:
    return {**state, "forensics": run_forensics(state["draft"])}

def score_node(state: GuardState) -> GuardState:
    d = state["draft"]
    resolved = sum(1 for r in state["citation_results"] if r["resolved"])
    total = len(d.citations)
    fabricated = total - resolved
    ai = state["forensics"].ai_likelihood
    fabricate_rate = fabricated / total if total else 0.0
    score = 0.4 * ai + 0.4 * fabricate_rate + 0.2 * (1.0 if d.authorship == "ai-claimed" else 0.0)
    reasons = []
    if fabricated:
        reasons.append(f"{fabricated}/{total} citations unresolved")
    if ai >= 0.6:
        reasons.append("machine-generation fingerprint detected")
    assessment = IntegrityAssessment(paper_id=d.paper_id, total_citations=total,
        resolved_citations=resolved, fabricated_citations=fabricated,
        ai_likelihood=ai, integrity_score=round(score, 2), reasons=reasons)
    return {**state, "assessment": assessment}

def route(state: GuardState) -> Literal["pass", "review", "reject"]:
    s = state["assessment"].integrity_score
    if s < float(os.getenv("RISK_THRESHOLD_PASS", "0.3")):
        return "pass"
    if s > float(os.getenv("RISK_THRESHOLD_REJECT", "0.6")):
        return "reject"
    return "review"

def pass_node(state: GuardState) -> GuardState:
    return {**state, "verdict": "pass"}

def review_node(state: GuardState) -> GuardState:
    return {**state, "verdict": "review"}

def human_reviewer(state: GuardState) -> GuardState:
    # Suspended: human reviews evidence bundle, returns approved or rejected
    return {**state, "verdict": "review"}

def reject_node(state: GuardState) -> GuardState:
    return {**state, "verdict": "reject"}

def audit_node(state: GuardState) -> GuardState:
    rec = {
        "paper_id": state["draft"].paper_id,
        "integrity_score": state["assessment"].integrity_score,
        "ai_likelihood": state["assessment"].ai_likelihood,
        "fabricated_citations": state["assessment"].fabricated_citations,
        "verdict": state["verdict"],
        "reasons": state["assessment"].reasons,
    }
    append_audit(rec)  # append-only writer, separate process
    return state

def build_graph():
    g = StateGraph(GuardState)
    g.add_node("verify", verify_citations_node)
    g.add_node("forensics", forensics_node)
    g.add_node("score", score_node)
    g.add_node("pass", pass_node)
    g.add_node("review", review_node)
    g.add_node("human", human_reviewer)
    g.add_node("reject", reject_node)
    g.add_node("audit", audit_node)
    g.set_entry_point("verify")
    g.add_edge("verify", "forensics")
    g.add_edge("forensics", "score")
    g.add_conditional_edges("score", route, {
        "pass": "pass", "review": "human", "reject": "reject"})
    g.add_edge("pass", "audit")
    g.add_edge("human", "audit")
    g.add_edge("reject", "audit")
    g.add_edge("audit", END)
    return g.compile()

main.py

import os, asyncio, json
from schemas import Citation, PaperDraft
from graph import build_graph

async def main():
    draft = PaperDraft(
        paper_id="2026-0817",
        title="Attention Economics in Agentic Markets",
        abstract="We study token allocation across autonomous brokers...",
        body="Our contributions are threefold. First, we formalize...",
        citations=[
            Citation(key="nash1950", claimed_doi="10.2307/1969529",
                     raw_text="Nash (1950) Equilibrium points in n-person games."),
            Citation(key="li2026", raw_text="Li (2026) A survey of agentic attention."),
        ],
        authorship="ai-claimed",
    )
    graph = build_graph()
    result = await graph.ainvoke({
        "draft": draft, "citation_results": [],
        "verdict": "",
    })
    print(json.dumps({
        "verdict": result["verdict"],
        "integrity_score": result["assessment"].integrity_score,
        "ai_likelihood": result["forensics"].ai_likelihood,
        "fabricated_citations": result["assessment"].fabricated_citations,
        "reasons": result["assessment"].reasons,
    }, indent=2))

if __name__ == "__main__":
    asyncio.run(main())

How the forensics module works

The forensics node is deliberately a signal, not a verdict. Machine text carries statistical fingerprints — low token burstiness, a flattened distribution that resembles low-temperature sampling, and an unusually uniform sentence cadence. The module measures Shannon entropy, burstiness, and a temperature-like uniformity score, then maps them to an AI-likelihood value with a confidence level. High-confidence machine fingerprints route straight to reject or human review; low-confidence results never decide a paper alone. Combined with citation resolution, the forensics score builds the integrity number that drives the gate. An unresolved citation list is a stronger signal than any style fingerprint — a paper that cites 30 sources and can resolve none of them is fabricating literature, whatever its prose looks like.

Retry rules

• Citation resolution retries twice with exponential backoff (2s, 4s) against CrossRef and Semantic Scholar on 5xx or timeout; a failed final attempt counts the citation as unresolved.
• Forensics is deterministic and never retried; it consumes only the submitted draft text.
• Human reviewer notifications retry every 60s for up to 30 minutes; if no reviewer responds, the paper expires in review and is never released.
• The audit log write retries three times; if it fails, the workflow aborts before release — no paper leaves the gate unlogged.

The human gate and QA integration

The review path exists for the 30-paper AI author exactly: a single agent producing volume will land mostly in the reject zone, but borderline papers need an accountable human. The reviewer sees the evidence bundle — integrity score, citation resolution table, forensics report, and the authorship claim — and returns approved or rejected. On approval, cite-guard calls the QA gate URL so the verdict integrates with existing pipelines rather than replacing them. On rejection, the paper is flagged with its forensics report and the reason list, so downstream systems can block derivative work. The gate makes provenance a checkpoint instead of an afterthought.

Testing with the August incident

Replay the August scenario in test: a draft with an ai-claimed authorship flag, a fabricated-looking citation key that resolves against nothing, and machine-typical text. Confirm it rejects. Then run a clean human-authored draft with real DOIs and varied prose, and confirm it passes without human intervention. Then run the borderline case — an AI-assisted draft with mostly real citations — and confirm it routes to the human reviewer with a full evidence bundle. These three tests mirror the incident: the mass-produced paper, the legitimate baseline, and the gray zone where humans must decide. The same gate discipline extends to the broader AI workflows library and its QA automation.

The audit trail

Every paper that reaches the gate is appended to audit/cite-guard.log: paper id, integrity score, AI likelihood, fabricated-citation count, verdict, and reasons. The log is append-only and written by a separate process, so a paper cannot be quietly re-released after rejection, and a reviewer cannot be retroactively misrepresented. When a cited-but-fabricated paper is discovered downstream — the DeepMind case — the audit record shows exactly whether cite-guard saw it, what it scored, and who released it.

The bottom line

Thirty papers in a month is not a productivity story — it is a provenance crisis, and Google DeepMind citing one of the thirty is the proof that human review alone cannot keep up. cite-guard makes the fix mechanical: verify every citation, fingerprint the text, score the integrity risk, and gate release behind a human. It composes with the QA and release automation patterns in the AI workflows library. Track the integrity debates on latest AI news.

Frequently Asked Questions

What is cite-guard?

A LangGraph workflow that verifies citations against source databases, runs AI-generation forensics using style and temperature fingerprints, scores citation-integrity risk, and gates publication and QA pipelines.

What happened in August 2026?

An AI produced roughly 30 research papers in a single month and Google DeepMind cited one of them, exposing how hard provenance verification has become at the volume AI makes possible.

How does the forensics module work?

It measures entropy, burstiness, and temperature-like uniformity across the paper's text, producing an AI-likelihood score with a confidence level — a signal for human review, not a verdict.

How does citation verification work?

Each citation is resolved against CrossRef by DOI or Semantic Scholar by title search; unresolved citations are flagged as potentially fabricated and count against the integrity score.

What does the gate do?

Papers with a clean score pass through; moderate-risk papers route to a human reviewer with an evidence bundle; high-risk papers are rejected and logged with the forensics report.

Closing thoughts

The August 2026 incident closed the debate about whether AI-generated research is a real problem — 30 papers in a month, one cited by DeepMind, settled that. cite-guard turns the integrity crisis into a gate: verify citations, fingerprint machine text, score the risk, and gate release behind a human. Run it in front of every QA and publication pipeline, and fabricated literature loses its fast lane. The defense-in-depth patterns in the AI workflows library complete the picture.