Build a Multi-Modal Regulatory Filing Understanding Workflow with Gemini & LlamaIndex in 2026

By Deepak Bagada, CEO at SaaSNext & Principal AI Architect.

Introduction

Regulatory filings were never a text problem. A 1,000-page SEC filing, an EU technical documentation package, or an FDA submission is a multi-modal document: obligations live in prose, material numbers live in tables, trends live in charts, and the caveats that change the legal meaning live in footnotes. Teams that extract only the text get confident answers about half the document — and compliance answers that cite the wrong half are worse than no answers at all.

In 2026 the extraction stack caught up with the problem. Gemini 3.1 Pro ingests a 900-page PDF or an hour of video in a single pass, and the multimodal models that read tables and charts are finally reliable enough to build on. This workflow combines that ingestion power with LlamaIndex's document pipelines for structured chunking, metadata, and retrieval, and a LangGraph QA graph that answers compliance questions with page-level citations — and refuses to answer when it cannot find the source. It is the same discipline of source-grounded answers we apply across our AI workflows library, with the tool integrations catalogued in the MCP directory.

Architecture Overview

graph TD
  subgraph Ingest[Ingestion Layer]
    F[1,000-page PDF] --> P1[PDF -> Images + Text]
    P1 --> G[Gemini 3.1 Pro Multi-modal Read]
    G --> O1[Markdown Output: prose + tables + chart descriptions]
  end
  subgraph Structure[Structuring Layer]
    O1 --> L[LlamaIndex Pipeline]
    L --> C1[Table-Aware Chunks]
    L --> C2[Chart Nodes with Captions]
    L --> C3[Obligation Nodes with Dates]
  end
  subgraph Answer[QA Layer]
    C1 --> V[Vector Index]
    C2 --> V
    C3 --> V
    Q[LangGraph QA Graph] --> V
    Q --> VG{Source Found?}
    VG -- yes --> A[Cited Answer]
    VG -- no --> R[Refusal + Suggest]
  end

The pipeline has three layers. The ingestion layer converts the PDF to pages, reads them multi-modally with Gemini, and emits structured Markdown that preserves tables and chart captions instead of flattening them into text soup. The structuring layer turns that Markdown into LlamaIndex nodes with careful metadata — page number, document section, node type (prose/table/chart/footnote). The QA layer retrieves and answers with citations, and refuses when the answer is not grounded. Each layer is independently replaceable, which is the whole point: models change quarterly, the pipeline does not.

Part 1 — Multi-modal ingestion

.env

GEMINI_API_KEY=sk-xxxx
GEMINI_MODEL=gemini-3.1-pro
INDEX_STORE=./store
CHUNK_SIZE=2048
CHUNK_OVERLAP=256
EMBED_MODEL=gemini-embedding-002

ingest.py

import os
from pathlib import Path
from google import genai
from llama_index.core import SimpleDirectoryReader, Document
from llama_index.core.node_parser import MarkdownNodeParser

client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

def read_filing_multimodal(pdf_path: str) -> str:
    """Single-pass multimodal read: images + text + tables preserved as Markdown."""
    f = Path(pdf_path)
    result = client.models.generate_content(
        model=os.environ["GEMINI_MODEL"],
        contents=[
            "Convert this filing to Markdown. Preserve every table verbatim, "
            "describe every chart in one caption, keep footnotes as footnotes, "
            "and include the page number as a heading per page.",
            genai.types.Part.from_bytes(data=f.read_bytes(), mime_type="application/pdf"),
        ],
    )
    return result.text

def structure(markdown: str):
    parser = MarkdownNodeParser()
    doc = Document(text=markdown, metadata={"source": "2026-Q2_10-K.md"})
    nodes = parser.get_nodes_from_documents([doc])
    # tag node types from headings for retrieval-time filtering
    for n in nodes:
        n.metadata["type"] = guess_node_type(n.text)
    return nodes

The single-pass read is the 2026 unlock: Gemini's long context means you do not page through a 1,000-page filing in 20-page windows and stitch the story back together. The output keeps tables as tables (not flattened CSV soup), charts as caption-plus-location (so a chart about revenue is retrievable even though the image itself is not in the text index), and footnotes as first-class citizens.

Part 2 — Obligation extraction with structured output

extract.py

from pydantic import BaseModel, Field
from llama_index.core.extractors import BaseExtractor

class Obligation(BaseModel):
    clause: str = Field(description="Short clause identifier, e.g. 10-K Item 9A")
    obligation_text: str = Field(description="What the filer must do, one sentence")
    deadline: str | None = Field(default=None, description="ISO date if stated")
    party: str = Field(description="Who owes the obligation")
    page: int = Field(description="Source page in the filing")

class ObligationExtractor(BaseExtractor):
    async def aextract(self, nodes):
        results = []
        for n in nodes:
            if n.metadata.get("type") != "footnote" and "obligation" in n.text.lower():
                r = await obligation_model.arun(n.text)  # typed Pydantic output
                results.append((n, r))
        return results

The obligation extractor converts free text into typed, structured obligations: clause, text, deadline, party, page. The typed output is what makes downstream compliance tracking possible — a calendar of deadlines, an owner map, a coverage report. The page field makes every obligation traceable back to the source, which is the difference between an audit artifact and a chatbot answer.

Part 3 — Source-grounded QA with LangGraph

graph.py

from langgraph.graph import StateGraph, END
from typing import TypedDict

class QAState(TypedDict):
    question: str
    context: list
    citations: list
    answer: str
    grounded: bool

def retrieve(state: QAState) -> QAState:
    state["context"] = index.as_retriever(similarity_top_k=6).retrieve(state["question"])
    return state

def synthesize(state: QAState) -> QAState:
    if not state["context"]:
        state["grounded"] = False
        return state
    state["answer"], state["citations"] = qa_llm.arun(state["question"], state["context"])
    state["grounded"] = verify_citations(state["answer"], state["context"])
    return state

def route(state: QAState) -> str:
    return "refuse" if not state["grounded"] else "answer"

g = StateGraph(QAState)
g.add_node("retrieve", retrieve)
g.add_node("synthesize", synthesize)
g.add_node("refuse", refuse_and_suggest)
g.add_node("answer", answer)
g.set_entry_point("retrieve")
g.add_edge("retrieve", "synthesize")
g.add_conditional_edges("synthesize", route, {"refuse": "refuse", "answer": "answer"})
g.add_edge("refuse", END)
g.add_edge("answer", END)
app = g.compile()

Retry rules: retrieval retries once with a broader top-k (6 → 12) when the first pass returns no nodes — a genuinely unfindable answer and a poor chunk boundary are different failures. Never retry a synthesis failure; if the model cannot produce a cited answer on the second retrieval, route to refuse with a suggestion of what evidence would be needed. A compliance QA system that fabricates a deadline is a liability; one that says "I cannot find that in the filing" is a trustworthy tool. The verification loop (verify_citations) checks that each citation points to a retrieved node's page — hallucinated page numbers get caught before they reach the user.

Part 4 — Verification and the audit trail

Compliance systems live or die on the audit trail, so the workflow records every step: which pages were ingested, which nodes were retrieved for each question, which citations were verified, and which refusals occurred. That trail is what a regulator asks for, and it is what your own legal team asks for first. The same verification discipline appears throughout our AI workflows library, and the retrieval stack is catalogued alongside the MCP directory integrations you would connect to downstream compliance systems.

Production checklist

1. Preserve modality. Tables stay tables, charts get captions, footnotes stay footnotes — flattening any of them loses legal meaning.
2. Type the obligations. Structured extraction (clause, text, deadline, party, page) makes compliance trackable, not just searchable.
3. Refuse over fabricate. A cited answer or a refusal — never a confident hallucination. Verify citations against retrieved page numbers.
4. Keep layers replaceable. Ingestion, structuring, and QA are separate so model upgrades never require pipeline rewrites.
5. Audit everything. Ingested pages, retrieved nodes, citations, and refusals go into the trail a regulator will ask for.

Frequently Asked Questions

Q: Can one model really read a 1,000-page filing in a single pass?

A: Yes — Gemini 3.1 Pro's long-context multimodal ingestion handles a 900-page PDF in one pass, which is exactly the workload this workflow uses. Smaller filings are proportionally faster and cheaper.

Q: How do tables and charts survive the pipeline?

A: The multimodal read emits tables verbatim as Markdown and charts as caption-plus-location nodes. Retrieval filters by node type, so "what was revenue in Q2" retrieves the table node, not prose that mentions revenue.

Q: What if the answer is not in the filing?

A: The QA graph routes to a refusal with a suggestion of the evidence that would be needed. Compliance answers must never be hallucinated deadlines or fabricated citations.

Q: Is the typed obligation output enough for an audit?

A: Combined with the audit trail of ingested pages, retrieved nodes, and verified citations, yes — that trail is the artifact a regulator or legal team asks for.