Agentic Doc Extraction (Beta)

Overview

The Agentic Doc Extraction node turns a document into structured, machine-readable output in a single flow step. Point it at a document URL, pick what you want back, and it runs OCR through a per-run isolated container and returns any combination of:

• Clean Markdown of every page, per page, or joined into one document.
• Structured JSON shaped to a schema you define, native document annotation, no separate parsing step.
• Every image in the document, stored for you and returned with its bounding box, an optional per-image annotation, a ready-to-use link, and a permanent storage key.

The key idea: extraction quality lives in your Output Schema field descriptions, not in a prompt. You describe what each field means and the OCR model reads the document for it. Each run spins up its own fresh container, so extractions are isolated from one another and scale independently.

Features

What Can I Build?

1. Invoice and receipt data extraction pipelines with structured JSON output.
2. Contract or report ingestion pipelines that need clean Markdown for search and summarization.
3. Form-processing pipelines with schema-driven field extraction from scanned or digital forms.
4. RAG ingestion pipelines that need both text and stored figures with durable references.

How It Works

1. You provide a document URL and select an OCR provider + model.
2. The node hands the job to a per-run container, which downloads the document, splits every page, and OCRs the pages in parallel.
3. When images are enabled, the container stores each extracted image in Lamatic-managed storage and returns a signed URL plus a durable key.
4. The node assembles the response, extractedText, structuredData, images, _meta, and returns it to your flow.

The endpoint is decided entirely by the credential you lock in: a Mistral key uses the public API; an Azure AI Foundry credential uses its private endpoint.

Prerequisites

Before adding the node, make sure you have:

• An OCR provider credential: a Mistral API key, or an Azure AI Foundry credential (endpoint + key).
• A document reachable by URL: either a direct URL or a value mapped from an upstream node.

Setup

Step 1: Configure OCR Credentials

Create a credential for your provider. The credential determines which endpoint the node calls.

Mistral (public API)

Azure AI Foundry (private endpoint)

Step 2: Add the Node to Your Flow

1. Add the Agentic Doc Extraction node.
2. Set the Document URL (direct or mapped, e.g. {{triggerNode_1.output.document_url}}).
3. Pick an OCR Model and its credential.
4. Choose an Output Format, Markdown or JSON, and toggle Include Images / Join Pages to taste.
5. (Optional) For JSON, define an Output Schema; for images, add a BBox Annotations schema; open Advanced Settings for pipeline knobs.

Configuration Reference

Settings

Outputs Pane: Schemas

These two schemas appear under the Outputs pane and are each gated on a setting:

Advanced Configs

Revealed by Advanced Settings. Every knob is optional, anything left off is never sent, and the provider default applies.

Output

The node returns four outputs. extractedText, structuredData, and images are always arrays; _meta is an object.

Example Output (Markdown mode, images on)

{
  "extractedText": [
    { "pageNumber": 1, "markdown": "# Quarterly Report\n\nRevenue grew 12%..." },
    { "pageNumber": 2, "markdown": "## Regional Breakdown\n\n![figure](https://...)" }
  ],
  "structuredData": [],
  "images": [
    {
      "pageNumber": 2,
      "id": "img-0",
      "key": "proj_abc/flow_123/req_789/agenticDocExtractionNode_1/2/000-img-0.png",
      "url": "https://...storage.../000-img-0.png?X-Amz-Expires=3600&...",
      "top_left_x": 72,
      "top_left_y": 140,
      "bottom_right_x": 520,
      "bottom_right_y": 460,
      "image_annotation": { "chartType": "bar", "title": "Revenue by region" }
    }
  ],
  "_meta": {
    "model_name": "mistral-ocr-latest",
    "pages_processed": 2,
    "total_cost": 0.002
  }
}

Example Output (JSON mode)

{
  "extractedText": [{ "pageNumber": 1, "markdown": "..." }],
  "structuredData": [
    {
      "pageNumber": 1,
      "annotation": {
        "invoiceNumber": "INV-2025-0042",
        "total": 1284.50,
        "dueDate": "2025-02-01"
      }
    }
  ],
  "images": [],
  "_meta": { "model_name": "mistral-ocr-latest", "pages_processed": 1 }
}

Low-Code Example

Markdown extraction

- nodeId: agenticDocExtractionNode_1
  nodeType: agenticDocExtractionNode
  nodeName: Agentic Doc Extraction
  values:
    documentUrl: "{{triggerNode_1.output.document_url}}"
    outputFormat: markdown
    joinPages: false
    includeImages: true
    ocrModelName:
      - configName: configA
        type: ocr/document
        provider_name: mistral
        model_name: mistral-ocr-latest
        credential_name: mistral-ocr
  needs:
    - triggerNode_1

Structured JSON extraction

- nodeId: agenticDocExtractionNode_2
  nodeType: agenticDocExtractionNode
  nodeName: Agentic Doc Extraction
  values:
    documentUrl: "{{triggerNode_1.output.document_url}}"
    outputFormat: json
    includeImages: false
    schema: |
      {
        "type": "object",
        "properties": {
          "invoiceNumber": { "type": "string", "description": "The invoice number, usually top-right" },
          "total":         { "type": "number", "description": "Grand total including tax" },
          "dueDate":       { "type": "string", "description": "Payment due date in YYYY-MM-DD" }
        }
      }
    ocrModelName:
      - configName: configA
        type: ocr/document
        provider_name: mistral
        model_name: mistral-ocr-latest
        credential_name: mistral-ocr
  needs:
    - triggerNode_1

Providers & Models

Model capability notes

• Structured extraction & image annotations (Output Schema, BBox Annotations) require mistral-ocr-2505 or newer.
• Table Format requires mistral-ocr-2512 or newer.
• Older OCR models still do Markdown extraction and image storage, they just skip the newer annotation/table features.

Limits & Security

• Document size: up to ~1,200 pages and ~100 MB per document (service defaults).
• Supported inputs: PDF and image files (JPEG, PNG, WebP) by URL.
• SSRF protection: document and OCR hosts are validated; private/internal addresses (localhost, 10.x, 172.16-31.x, 192.168.x, link-local, and IPv6 equivalents) are blocked, and redirects are re-checked at every hop.
• Encrypted or non-PDF files are rejected with a clear error rather than a partial result.
• Image storage: signed URLs are capped at 7 days; the durable key never expires and can be re-signed later.

Troubleshooting

Example Use Cases

Document Intelligence

• Invoices & receipts: extract totals, line items, and dates as structured JSON.
• Contracts & reports: clean Markdown for search, summarization, and review.
• Forms: schema-driven field extraction from scanned or digital forms.
• Figures & charts: pull images with bounding boxes and per-figure annotations.

RAG Applications

• Ingestion: turn PDFs and scans into clean Markdown for vectorization and indexing.
• Multimodal retrieval: index both text and stored figures with durable references.
• Question answering: build assistants that answer over document content and structured fields.

Migrating from Doc Extractor

The Doc Extractor node is deprecated in favor of Agentic Doc Extraction, the same OCR/document-extraction job, rebuilt with native structured extraction, image storage with bounding boxes, and a per-run isolated container.

Why migrate

Agentic Doc Extraction is a superset of Doc Extractor for the providers it supports, and adds:

• Image storage: images come back as durable storage keys + signed URLs (not inline base64), and their URLs are rewritten into the Markdown.
• Per-image bounding-box annotations: an optional schema reads each figure/chart.
• A structured images output and richer _meta cost telemetry.
• Advanced pipeline controls: headers/footers, layout blocks, confidence scores, image count/size limits.
• Isolated, scalable execution: one fresh container per run.

Before you start: three breaking changes

Field mapping

Output mapping

Updating downstream references

Migration steps

1. Add an Agentic Doc Extraction node next to (or in place of) the old one. Doc Extractor can’t be re-added, so build the new step fresh.
2. Re-select the provider + model. Pick a Mistral or Azure AI Foundry credential. Gemini users: move to a supported provider.
3. Map the basics: Document URL, Output Format (Markdown or JSON), Output Schema (if JSON), Join Pages.
4. If you used Include Annotations, turn on Include Images, and (optionally) add a BBox Annotations schema. Update anything downstream to read from the new images output.
5. Update downstream references for the new array shapes of extractedText and structuredData (see the table above).
6. Test the node, then re-point the rest of the flow and remove the old step.

Migration FAQ

Will my existing Doc Extractor flows break? No. Deployed flows keep executing exactly as before, only the ability to add new Doc Extractor nodes is removed.

Can I still use Gemini for OCR? Not in Agentic Doc Extraction. Use Mistral or Azure AI Foundry. If you specifically need Gemini, keep the existing Doc Extractor flow until you can switch providers.

Where did the inline base64 images go? Images are now stored in Lamatic-managed storage and returned as a signed url plus a durable key, which is more efficient and keeps large payloads out of the flow. Store the key for long-lived access, signed URLs expire (up to 7 days).

Do I have to switch to JSON output? No. Markdown is still the default and works the same way (just returned as an array now). JSON is only needed when you want native structured extraction via an Output Schema.

Additional Resources

• Mistral OCR Documentation
• Azure AI Foundry Documentation
• JSON Schema Reference