Tabstack Blog

Redact PII before your model searches the web

Mon, 12 Oct 2026 14:00:00 GMT

Support chats, intake forms, benefits questions, patient messages. The messages people send your app are full of personal details: names, addresses, account numbers, IDs. And many of those questions need a current answer from the web, like a program’s income limits, a bank’s wire rules, or a state’s renewal process.

The answer depends on what they’re asking, not on who’s asking. This tutorial builds a loop that takes personal details out before the question leaves your machine: Rampart’s open model redacts personal details on your machine, Tabstack runs the search-to-answer loop on the redacted question, your own model in Ollama writes the reply, and the details go back in on your side.

Rampart, from National Design Studio

Rampart is an open PII redaction library from National Design Studio. It runs in the browser, in the moment between typing a message and sending it, and replaces personal details with placeholders like [GIVEN_NAME_1] before anything reaches a server. The mapping back to real values never leaves the device.

At its core is a 14.7 MB MiniLM model that recognizes names, phone numbers, bank account and routing numbers, tax and government IDs, passports, driver’s licenses, and every part of a street address. Rules cover card numbers, SSNs, emails, URLs, and IP addresses. Together they work across seven Latin-script languages and report 98.42% private-term recall on Rampart’s held-out test set. National Design Studio is clear about what it is for: harm reduction for people entering their own information.

Two choices make it a natural fit for web research. It keeps city, state, and ZIP by default, because location is what most “what applies to me” questions turn on. And the model is published openly on Hugging Face, so it runs from Python on your server as well as in the browser. That is how we use it here.

What Rampart’s model catches

Here is what the code in this tutorial sends onward for a few kinds of questions. Everything in brackets stays on the machine.

Intake	What leaves your machine
Benefits	My name is [GIVEN_NAME_1] [SURNAME_1], I live at [BUILDING_NUMBER_1] [STREET_NAME_1], Brookvale, CT 06482, and I make $1,950 a month. What housing assistance in Connecticut am I eligible for?
Banking	Hi, this is [GIVEN_NAME_1] [SURNAME_1] from Acme Corp, account [BANK_ACCOUNT_1], routing [ROUTING_NUMBER_1]. Why was my wire held?
Travel documents	Passport [PASSPORT_1], driver’s license [DRIVERS_LICENSE_1] issued in California. Can I renew my passport by mail from Fresno?
Health	I’m [GIVEN_NAME_1] [SURNAME_1], DOB 03/14/1988, member ID [GOVERNMENT_ID_1], diagnosed with type 1 diabetes. Does Anthem cover insulin pumps in Ohio?
Spanish	Mi nombre es [GIVEN_NAME_1] [SURNAME_1], vivo en [STREET_NAME_1] [BUILDING_NUMBER_1], Madrid. ¿Qué ayudas hay para alquiler?

The names and IDs are gone, and the state, program, amount, and topic are still there, so each question can still be answered. The health row also shows what the model leaves alone, like a date of birth, which we cover in tuning.

What you’ll build

View the diagram in the original post.

Four short files:

redact.py finds PII with Rampart’s model and swaps it for placeholders.
research.py sends the redacted question to Tabstack and gets back a cited report.
reply.py has your model write the answer from that report.
ask.py runs the loop and restores the details.

Set up

You need Python 3.9 or later, Ollama with a model pulled, and a Tabstack API key.

pip install onnxruntime tokenizers huggingface_hub numpy tabstack ollama
ollama pull llama3.1
export TABSTACK_API_KEY=...

The first run downloads Rampart’s model files from Hugging Face and caches them.

Step 1: Redact PII with Rampart’s model

The model labels each token: B-GIVEN_NAME starts a name, I-STREET_NAME continues a street, O is ordinary text. We join those labels into spans, skip the ones in KEEP, and run a short list of rules for the classes Rampart handles with rules instead of the model: card numbers, SSNs, IP addresses, emails, and URLs. Long messages run in overlapping 512-token windows, the model’s limit.

# redact.py
import json
import re
from typing import Callable, Optional

import numpy as np
import onnxruntime as ort  # type: ignore[import-untyped]
from huggingface_hub import hf_hub_download
from tokenizers import Tokenizer  # type: ignore[import-untyped]

REPO = "nationaldesignstudio/rampart"
KEEP = {"CITY", "STATE", "ZIP_CODE"}  # the answer needs these; Rampart keeps them too
MIN_SCORE = 0.4  # start a span
EXTEND_SCORE = 0.15  # continue a span across punctuation, as Rampart does

tokenizer = Tokenizer.from_file(hf_hub_download(REPO, "tokenizer.json"))
tokenizer.enable_truncation(max_length=512, stride=64)  # long messages run in overlapping windows
with open(hf_hub_download(REPO, "config.json")) as f:
    id2label = json.load(f)["id2label"]
label2id = {label: int(i) for i, label in id2label.items()}
model = ort.InferenceSession(hf_hub_download(REPO, "onnx/model_q4.onnx"))

GAP = re.compile(r"[\s\-.()/]*")


def luhn_ok(number: str) -> bool:
    digits = [int(d) for d in re.sub(r"\D", "", number)][::-1]
    total = sum(d if i % 2 == 0 else (d * 2 - 9 if d > 4 else d * 2) for i, d in enumerate(digits))
    return total % 10 == 0


# Rampart catches these with rules rather than the model, so we do too.
# Each rule is (pattern, label, optional check). Append your own for IDs the model doesn't know.
Rule = tuple[re.Pattern[str], str, Optional[Callable[[str], bool]]]
RULES: list[Rule] = [
    (re.compile(r"\b\d{3}-\d{2}-\d{4}\b"), "SSN", None),
    (re.compile(r"\b(?:\d{4}[ -]?){3}\d{4}\b|\b\d{4}[ -]?\d{6}[ -]?\d{5}\b"), "CARD_NUMBER", luhn_ok),
    (re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b"), "IP_ADDRESS", None),
    (re.compile(r"\b[\w.+-]+@[\w-]+(?:\.[\w-]+)+\b"), "EMAIL", None),
    (re.compile(r"\bhttps?://[^\s<>\"')\]]*[^\s<>\"').,;:!?\]]"), "URL", None),
]


def find_pii(text: str) -> list[tuple[int, int, str]]:
    spans: list[tuple[int, int, str]] = []
    first = tokenizer.encode(text)
    for enc in [first, *first.overflowing]:
        feeds = {
            "input_ids": np.array([enc.ids], dtype=np.int64),
            "attention_mask": np.array([enc.attention_mask], dtype=np.int64),
            "token_type_ids": np.array([enc.type_ids], dtype=np.int64),
        }
        logits = model.run(None, feeds)[0][0]
        scores = np.exp(logits) / np.exp(logits).sum(axis=-1, keepdims=True)

        window: list[tuple[int, int, str]] = []
        for (start, end), row in zip(enc.offsets, scores):
            if start == end:
                continue
            label = id2label[str(int(row.argmax()))]
            kind = label[2:]
            last = window[-1] if window and GAP.fullmatch(text[window[-1][1]:start]) else None
            # Join word pieces and multi-part values ("Cedar Lane", "203-555-0142") into one span.
            if last and (kind == last[2] or (label == "O" and row[label2id["I-" + last[2]]] >= EXTEND_SCORE)):
                window[-1] = (last[0], end, last[2])
            elif label != "O" and row.max() >= MIN_SCORE:
                window.append((start, end, kind))
        spans += window

    rules = [
        (m.start(), m.end(), label)
        for pattern, label, check in RULES
        for m in pattern.finditer(text)
        if check is None or check(m.group())
    ]

    # Rule matches win over the model, longer spans win over shorter ones, and none overlap.
    chosen: list[tuple[int, int, str]] = []
    for span in sorted(rules, key=lambda s: s[0] - s[1]) + sorted(spans, key=lambda s: s[0] - s[1]):
        if span[2] not in KEEP and all(span[1] <= c[0] or span[0] >= c[1] for c in chosen):
            chosen.append(span)
    return sorted(chosen)


class Redactor:
    """Swaps PII for numbered placeholders and keeps the mapping in memory."""

    def __init__(self) -> None:
        self.values: dict[str, str] = {}  # placeholder -> original value

    def redact(self, text: str) -> str:
        out, last = [], 0
        for start, end, kind in find_pii(text):
            value = text[start:end]
            placeholder = next((p for p, v in self.values.items() if v == value), None)
            if placeholder is None:
                count = sum(p.startswith(f"[{kind}_") for p in self.values) + 1
                placeholder = f"[{kind}_{count}]"
                self.values[placeholder] = value
            out += [text[last:start], placeholder]
            last = end
        return "".join(out + [text[last:]])

    def restore(self, text: str) -> str:
        for placeholder, value in self.values.items():
            text = text.replace(placeholder, value)
        return text

The placeholder table lives in memory in your process. The same value always gets the same placeholder, so a name that comes up again in a follow-up message keeps its placeholder.

Step 2: Research the redacted question with Tabstack

/research plans searches, reads sources, and returns a Markdown report with the pages it cited, all inside one call. Your model never fetches a page or reads raw HTML, and Tabstack only receives the redacted question.

# research.py
from tabstack import Tabstack

client = Tabstack()  # reads TABSTACK_API_KEY


def research(question: str) -> tuple[str, list[dict[str, str]]]:
    stream = client.agent.research(query=question, mode="fast")
    for event in stream:
        if event.event == "complete":
            pages = event.data.metadata.cited_pages or []
            sources = [{"title": page.title or "(untitled)", "url": page.url} for page in pages]
            return event.data.report, sources
        if event.event == "error":
            raise RuntimeError(f"Research failed: {event.data.error.message}")
    raise RuntimeError("Research stream ended without a result")

On the redacted benefits question, a fast call came back in about 17 seconds with a report on Connecticut income limits, the Section 8 Housing Choice Voucher Program, and the state Rental Assistance Program, citing five sources including the Connecticut Department of Housing. Use mode="balanced" when a question needs more sources.

Tabstack is built by Mozilla. Every request is covered by two commitments on our Data Handling page: never trained on, and private by default.

Step 3: Write the reply with your model

Your model gets the redacted question and the finished report. Tell it to keep placeholders exactly as written, so the details can go back in.

# reply.py
import ollama

MODEL = "llama3.1"

SYSTEM_PROMPT = (
    "Answer the user's question using only the research report. "
    "Keep every placeholder, such as [GIVEN_NAME_1], exactly as written."
)


def write_reply(question: str, report: str) -> str:
    response = ollama.chat(
        model=MODEL,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Question:\n{question}\n\nResearch report:\n{report}"},
        ],
    )
    return response.message.content or ""

Any model you run in Ollama works here. Swap MODEL for Qwen, Mistral, or whatever you already use.

Step 4: Run the loop

# ask.py
import sys

from redact import Redactor
from reply import write_reply
from research import research


def ask(question: str) -> None:
    redactor = Redactor()  # one per conversation; the mapping never leaves this process
    safe_question = redactor.redact(question)
    print(f"Sent: {safe_question}\n")

    report, sources = research(safe_question)
    reply = write_reply(safe_question, report)

    print(redactor.restore(reply))
    print("\nSources:")
    for source in sources:
        print(f"- {source['title']}: {source['url']}")


if __name__ == "__main__":
    if len(sys.argv) < 2:
        sys.exit('Usage: python ask.py "your question"')
    try:
        ask(" ".join(sys.argv[1:]))
    except Exception as err:
        sys.exit(f"Could not answer: {err}")

Run it with any question from the table above, or one of your own:

python ask.py "Hi, this is Tom Becker from Acme Corp, account 0042981173, routing 021000021. Why was my wire held?"

The first line shows exactly what left your machine. The answer that follows has the real details back in it, with the sources it came from.

Tune it for your data

Three adjustments cover most intakes.

Decide what stays. KEEP is the list of labels that pass through untouched. City, state, and ZIP are the default because local rules depend on them. A question about a federal program may not need the ZIP; a question about a city service needs the city. Set KEEP per use case, not once for the whole app.

Send structured fields by choice. When the intake is a form, you already know which field holds the name. There is no need to guess with a model: send the fields the answer needs, run the model only over the free text, and never send the rest.

# intake.py
from redact import Redactor

SEND = {"state", "zip_code", "household_size", "monthly_income"}  # fields the answer needs
FREE_TEXT = ("question", "notes")  # fields people type into


def build_question(form: dict[str, str], redactor: Redactor) -> str:
    lines = [f"{field.replace('_', ' ')}: {value}" for field, value in form.items() if field in SEND]
    lines += [redactor.redact(form[field]) for field in FREE_TEXT if field in form]
    return "\n".join(lines)

Research a form the same way as a question:

from intake import build_question
from redact import Redactor
from reply import write_reply
from research import research

form = {
    "full_name": "Maria Garcia",
    "email": "maria@example.com",
    "state": "CT",
    "zip_code": "06482",
    "household_size": "2",
    "monthly_income": "$1,950",
    "question": "My landlord Dave Kim raised the rent 30%. What help is there?",
}

redactor = Redactor()
question = build_question(form, redactor)
report, sources = research(question)
print(redactor.restore(write_reply(question, report)))

The question that goes to Tabstack is:

state: CT
zip code: 06482
household size: 2
monthly income: $1,950
My landlord [GIVEN_NAME_1] [SURNAME_1] raised the rent 30%. What help is there?

The name and email fields are never sent at all, and the one name inside the free text is redacted.

Add rules for your own identifiers. Case numbers, patient IDs, policy numbers, and dates of birth follow formats you know. Append them to RULES:

import re

from redact import RULES

RULES.append((re.compile(r"(?i)(?<=dob )\d{1,2}/\d{1,2}/\d{4}"), "DATE_OF_BIRTH", None))
RULES.append((re.compile(r"\bCASE-\d{6}\b"), "CASE_NUMBER", None))

With those added, “I’m Priya Nair, DOB 03/14/1988, case CASE-204871. Does Ohio Medicaid cover insulin pumps?” goes out as “I’m [GIVEN_NAME_1] [SURNAME_1], DOB [DATE_OF_BIRTH_1], case [CASE_NUMBER_1]. Does Ohio Medicaid cover insulin pumps?”

Send only the question. Redaction removes identifiers, but the rest of the message still goes to Tabstack: greetings, background, details the answer doesn’t need. Your model is already running locally, so have it rewrite the redacted message as one standalone research question first.

# focus.py
import ollama

from reply import MODEL

FOCUS_PROMPT = (
    "Rewrite the user's message as one standalone research question. "
    "Keep only the facts needed to answer it, such as location, program, amounts, dates, or conditions. "
    "Drop greetings, names, placeholders like [GIVEN_NAME_1], and any detail the answer does not depend on. "
    "Reply with the question only."
)


def focus(message: str) -> str:
    response = ollama.chat(
        model=MODEL,
        messages=[
            {"role": "system", "content": FOCUS_PROMPT},
            {"role": "user", "content": message},
        ],
    )
    return (response.message.content or message).strip()

In ask.py, import focus, research and print the focused question, and keep the full redacted message for the reply, so your model can still answer the person directly:

from focus import focus  # add to the imports at the top of ask.py

safe_question = redactor.redact(question)
research_question = focus(safe_question)
print(f"Sent: {research_question}\n")

report, sources = research(research_question)
reply = write_reply(safe_question, report)

With the date-of-birth and case rules from above, one run sent Priya’s message to Tabstack as just “Does Ohio Medicaid cover insulin pumps?” and turned the redacted housing question into “What housing assistance programs in Connecticut are available to individuals with a monthly income of $1,950?” The wording changes from run to run.

Redact first and focus second. The rewrite is a model following instructions, so it can miss. In testing it often kept placeholders like [BANK_ACCOUNT_1], which is harmless because a placeholder contains no personal data. It also sometimes added details the person didn’t give, like “for individuals with diabetes”, so check the focused question in the Sent: line.

Know what Rampart leaves alone

Rampart redacts identifiers and leaves the rest of the message alone, which is usually what the answer needs: a diagnosis for a coverage question, an income for an eligibility question, an employer for a benefits question. Dates of birth, ages, usernames, and your own internal IDs also pass through until you add a rule.

Names in non-Latin scripts are out of scope for this release of Rampart, and so are indirect identifiers: details that only point to someone in combination. National Design Studio describes Rampart as harm reduction for people entering their own information, and that is the right way to use this loop too. This tutorial is a minimal Python port of Rampart’s approach, so Rampart’s published evaluation numbers apply to the library itself, not to this code.

Where to take it next

Redact in the browser. For a web app, use Rampart’s own JavaScript library so questions are redacted before they reach your server at all: npm install @nationaldesignstudio/rampart @huggingface/transformers.
Redact replies before you log them. Run the model’s output through Redactor().redact() before it goes into logs or analytics.
Keep one redactor per conversation. Placeholders stay consistent across turns, so follow-up questions about the same person keep working.

Start with your own question: get a Tabstack API key and make a research call.

The production-readiness checklist for current-answer features

Fri, 09 Oct 2026 14:00:00 GMT

Before you ship a feature that answers questions from the live web, define which answers it can accept and what happens when a request fails. Review the evidence for answer quality alongside timing, cost, data handling, and the team’s ability to disable the feature.

A documentation assistant makes the distinction clear. It can cite a real release page and still recommend a runtime that no longer receives support. The request completed, but the answer does not meet the application’s requirements.

Use the seven gates below to review one workflow. Each gate needs an owner, dated evidence, and a decision. Set thresholds for your application rather than borrowing a universal quality score or latency target.

1. Define the answers your feature can accept

Specify the projects, question types, and source requirements the feature supports. “Answer questions from the web” gives the reviewer no useful target.

For a documentation assistant, an acceptable answer might need to identify a supported version, explain its relevant limits, and cite the official documentation. An undocumented parameter should produce an evidence gap rather than an invented definition.

Define the supported questions and the response to requests outside that scope.
Write the required answer elements and critical failure conditions before testing.
Separate accepted, review-needed, unanswered, and failed results in the application.
Name the owner who approves the acceptance rules.

Evaluate the full path from question to displayed answer, including any work your own model performs after receiving a web result.

2. Check coverage and source support

Coverage asks whether the answer addresses the question. Source support asks whether the cited passages establish its claims. An answer can name the right runtime but omit its support window. It can also include every requested field while attaching a date from the wrong release.

The citation review guide explains how to inspect a claim’s passage, qualifiers, and version. The traced Python example preserves the report and source mapping for that review.

Review each required answer element and each decision-driving claim.
Record the source’s relevant date or version, including when it is unknown.
Keep inaccessible sources, unreviewed claims, and duplicate references visible.
Apply critical failure rules before considering average scores.

Freeze the question set, expected elements, scoring rules, and implementation configuration before running the evaluation. Keep every attempt, including failures, and separate live results from pilots and synthetic tests. A small sample helps identify defects; stronger reliability claims require a broader evaluation. The 20-question evaluation set walkthrough builds this with a Python kit that hashes the frozen set, keeps every attempt, and labels pilot and synthetic runs.

Automatic checks can flag missing citations or malformed URLs. Use human review or a separately validated verifier to decide whether a passage supports a claim. In that kit’s pilot, a second blind review caught a claim the first review had scored as fully supported.

3. Test the wait your product allows

Set expectations for first feedback and the final answer. Progress tells the user that work continues; it does not mean the answer is ready.

Tabstack Research streams progress and returns the report on complete. It has no overall server-side timeout. The fetch_timeout parameter limits one page fetch, not the whole request.

Your client owns the product’s wait policy. A silence timer catches a gap between events. A total deadline limits how long the user waits even when progress continues. The evaluation kit sets both (--silence-timeout and --deadline) and records each as its own failure state.

Define first-feedback and final-result expectations for the interface.
Measure elapsed time from client dispatch to terminal success or failure.
Test both a stalled stream and an active request that exceeds the deadline.
Show a clear timeout state without claiming server-side cancellation.

Stopping the client does not establish that the server stopped processing. Account for that uncertainty before retrying or interpreting usage.

4. Contain failures before they reach the user

Test the interface as well as the exception handler. Exercise HTTP rejection, streamed task errors, interrupted connections, malformed output, missing sources, and a stream that closes without a terminal event.

For Research, an HTTP success does not establish task completion. Handle both complete and error; record an incomplete result if the stream closes without either.

Give each failure a defined application state and user-visible response.
Apply the evidence rule when a response has no source list.
Record SDK and application retries separately, including possibly accepted requests. The evaluation kit sets SDK retries to zero, so one recorded attempt is one request.
Test fallbacks so they cannot silently present old or unchecked answers as current.

A cached answer is a valid fallback only when its retrieval date remains visible and its age fits the question. A runtime-support feature should not relabel an old recommendation as current.

5. Measure cost from the same requests you reviewed

Set a workflow budget and decide what the application does at its limit. Include retries, failed requests, and model work your application performs after receiving the answer.

Research can use a variable number of actions per task, as the pricing page explains. A per-action rate alone does not establish the cost of a complete request. The Research stream reports token counts per model but no per-request credit record, so attach usage from your account records.

Attach authoritative usage records to the requests they cover.
Include billed failures and retries; leave missing usage unavailable rather than zero.
Set usage limits or alerts appropriate to the rollout.
Review plan limits, expected volume, and support requirements.

Calculate cost per accepted answer only for a fully reviewed batch with complete usage records. Divide the cost of all attempts in that batch by its accepted answers. Do not combine full-batch spend with a selected subset of successful responses.

6. Review where data goes and what it can do

Map which services receive the question, retrieved content, and instructions. Review your application’s logs and access controls as well as the provider’s handling. Running your own model does not make a hosted web call local.

The Privacy Notice is the canonical statement of how Tabstack handles your data. It says Tabstack processes your Inputs to run them through third-party models and generate Outputs, and that it stores history, including your content and Outputs, for 90 days or until you delete it. It also asks you to be careful about sharing confidential or sensitive information in your Inputs.

Check the processing path against the workflow’s privacy requirements.
Keep credentials and private customer data out of public test artifacts.
Allowlist logged fields and define who can inspect them.
Treat retrieved pages as untrusted input, validate source URLs, and restrict downstream actions.

A retrieved page cannot grant an agent permission to run commands or change an account. Set tool permissions and approval rules in the application.

If the requirement is that data never leave your controlled environment, hosted Research does not fit. Pilo provides self-hosted automation, not a self-hosted Research service.

7. Assign an owner and test the rollback

Name who responds when a source changes, a dependency fails, quality regresses, or usage exceeds its budget. A managed research service removes work from the application, but your team still owns the feature’s behavior.

Give every release gate an owner, evidence, and a recorded decision.
Limit the initial rollout and define when to expand it.
Test the path that disables the feature or selects a labeled fallback.
Document escalation contacts and repeatable regression checks.

Keep configuration and dataset versions with the runbook so another engineer can reproduce the result and identify what changed.

Record the release decision

Use a compact record that links each decision to its evidence:

Gate	Evidence to attach	Decision
Scope and acceptance	Supported workflow and critical failure rules	What the feature handles or refuses
Answer quality	Coverage and claim-support reviews	Which answers the application accepts
Time budget	Client timings and deadline tests	Whether the interaction meets its wait policy
Failure handling	Attempt log, failure fixtures, and fallback tests	How the application contains failures
Cost	Matched usage records and budget controls	Whether the workload fits the budget
Data and security	Processing map and reviewed controls	Whether the architecture meets requirements
Ownership and rollback	Runbook and disable-path test	Who acts when behavior changes

The evaluation kit includes a blank release-review template with these seven gates. Mark each gate pass, fail, or unknown. Resolve critical unknowns before expanding the rollout, and retain the evidence behind any decision to accept a remaining risk.

Start with one public question and one failure case. Review the answer’s source support, then test what the user sees when the request cannot finish. Those two records give the team concrete work to resolve before release.

Read the Research guide to check the endpoint contract. When you need to discuss fit, bring your workflow and constraints to the Tabstack team.

The 22-line demo and everything it was missing

Thu, 08 Oct 2026 14:00:00 GMT

Have you ever wanted to ask a question about a framework or package, and just get a simple answer? I have. Something like “what has changed in the latest Laravel release that affects queue workers?”. Important questions that you could answer from the changelog, if you have the time and patience (which I don’t).

So, 22 lines of Python later (thanks Claude), and 14.7 seconds after asking, it told me “The latest Laravel release, 12.x, …”. Oops. Packagist was listing v13.34.0 as the latest stable release when I ran it.

Nothing crashed, to be fair. It did exactly what I asked: one research call, print whatever comes back. The problem is everything around that call. It had no idea when not to make it, no idea how it ended, kept no record of what it said, and couldn’t tell a good answer from a confidently wrong one (see above).

So that’s what this tutorial is about. We’re going to take that demo to something I’d actually put in production, one step at a time. It’s all in JustSteveKing/what-changed, and every step below is in the commit history, so you can follow along with the diffs if you like.

Fair warning before we start: all the results in here come from twenty questions, run on a single day in October. That’s a small sample. It tells you what happened that day with those questions, and I’m not going to pretend it says anything about latency, reliability or correctness in general.

What we’re building

A “what changed?” answerer for PHP and Laravel packages. You ask it a question, and you get back an answer plus the sources it came from, in order, so you can check it.

It knows about Laravel, Livewire, Pest, Filament and Inertia’s Laravel adapter. The code is Python, but if you spend your days in PHP the domain will feel familiar, and none of the patterns care which language you use. You don’t need to know anything about Tabstack.

The stack is small on purpose: Python with uv, pydantic for the data contracts, pytest and mypy, the official Tabstack SDK, and httpx for talking to Packagist and GitHub. No frameworks.

If you want to follow along, clone the repo and start at the demo tag:

git clone https://github.com/JustSteveKing/what-changed.git
cd what-changed
make install
git checkout demo

You’ll need a TABSTACK_API_KEY for anything that makes a research call (those spend credits). A GITHUB_TOKEN is optional, but it helps once we get to the GitHub lookups. To see every step after the demo, git log --reverse demo.. lists them in order.

Step 1: the demo

Here’s the whole thing:

# The demo: one Research call, print the answer, done.
# Deliberately naive. See the git history for what changes after this.
import sys

from tabstack import Tabstack

question = " ".join(sys.argv[1:])
if not question:
    print('Usage: uv run src/demo.py "your question"', file=sys.stderr)
    sys.exit(1)

client = Tabstack()
stream = client.agent.research(query=question, mode="fast")

for event in stream:
    if event.event == "complete":
        print(event.data.report)
        print("\nSources:")
        for page in event.data.metadata.cited_pages or []:
            print(f"- {page.title or '(untitled)'}: {page.url}")
    if event.event == "error":
        raise RuntimeError(event.data.error.message)

If you’ve looked at the Tabstack Python SDK, that’s very nearly the official example. Run it with a question and it works:

uv run src/demo.py "What changed in the latest Laravel release that affects queue workers?"

And it does work, in the sense that it runs and prints something. That’s about where it stops, though. Every question costs a research call, including the ones a quick metadata lookup could answer and the ones that shouldn’t be answered at all. There’s no deadline. If the stream closes without a complete or error event, it prints nothing and exits 0, so whatever called it thinks it succeeded. Nothing gets recorded, the SDK’s default retries are still on, and there’s no way to tell a good answer from a bad one. Which is how we ended up with “12.x”.

The rest of this tutorial closes those gaps, one at a time.

Step 2: write the questions before you fix anything

The obvious next move is to start fixing code. I didn’t, and here’s why: if you change things before you’ve decided what a correct answer looks like, you’ve no way of knowing whether a change helped. You end up judging answers on how convincing they sound (and “12.x” sounded very convincing).

So the first real change is a set of twenty test questions. Some should be answered with a quick lookup, most need actual research, and a few should be declined. For each one I wrote down where it should go, what a correct answer has to include, where the truth lives (changelogs, release notes, upgrade guides), and when the answer is likely to go stale.

Here’s the queue-worker question, trimmed down a bit:

{
  "id": "q06",
  "question": "What changed in the latest Laravel release that affects queue workers?",
  "expectedRoute": "research",
  "expectedOutcome": "answered",
  "requiredElements": [
    { "id": "release", "check": "Identifies which release is the latest (v13.34.0 when written).", "source": "https://github.com/laravel/framework/releases/tag/v13.34.0" },
    { "id": "max-exceptions", "check": "Mentions that jobs can count worker crashes towards maxExceptions (PR #61737).", "source": "https://github.com/laravel/framework/releases/tag/v13.34.0" }
  ],
  "freshness": "Written against v13.34.0 (published 2026-09-29). If a newer laravel/framework release exists at run time, the 'latest' is different and these elements no longer apply; the reviewer must score against that release's notes or mark the question stale."
}

I read the ground truth from those official sources rather than from memory. If an answer and the changelog disagree, the changelog wins.

A few of the questions are there to catch one specific failure:

“What changed in Laravel 14?” should be refused, because there is no Laravel 14.
“What changed in the latest Livewire release?” is a trap of a different kind. The official release notes for that version are just a compare link, so an honest answer is thin, and only a person reading it can tell.
One question says “Inertia” and means the Laravel adapter. That ambiguity is deliberate.

Anything pinned to “the latest release” goes stale the day the next one ships, which is what the freshness note is for.

Step 3: route before you call

The biggest change, and the one that does most of the work, is a router that decides what to do with a question before anything expensive happens.

Every question goes one of three ways. A direct question gets answered by one lookup against Packagist or GitHub, with no model involved. A research question needs several sources read and pieced together, so it gets one Tabstack Research call. Anything else gets a decline: opinions, questions about the future or unreleased versions, and packages it doesn’t know about.

Declining is a proper route here, with its own reason and its own exit code. Most of the demo’s worst answers were to questions it should have just said no to.

The router is plain rules, no model. I wanted something I could unit test, explain in an article, run for free and get an answer from instantly, and a handful of regexes and if statements does all of that. The rules run in order and the first match wins:

Empty, or far too long: decline.
No supported package found: decline.
Opinion or comparison (“should I”, “better than”, “vs”): decline.
Future or unreleased (“upcoming”, “when will”, “next major”): decline.
More than one package: research.
One package, asking only for the latest version, date or release notes: direct.
Change words, or a named version: research.
Anything else: decline.

Here’s the package list and the patterns behind rules 3 and 4:

# The GitHub repo is listed explicitly because it does not always match the Packagist
# name: filament/filament's Packagist source is the filamentphp/panels subsplit, but
# its GitHub releases are published on filamentphp/filament.
KNOWN_PACKAGES = (
    KnownPackage("laravel/framework", "laravel/framework", ("laravel framework", "laravel")),
    KnownPackage("livewire/livewire", "livewire/livewire", ("livewire",)),
    KnownPackage("pestphp/pest", "pestphp/pest", ("pestphp", "pest")),
    KnownPackage("filament/filament", "filamentphp/filament", ("filament",)),
    KnownPackage("inertiajs/inertia-laravel", "inertiajs/inertia-laravel", ("inertia",)),
)

OPINION = re.compile(r"\b(should i|should we|which is better|better than|recommend\w*|vs\.?|versus|compare\w*|comparison)\b")
FUTURE = re.compile(r"\b(upcoming|roadmap|unreleased|planned|when will|be released|next (major|minor|release|version))\b")

That comment about Filament is one of those things you only find out by looking. Its Packagist source points at a subsplit repo, but the releases are published on the main one, so the mapping has to be written down.

The future pattern is narrower than you might expect. “will” on its own doesn’t count, so “Will my workers break when upgrading?” still goes to research. The tests also make sure “Laravel Livewire” counts as one package, and that “PHP 8.5” never gets read as a package version.

The top of the router looks like this:

def route(raw_question: str) -> RouteDecision:
    question = raw_question.strip()
    if not 1 <= len(question) <= 2000:
        return DeclineDecision(route="decline", reason="The question is empty or longer than 2000 characters.")

    text = normalise(question)
    mentions = find_packages(text)

    if not mentions:
        return DeclineDecision(route="decline", reason="No supported PHP package was identified in the question.")
    if OPINION.search(text):
        return DeclineDecision(route="decline", reason="The question asks for an opinion or comparison, not what changed.")
    if FUTURE.search(text):
        return DeclineDecision(route="decline", reason="The question asks about a future or unreleased version.")

Every decline carries a reason, so when the router says no, you can see why.

Step 4: check the version exists

There’s one more guard before research, and it’s tiny. If a question names a version, ask Packagist whether that version exists before going any further:

    # A research call about a version that has never been tagged would answer
    # from speculation, so check Packagist first.
    if decision.version_mentioned is not None:
        package, version = decision.version_mentioned.package, decision.version_mentioned.version
        versions = await fetch_versions(package, config, cancel)
        if not version_exists(version, [v.version_normalized for v in versions]):
            return declined(f"Packagist lists no tagged release of {package} matching {version}."), False
    return await research(question, config, cancel), False

So “What changed in Laravel 14?” now gets declined with “Packagist lists no tagged release of laravel/framework matching 14.” and never reaches the research API. Before this, the same question came back with a confident description of a release that doesn’t exist. One Packagist lookup, and that research call never happens.

Step 5: one answer shape for every route

Whatever route a question takes, it comes back in the same shape:

class RouteResult(Model):
    terminal_state: TerminalState
    # True when the call completed but the answer cannot be relied on as a full answer.
    incomplete: bool
    incomplete_reasons: list[str]
    text: str
    sources: list[SourceRecord]
    error: AnswerError | None
    # Only ever a value the API itself reported. Never estimated.
    cost: None
    cost_note: str


class ApplicationAnswer(RouteResult):
    question: str
    decision: RouteDecision
    # True when a direct lookup was served from the cache. Defaults to False so
    # records written before caching existed still parse.
    cached: bool = False
    started_at: AwareDatetime
    ended_at: AwareDatetime

The Tabstack event objects never leave the code that talks to Tabstack, so the CLI, the stored records and the scorer don’t have to know what a streamed event is (and they’re much simpler for it).

And yes, cost is typed as None. The API doesn’t report cost, so I made it impossible to put anything else there. No estimates, no guesses.

Step 6: read what the APIs actually return

The direct routes look like the easy part. Then you read the actual responses.

Packagist’s metadata is minified. Each version entry only carries the fields that changed from the one before it, and "__unset" removes a field. You have to expand it before you can use it, so the code does what Composer itself does:

# The p2 format is minified: each entry only carries the fields that changed from the
# previous one, and "__unset" removes a field. This mirrors Composer's own
# MetadataMinifier::expand (github.com/composer/metadata-minifier).
def expand_minified(versions: list[dict[str, Any]]) -> list[dict[str, Any]]:
    expanded: list[dict[str, Any]] = []
    previous: dict[str, Any] = {}
    for entry in versions:
        current = dict(previous)
        for key, value in entry.items():
            if value == "__unset":
                current.pop(key, None)
            else:
                current[key] = value
        expanded.append(current)
        previous = current
    return expanded

The “stable” metadata also still has pre-release tags in it (233 of them for Filament alone when I checked), so the code only treats plain numeric versions as stable. The order isn’t documented either, so it sorts versions itself.

GitHub held fewer surprises. The “latest release” endpoint skips drafts and pre-releases, and the unauthenticated rate limit is low, which is why the token is worth setting.

What I checked, and when, is all written down in docs/VERIFIED.md.

Step 7: measure the demo

Before improving anything, you need a baseline. So I ran all twenty questions through the demo’s behaviour, every one straight to a research call, and saved the results in the same shape as everything else.

I ran it twice, once in fast mode and once in balanced mode. Fast mode got through the lot in about four minutes. Balanced mode took nearly three quarters of an hour, with a typical call taking around two minutes and the slowest over four. In both runs, only 13 of the 20 questions ended up where they should have.

The summary looks fine. Open the stored answers and it looks a lot less fine:

The Laravel 14 question got a description of Laravel 14’s changes in both runs, though no such release existed.
“When will Pest 6 be released?” opened, in balanced mode, by talking about PEST 6.05, an environmental modelling package from 2003.
An opinion question got a full comparison instead of a refusal, both times.
The Inertia question answered about the JavaScript client instead of the Laravel adapter, both times.

Every one of those runs completed, and most of them passed most of the automatic checks. Hang on to that, we’ll come back to it.

Step 8: put a deadline on the whole call

The SDK does give research calls a ten minute timeout, but it took reading the source to see what it covers. It applies to each read, so a stream that keeps sending events never hits it.

The fix is one helper that wraps every outbound call with a deadline on the whole thing:

# Runs `awaitable` with a limit on the whole call, and stops it early if `cancel` is
# set (Ctrl+C in the CLI). Either way the work in progress is cancelled, which closes
# any open stream, and the exception says which of the two happened.
async def with_deadline(awaitable: Awaitable[T], seconds: float, what: str, cancel: asyncio.Event | None = None) -> T:
    work = asyncio.ensure_future(awaitable)
    stop = asyncio.ensure_future(cancel.wait()) if cancel is not None else None
    waiting: set[asyncio.Future[Any]] = {work, stop} if stop is not None else {work}
    try:
        done, _ = await asyncio.wait(waiting, timeout=seconds, return_when=asyncio.FIRST_COMPLETED)
    finally:
        if stop is not None:
            stop.cancel()

    if work in done:
        return work.result()

    work.cancel()
    with contextlib.suppress(asyncio.CancelledError):
        await work
    if cancel is not None and cancel.is_set():
        raise Cancelled(f"{what} was cancelled")
    raise TimedOut(f"{what} timed out after {round(seconds * 1000)} ms")

The important line is work.cancel(). Cancelling the task is what closes the stream, so hitting the deadline actually frees the connection rather than leaving it running in the background. The limits are configurable, with 10 seconds for lookups and five minutes for research by default.

The test is a local server that sends one event and then hangs, and the call has to give up at the limit.

Step 9: give every ending a name

A request can end in more ways than “worked” or “threw”. Name each one, and give each its own exit code so scripts can tell them apart:

TerminalState = Literal["completed", "failed", "stream_closed", "timed_out", "cancelled", "declined"]

# Each terminal state has its own CLI exit code so scripts can tell them apart.
# 130 follows the shell convention for a process stopped by Ctrl+C (SIGINT).
EXIT_CODES: dict[TerminalState, int] = {
    "completed": 0,
    "failed": 1,
    "declined": 2,
    "stream_closed": 3,
    "timed_out": 4,
    "cancelled": 130,
}

The one the demo got silently wrong was a stream that just stops. Now it gets a result of its own:

    # The stream ended without a complete or error event.
    return RouteResult(
        terminal_state="stream_closed",
        incomplete=True,
        incomplete_reasons=["The stream ended without a terminal event."],
        text="",
        sources=[],
        error=None,
        cost=None,
        cost_note=COST_NOTE,
    )

In the demo that case printed nothing and exited 0. Now it says what happened.

Step 10: handle Ctrl+C properly

You might have noticed the deadline helper already waits on a cancel event alongside the work. That’s the hook. The CLI sets it on the first Ctrl+C:

async def ask(question: str) -> ApplicationAnswer:
    # First Ctrl+C cancels the in-flight call and lets it report a cancelled state.
    # A second Ctrl+C exits immediately.
    cancel = asyncio.Event()

    def on_interrupt() -> None:
        if cancel.is_set():
            os._exit(EXIT_CODES["cancelled"])
        print("Cancelling. Press Ctrl+C again to exit immediately.", file=sys.stderr)
        cancel.set()

    asyncio.get_running_loop().add_signal_handler(signal.SIGINT, on_interrupt)
    return await answer(question, cancel=cancel)

The first press lets the call wind down and record a cancelled answer. The second gets you out immediately. The test runs the real CLI, sends it a Ctrl+C once the request is in flight, and checks it exits with the cancelled code.

Step 11: store what you answered

Every answer is saved exactly as it came back, with its sources in order. It’s a small change, and it’s what made the rest of this possible: everything I’ve said about the baseline came out of these files.

Step 12: log every request

One JSON line per request, with the route, how long it took, how it ended, whether it came from the cache, how many sources, and any error. The question text is left out on purpose, because logs end up in far more places than stored answers do.

Step 13: retry the cheap things only

Lookups against Packagist and GitHub are cheap and safe to repeat, so they get a few attempts with backoff. Not every failure is worth another go, though:

# Network errors, per-attempt timeouts, 429 and 5xx are worth another try.
# Other 4xx responses will not change on retry.
def is_retryable(error: Exception) -> bool:
    if isinstance(error, Cancelled):
        return False
    if isinstance(error, HttpError):
        if error.rate_limit_exhausted:
            return False
        return error.status == 429 or error.status >= 500
    return True

It also gives up straight away if GitHub’s rate limit is used up, or if the server asks it to wait more than a few seconds.

Research is never retried. Each call is billed, it takes minutes, and asking again can get you a different answer built from different sources, which is closer to a second opinion than a retry. And this is the bit you’d only find by reading the SDK: it retries twice by default. Leave the client alone and you’re retrying research calls without knowing it, so switch it off:

    # No automatic retries for research. A Research call is billed per call, takes
    # seconds to minutes, and a second attempt can return a different answer from
    # different sources. Retrying quietly would hide a failure and could double the
    # spend, so a failed call is reported and the caller decides whether to ask again.
    # The SDK retries twice by default, so this has to be switched off explicitly.
    client = client or AsyncTabstack(max_retries=0)

    async def call() -> RouteResult:
        stream = await client.agent.research(query=question, mode=config.research_mode)
        async with stream:
            return await collect_research(stream)

    # The SDK's timeout is per read, so a stream that keeps sending events is never
    # cut off by it. This deadline covers the whole call.
    # Setting `cancel` (Ctrl+C in the CLI) stops it early and closes the stream.
    return await with_deadline(call(), config.research_timeout_s, "Research", cancel)

There’s a test that fails if the default ever sneaks back in. With it left on, the local test server saw three requests for a single call.

Step 14: cache the lookups, not the research

    if decision.route == "direct":
        key = normalise(question)
        hit = cache.get(key)
        if hit is not None:
            return hit, True
        result = await direct(decision.lookup, config, cancel)
        # Only successful lookups are cached; a failure should be retried next time.
        if result.terminal_state == "completed":
            cache.set(key, result)
        return result, False

Lookups are cached for a minute, and only when they succeed. Research isn’t cached at all, because the whole point of asking is a current answer. Tabstack does keep its own server-side cache, though, and every run here used the default setting for it.

Step 15: test the stream without the network

You don’t want your tests making billed calls, but you also don’t want to mock away the bit most likely to break. So the tests point the real SDK at a small local server that replays recorded events. The SDK’s own parsing, streaming and cancelling code all gets exercised, with no API key and no network.

There are three replay files, and they didn’t all come from the same place. The successful one was recorded from a real call. The “stream just stops” one is that recording with the final event taken off. The error one is real events followed by an error I built by hand, because I never actually saw a real streamed error. Each file says which of those it is, so nobody mistakes the made-up one for a recording later.

There’s also a test that makes sure neither API key ever ends up in a saved answer or a replay file, which you want in place before you record anything live.

Back on main, make check runs the whole suite plus mypy, without touching the network or spending any credits:

git checkout main
make check

To ask the finished version a question yourself:

export TABSTACK_API_KEY=...
make ask Q="What is the latest stable version of livewire/livewire?"

That one’s a direct Packagist lookup, so it costs nothing. Only questions that get routed to research spend credits.

Step 16: run it again and compare

With everything in place, the same twenty questions went through again in balanced mode, this time through the router.

Every question went where it was supposed to. Five turned into lookups that came back in well under a second, three were turned away without any call at all, and only twelve needed research instead of twenty. The whole run took about 24 minutes instead of 46.

None of the safety nets fired, though. No failures, timeouts, dropped streams or cancellations in any live run, so everything I’ve said about deadlines and cancellation is backed by the tests and nothing else.

The research questions went through exactly the same call both times, so any difference in those answers comes from Tabstack, not from anything I changed. They took about as long as before too, a little under two minutes for a typical one.

A couple of the automatic checks look worse on paper, but only because a declined answer has no sources by design. Leave the declines out and every answer has sources.

One thing got worse for real, which you’ll see in the next step: the research answers scored slightly lower than before, with the same call on the same questions. The router didn’t cause that and couldn’t have prevented it.

The saved answers turned up one more oddity, too. The Livewire question dated Livewire 4’s first release to August 2025. Packagist has v4.0.0 in January 2026.

Step 17: score the answers separately

Remember those baseline runs that completed and passed most of the checks? Those checks look at things like whether the route was right, whether there are sources, and whether the URLs are valid. All plumbing. The Livewire question passes every one of them in every run without ever saying what changed in that release.

So I scored the answers on their own, against the things each correct answer had to include, marking each one as fully there, partly there or missing.

Full disclosure on how that was done: a model (Claude Opus 5.5) drafted the scores, and for each one it had to quote the exact bit of the answer it was judging. A validator rejects any quote that isn’t really in the answer. I went through every judgement call myself (21 of them, every “partly” plus the borderline ones) and agreed with all of them. The rest I accepted as drafted, and the scoring sheets say which is which.

Here’s how many of the 44 required points each run got fully right:

Run	Fully right
Demo, fast mode	22 of 44
Demo, balanced mode	33 of 44
With the router	35 of 44

The improvement comes from the parts the router controls. Direct lookups went from 7 of 9 to all 9, and declines from 1 of 3 to all 3. The research answers went from 25 of 32 to 23, and that’s Tabstack on the day.

The validator stops the model inventing quotes. It can’t stop it misreading a quote that’s really there, which is why I went through the judgement calls myself, and why I’m telling you which ones I didn’t.

Step 18: the production checklist

The last piece is a checklist covering quality, latency, cost, failure handling, privacy and ownership. Each item is a pass, a fail, or “can’t verify from outside”, because some of this you just can’t check from your side of the API.

Cost is the clearest case. Tabstack’s pricing page says plans, spend and credit balance are in the console, “not in any API response”. So the app can’t record cost per request at all, and there’s no cost figure anywhere in this tutorial.

Retention comes from Tabstack’s Privacy Notice, which is the source of truth for it. Research questions are run through third-party models, and history, including your content and outputs, is stored “for 90 days or unless you delete it.” So that’s the number the checklist works from: anything you put in a question is kept for up to 90 days unless you delete it, which is worth knowing before you send anything sensitive.

The data handling page adds that SOC 2 and ISO 27001 aren’t currently held. Read the Privacy Notice itself rather than relying on my summary.

The app has fails of its own. Nothing spots a stalled stream before the overall deadline, there’s no spending limit, and Tabstack’s rate-limit headers aren’t checked up front. Saved answers never expire. Nobody is named as the owner, and it’s nobody’s job to refresh the questions as they go stale.

And on latency, the slowest research call took just over four minutes against a five minute deadline, which doesn’t leave much margin.

What I’d take from this

The demo works. Production is everything around the call, and for this project most of that turned out to be deciding when not to make it.

If I was starting again, the bits I’d do on day one are: write the questions and the right answers down first, route before you call, check a version exists before asking about it, and give every way a request can end its own name. Then read the SDK source (the default retries and the timeout behaviour weren’t obvious anywhere else), and never retry anything that costs money or can come back with a different answer.

And be upfront about what you don’t know. Twenty questions on one day is a small sample, a model drafted most of the scores, and some things, like cost, you can’t verify from outside at all.

Where it goes next

The checklist fails are basically my to-do list: something to catch a stalled stream well before the overall deadline, a spending limit, actually reading the rate-limit headers instead of finding out the hard way, an expiry for saved answers, a named owner, and a schedule for refreshing stale questions. I also want to try turning off Tabstack’s cache and see what a guaranteed fresh fetch does to the answers.

If you want to check any of the numbers, docs/EVIDENCE.md has the lot.

Grab the demo tag, ask it about a package you actually use, and write down the right answer before you read what it says. Then go and have a look at your own demos (mine clearly needed it).

Build a 20-question evaluation set for a live-web feature

Wed, 07 Oct 2026 14:00:00 GMT

Before you ship a live-web feature, test it on questions your users will ask. Choose 20 from one workflow and define what each answer must include. Then check whether the answer covers the question and whether its sources support the claims.

The Python evaluation kit handles the run records and review files. It includes 20 candidate questions for an API-documentation assistant and one live pilot run. That pilot shows why coverage and source support need separate checks.

Define what a useful answer includes

For a documentation assistant, select questions that test five common problems:

Category	What to check
Source discovery	Finding the relevant official documentation
Exact contract	Returning the right fields, parameters, and limits
Synthesis	Combining pages without losing qualifications
Freshness and version scope	Using the requested date, version, or plan
Unresolved evidence	Recognizing facts the sources cannot establish

We use four questions per category. Adjust the mix to your workflow, including straightforward lookups as well as harder research tasks.

Write the required answer elements before running each question. For example:

What fields does Ollama’s web-search API return for each result, and does its content field contain the full page?

A complete answer must name title, url, and content, explain that content contains a relevant snippet, and cite the official response definition. The Ollama documentation provides the reference:

content (string): relevant content snippet from the web page

The kit stores each question in JSON Lines. Here is Q05, expanded for readability:

{
  "id": "Q05",
  "category": "exact_contract",
  "question": "What fields does Ollama's web-search API return for each result, and does its content field contain the full page?",
  "required_elements": [
    {
      "id": "E1",
      "criterion": "Name title, url, and content as the result fields."
    },
    {
      "id": "E2",
      "criterion": "Describe content as a relevant snippet, not the full page."
    },
    {
      "id": "E3",
      "criterion": "Cite the official response definition."
    }
  ],
  "reference_sources": [
    {
      "url": "https://docs.ollama.com/capabilities/web-search",
      "passage": "content (string): relevant content snippet from the web page",
      "retrieved_at_utc": "2026-10-06T17:37:54Z",
      "date_or_version": "Documentation as retrieved; no version identified in this passage."
    }
  ],
  "evidence_status": "reviewed",
  "critical_failure_conditions": [
    "States that web_search returns full page content."
  ]
}

The critical failure condition identifies an error your application should reject even when the rest of the answer is correct. Set these rules yourself and keep them consistent across implementations.

Review the evidence, then freeze the set

Open each reference and save the relevant passage, URL, and retrieval time. Record the source’s version or date when available. For time-sensitive questions, specify the date in the question rather than asking only for “current” information.

Only Q05 has reviewed evidence in the candidate set. The other 19 questions remain pending, so the kit refuses to run the full set. Review their references before including them in scored results.

Freezing records the dataset version and hash. The kit rejects a changed dataset or evidence older than the freeze record allows, which defaults to 30 days. Once the committed Q05 evidence is more than 30 days old, refresh it and freeze a new version before you run the pilot.

If a source changes, update the reference under a new dataset version. Do not rewrite the expected answer to fit a result you have already seen.

Run a one-question pilot

The kit extends the traced Python Research example, which saves the report, ordered sources, and request timeline. Start with one question to check the output and review process before expanding the run.

The recorded pilot used macOS arm64, Python 3.12.13, uv 0.11.6, and the tabstack Python SDK 2.8.5. Create a key in the Tabstack console, then run:

git clone https://github.com/Mozilla-Ocho/tabstack-cited-research-python.git
cd tabstack-cited-research-python
git checkout 460b72e174187796eae5707b7a028aa13b8af09c
uv sync --frozen
export TABSTACK_API_KEY=...   # Set your key locally; do not commit it.

cp acceptance-evals/pilot/questions.pilot-q05.jsonl acceptance-evals/pilot/questions.my-pilot.jsonl
uv run cited-research-accept freeze \
  --dataset acceptance-evals/pilot/questions.my-pilot.jsonl \
  --version my-pilot-v1 \
  --allow-subset

uv run cited-research-accept run-one \
  --dataset acceptance-evals/pilot/questions.my-pilot.jsonl \
  --question Q05 \
  --run acceptance-evals/runs/my-pilot-q05 \
  --mode fast \
  --nocache \
  --silence-timeout 120 \
  --deadline 300 \
  --pilot

run-one disables SDK and application retries, so each attempt represents one request. It sends the question and a neutral output instruction to Research; the expected answer elements stay in reviewer-side files. The --pilot flag keeps this run out of scored totals.

The committed pilot’s terminal output includes:

complete  report -> acceptance-evals/runs/20261006-pilot-q05/answers/Q05-a1/report.md
sources (3) -> acceptance-evals/runs/20261006-pilot-q05/answers/Q05-a1/sources.json
  1. Web search  https://docs.ollama.com/capabilities/web-search
  2. Web search  https://docs.ollama.com/capabilities/web-search.md
  3. Testing Ollama Web Search 🔍 and a Thinking Model 💭  https://dev.to/aairom/testing-ollama-web-search-and-a-thinking-model-1dh7
review sheet -> acceptance-evals/runs/20261006-pilot-q05/answers/Q05-a1/review-sheet.csv (unreviewed)
attempt Q05-a1: complete (exit 0), 3 sources -> acceptance-evals/runs/20261006-pilot-q05/attempts.jsonl

Use a new run directory. The kit refuses to overwrite an existing attempt; --another-attempt creates a separate record such as Q05-a2.

Check coverage and source support

Prepare the review files:

uv run cited-research-accept prepare-review \
  --run acceptance-evals/runs/my-pilot-q05

In reviews/coverage.csv, score each required answer element. In reviews/claims.csv, check individual claims against the cited passages. Both use the same scale:

Score	Coverage	Source support
`2`	Covers the element fully	Supports the claim at its stated scope
`1`	Covers part or misses a qualifier	Supports part or requires a qualifier
`0`	Omits or contradicts the element	Does not support or contradicts the claim
`U`	Unresolved after review	Uninspected or inaccessible evidence

Leave a cell blank if you have not reviewed it yet.

Record the answer passage and your reason for each score. The citation review guide explains the source checks in more detail.

The Q05 pilot answered the main question correctly:

Crucially, the content field within each web search result contains a relevant content snippet from the web page, not the full page content [1][2][3].

The first review scored all three required elements 2. But the answer also added a sentence saying that web_fetch retrieves the full content of a page. The documentation says main content, so the claim review scored that statement 1. The review split that sentence into two claims.

That extra claim did not affect coverage because the question did not ask about web_fetch. It still needs correction before a developer relies on it.

A second, blind review that could not see the first scores caught one more. The answer says the search API “returns an array” of results, but the documentation says it returns an object containing a results array. That claim also scored 1, so three of the five claims scored 2.

Sources [1] and [2] are HTML and Markdown versions of the same official page, not independent evidence. Source [3] was not opened, and the review records that gap. The CLI preserves source order so each citation number continues to point to the correct page.

An AI agent did both reviews; the second could not see the first one’s scores. The final decision remains blank because the pilot defined only a critical failure condition, not full release criteria.

Read the summary without hiding failures

After completing the reviews, run:

uv run cited-research-accept summarize \
  --run acceptance-evals/runs/my-pilot-q05

The summary separates completed requests from accepted answers. It includes failed requests in the attempt total and shows unfinished reviews separately. For coverage and support scores, it uses inspected rows and leaves U and blank cells unscored.

For example, the synthetic offline test has six attempts: three completed and three failed. One answer was accepted, giving 1/6 accepted attempts. Coverage was 2/4 because two of the four inspected elements were fully covered. The remaining U and blank cells stayed visible but unscored. These are test-fixture results, not live API performance.

Cost remains unavailable until you attach authoritative usage records in reviews/usage.csv. Research streams model token counts but no per-request credit record. Include billed failures and retries; do not treat missing usage as zero.

Handle the stream’s error event explicitly. A client timeout stops the wait but does not establish server-side cancellation. The attempt log preserves these outcomes so a failed request cannot disappear from the evaluation.

Expand after the pilot

The test receipt reports 255 passing tests, with Ruff and Pyright passing. The tests cover dataset validation, terminal states, incomplete reviews, source order, and summary calculations.

Only one question has run live. All failure tests use offline fixtures. Balanced mode, --fetch-timeout, and --another-attempt remain untested against the API, as do Linux, Windows, and reproduction by a second engineer. Cost calculations use synthetic values only. Twenty questions can reveal specific defects, but they cannot establish production reliability or provider superiority.

Finish reviewing the references for your remaining questions, then use the results to choose the next engineering change. Wrong version scope calls for better version selection. Unsupported claims call for a better evidence path or narrower wording. Slow responses call for a separately evaluated configuration or a different interaction.

Keep private questions and credentials out of the public dataset. Review the Research guide, Trust page, and Privacy Notice before using hosted processing for a sensitive workflow.

Run the kit on one question and complete both reviews before expanding to the full set.

When should you use search, agentic search, a Research API, or build the loop yourself?

Mon, 05 Oct 2026 14:00:00 GMT

Use a search API when your application needs sources. Use agentic search when your model needs to choose what to investigate next. Use a Research API when your application needs a finished answer with citations. Build the loop yourself when control over source selection, evidence handling, or synthesis is part of your product.

The decision starts with the job. If a coding assistant needs a parameter definition from a known documentation page, fetch that page. If it needs to compare current capabilities across several projects, someone must find the sources, read them, check coverage, and reconcile the evidence before writing the answer.

Choose where that work belongs and which parts your team needs to control.

First, define what your application must receive

“Add web access” leaves the output undefined. Write the result your application actually needs:

Required output	Starting point
Relevant URLs and snippets	Search
Clean content from a known URL	Fetch or Markdown extraction
Specific fields from a known page	Structured extraction, followed by value checks
An answer supported by current sources	A research loop, operated by your application, agent, or managed service
A completed website task	Browser automation with explicit action boundaries

These jobs can exist inside the same application. A pasted link does not need source discovery. A request to find current deployment guidance might. Reading a pricing page and comparing several vendors’ pricing are different workflows even when both begin with a URL.

The search-versus-research guide explains the work between finding sources and producing a cited answer. This guide helps you decide who should perform it.

What each approach gives you

A search API returns candidates for your application to use

A search API fits a source picker, a documentation finder, or a system that already knows how to build an answer from retrieved material.

For example, Ollama’s Web Search API returns results with title, url, and a relevant content snippet. Its Web Fetch API returns a page’s main content and links. The two responses support different next steps: a snippet helps identify a source; fetched content gives your application material to inspect.

If your feature displays links, search can finish the job. If it displays a cited explanation, your application still owns source selection, reading, answer construction, and the connection between claims and sources. That can be a small, bounded implementation when one authoritative page contains the answer.

Choose search when finding sources is the required output, or your existing downstream system already meets the answer requirements.

Agentic search lets the model decide what to do next

In model-operated agentic search, the application gives the model search and fetch tools. The model chooses a query, reads the results, and decides whether to open a page, search again, or answer. The application executes those tool calls and enforces their limits.

Ollama’s search-agent example demonstrates this pattern. It adds tool results to the conversation and continues until the model stops requesting tools.

This approach fits a task whose next step depends on what the agent learns. A coding assistant can inspect a local dependency version, retrieve the corresponding documentation, and use it while proposing a change. The application decides which local context the model can use and which information it can send in a public-web query.

You still maintain tool definitions, context handling, loop limits, source mappings, and failure behavior. Treat web content as untrusted input; it must not grant the agent new permissions or override the application’s instructions.

Choose agentic search when the model’s adaptive investigation belongs inside the larger task.

A Research API returns the answer before your model continues

A managed Research API takes a question and performs the answer-building work inside the service. Your application receives the researched output instead of executing each search and fetch decision itself.

Tabstack Research plans queries, discovers and reads sources, checks gaps, follows up when needed, and synthesizes a Markdown report with citation metadata. It returns the report on the stream’s terminal complete event. Your application can give that result to your own model as an input to its next step.

That boundary suits a briefing feature or assistant that needs a current public-web answer but does not need to control every intermediate retrieval decision.

Your application still owns acceptance. Check whether the answer covers the question and whether its decision-driving claims have support. Handle task-level error events, and treat a stream that ends without a terminal event as incomplete. Research always streams; receiving progress is not the same as receiving the finished report.

Choose a Research API when you need the answer as an input and the service’s exposed controls satisfy the job.

A DIY loop gives your team control over the implementation

A custom loop lets you choose the retrieval components, source-selection rules, context limits, reconciliation method, synthesis prompts, and citation construction. You can combine deterministic steps with model decisions.

Build when those choices create product value or satisfy a requirement a managed service cannot meet. Examples include a prescribed source hierarchy, retrieval across private and public systems, or a requirement to preserve selected evidence passages in your own format.

Ownership includes maintenance. Your team handles changed pages, failed fetches, provider limits, model changes, incomplete evidence, and malformed output. It also maintains the evaluation that determines whether the loop still works.

Choose DIY when you can name the intermediate decisions you must control and assign the engineering capacity to operate them.

Agentic search and DIY overlap. “Agentic” describes how the system chooses its next action; “DIY” describes what your team builds and owns. A managed Research service can also run an agentic loop internally. Inspect the response contract and control boundary rather than choosing from the label alone.

The decision matrix

Approach	What your application receives	Who chooses follow-up work?	What your team maintains	Prefer another path when…
Search API	Candidate sources, commonly URLs and snippets	Your application or model	Retrieval integration and any downstream answer-building	You need a finished answer but have no dependable downstream loop
Model-operated agentic search	The configured agent’s answer or task result	Your model, within application limits	Tools, prompts, context, orchestration, and acceptance checks	The web question is a separable subtask and the model does not need intermediate control
Managed Research API	The service’s documented report and citation format	The service, within its exposed controls	Integration, output acceptance, and provider fit	Required sources, controls, or processing boundaries fall outside the service’s contract
DIY research loop	The output contract you implement	Your code, models, or both	Components, orchestration, evidence handling, evaluation, and operations	The custom loop adds no necessary control and your team cannot justify maintaining it

The architecture does not establish which implementation answers better, runs faster, or costs less. Compare the complete configurations on your questions. Include failed attempts and incomplete outputs rather than pricing only the successful demo.

Where does the work run?

Use this process model before selecting a provider:

View the diagram in the original post.

If a required processing boundary rules out a provider, eliminate it before testing answer quality. A good report cannot make an incompatible data path acceptable.

Apply the guide to a concrete job

Suppose a developer assistant receives this question:

What does Ollama’s Web Search API return, how does that differ from Web Fetch, and can I display the search response as a cited answer?

Before running anything, define the required answer elements:

Name the documented fields in each response.
Distinguish a relevant snippet from fetched page content.
Explain the application work required to turn those responses into an answer.
Cite the documentation for the API facts.

Here is a complete, documentation-derived answer example, not output from a new API run:

Ollama’s Web Search API returns a results array whose entries contain title, url, and a relevant content snippet. Web Fetch returns title, the page’s main content, and links. Neither response defines a finished cited-answer format. Your application can display search results directly, but it must construct the explanation and attach supporting sources if the feature promises a cited answer. See the Web Search and Web Fetch documentation.

For this job, start by reading the known documentation page. It contains the response definitions. If your application can write the explanation and preserve its source, it does not need an open-ended research loop.

Change the job and the choice can change:

Find official web-search documentation. Use search when you do not already know the destination.
Inspect this repository’s dependency version and work out the relevant integration steps. Use an agentic path when the model needs to move between local inspection and public documentation. Keep private context out of public queries unless your policy explicitly permits it.
Compare current retrieval approaches for a developer assistant using evidence across several projects. Evaluate a Research API if your application needs the comparison report before its model continues. Freeze the comparison criteria first.
Use only approved sources and preserve the exact passages selected at every stage. Build or select a loop that exposes those controls. Do not infer that a report and citation list satisfy an evidence-archive requirement.

The question determines the workflow. A product does not need one architecture for every web request.

When Tabstack is not the answer

We built Tabstack to return finished web output to applications. These requirements point elsewhere:

Your requirement	Better starting point	Why
You only need ranked links and snippets	A search API	Research performs answer-building work the feature does not need
Your existing assistant or search loop already meets the job	Keep the existing path	A new provider needs a concrete reason to enter the architecture
Your product must control intermediate source selection or synthesis beyond the API’s controls	A custom loop or service exposing those controls	The required behavior belongs in the implementation contract
Your workload must stay inside an air-gapped or exclusively controlled environment	An architecture compatible with that boundary	Hosted Tabstack Research processes requests outside your environment
Your workflow needs private sources the public-web service cannot access	An authorized private-data retrieval path, optionally combined with public research	Public-web research cannot supply inaccessible evidence
You need a hosted browser task behind a login	An authenticated automation system you control	Hosted Tabstack Automate handles public websites and has no credential store or session management

A known URL is a separate case: it can fit Tabstack, but usually through Extract, not Research. Ask for Markdown when the next step needs page content, or matching JSON when it needs fields. Validate the values as well as the shape.

Pilo provides a self-hosted automation path. It is not a self-hosted deployment of Tabstack Research.

Check the processing boundary

Running your own model does not make external web services local. Search queries and research requests follow the data paths of the providers your application calls.

Tabstack Research runs on hosted infrastructure. Contracted model providers process the query and web content to produce the answer. Our Data Handling documentation describes that path, and the Privacy Notice is the canonical policy. Confirm workload-specific retention and provider requirements before sending sensitive inputs. Do not interpret “never trained on” as a promise that no hosted processing or storage occurs.

Choose a path for one real job

Write down the output your feature promises, the intermediate decisions you must control, and the data it can send outside your environment. Then test the smallest approach that satisfies those requirements.

Use the citation review method to check claim support and answer coverage. The traced Python example shows how to preserve the report, cited pages, and request timeline. For a comparison between complete systems, use the search + fetch + model evaluation plan.

If your job needs a finished answer from public sources, bring that question to the Research playground, which requires sign-in. Write its required answer elements first, then check the returned report against them. If search or a known-page fetch already meets the job, use it.

What "private by default" means for a managed web API

Fri, 02 Oct 2026 14:00:00 GMT

If you run your own model, you still need to know what happens to anything you send to an external service. A Tabstack Research call sends your query to our hosted API. Contracted model providers process the query and web content to produce the answer.

We describe Tabstack as “private by default.” This post explains what we retain after a request and what the detailed data collection setting changes.

What happens during a Research call

Suppose your query is “What changed in the latest Node.js LTS release?” Tabstack receives that query and reads public sources to produce the answer. Your application can then pass the returned report to your own model.

View the diagram in the original post.

Your application calls the API with your key over TLS 1.2 or higher. We fetch the public pages the answer needs, and our direct fetches identify themselves as Mozilla-Tabstack/1.0 (+https://tabstack.ai). Your query and the page content go to a contracted model provider, which produces the report. The report and the pages it cites come back to your application. All of this runs on our hosted infrastructure, not inside your local model.

What we retain after the request

Request metadata is recorded on every request, on every plan: the endpoint, success or failure, credits spent, timestamps, and the organization and key that made the call. The Data Handling page doesn’t state how long request metadata is kept.

Payloads are the target URL, the request parameters, the response data and the output. For Research, that includes your query. We store payloads only while your organization has detailed data collection on. New organizations start with it on, and an organization admin can turn it off.

With detailed data collection on, payloads are kept for 90 days and appear in your console, which helps when you’re debugging results or auditing what a task did. With it off, new payloads aren’t stored. If your workload can’t have payloads stored at all, turn the setting off before you send anything.

Fetched page content is cached by URL, effort and region, with a short time-to-live. The cache isn’t per account, so public page content fetched for one account can be served to another. That matters when you need a fresh copy of a page. Pass nocache: true to bypass the cache for that request.

Our Privacy Notice is the canonical statement of what we retain. It applies to the API as well as the website and our other services, and it says: “We store history (including your content and/or Outputs) for 90 days or unless you delete it.” The notice also asks you to be careful about putting confidential or sensitive information in your inputs. For privacy questions, our Data Protection Officer is at dpo@mozilla.com.

What “never trained on” covers

Our Data Handling page states it directly: your requests and the content Tabstack processes for you are not used to train models.

Research is one of the endpoints that sends your query and the page content to a contracted model provider. The Privacy Notice names Google Cloud, including Vertex, and OpenAI as contracted providers. Each provider’s own retention and training terms for that traffic are a separate question, and the public documents don’t list them per endpoint.

When a hosted API is not suitable

If your workload requires that data stays inside an environment you control, a hosted Research call is the wrong tool, and no setting changes that. There is no self-hosted deployment of the API. Pilo, our open-source automation engine, runs browser automation in your own environment. It covers automation only, not Research.

Where to review the details

Tabstack is built by Mozilla, which is why we state these limits as plainly as the commitments. Read our Data Handling documentation for the request flow and retention settings. If your review requires provider-specific terms, contact us before sending sensitive data.

Trace a question from request to source-backed answer

Wed, 30 Sep 2026 14:00:00 GMT

A Tabstack Research call returns a Markdown report and the pages it cites. The Python example saves both, records the request’s progress events, and generates a CSV review sheet that connects each report sentence to its cited pages.

Use the timeline to inspect the request’s event order and elapsed time. Use the review sheet to check whether a source supports the claim attached to it. For example, the recorded answer describes a search result as page content, while the cited documentation specifies a content snippet. That distinction affects what your application can do with the result.

This tutorial uses the committed Ollama web-search example. You will run the CLI, inspect its output files, and follow one sentence through the source review. The implementation extends Build a cited live-web answer in Python with one Research call.

Run the traced request

You need Python 3.9 or later, uv, and a Tabstack API key. The example’s lockfile pins the tabstack SDK at 2.8.5. Run the following commands in Bash:

git clone https://github.com/Mozilla-Ocho/tabstack-cited-research-python.git
cd tabstack-cited-research-python
uv sync --frozen

# Read the key without echoing it or saving it to shell history.
read -rs TABSTACK_API_KEY && export TABSTACK_API_KEY

uv run cited-research \
  --query "What are the current ways to add web search to an Ollama-based application, and what output does each approach return?" \
  --mode fast --nocache --silence-timeout 120 \
  --output artifacts/my-trace

The CLI reads TABSTACK_API_KEY from the environment. The command has no API-key argument. --nocache bypasses the fetched-page cache, and --silence-timeout 120 stops the client after 120 seconds without an event.

The command writes your files to artifacts/my-trace. The committed example below uses artifacts/trace-run; its paths identify that saved run, not the directory you just selected.

The recorded run on 29 September 2026 produced this terminal output:

start Starting research
planning:start Planning research strategy
planning:end Planning complete: 6 queries
iteration:start iteration 1/1 Starting iteration 1 of 1
searching:start iteration 1 Searching with 6 queries
searching:end iteration 1 6 new urls Found 6 URLs
iteration:end iteration 1 Iteration 1 complete (fast mode)
writing:start Writing report
writing:end Report draft complete
complete report -> artifacts/trace-run/report.md
sources (3) -> artifacts/trace-run/sources.json
  1. Web search https://docs.ollama.com/capabilities/web-search
  2. Web search https://docs.ollama.com/capabilities/web-search.md
  3. Subagents and web search in Claude Code https://ollama.com/blog/web-search-subagents-claude-code
review sheet -> artifacts/trace-run/review-sheet.csv (unreviewed)

The first event arrived after 496 ms, and the request completed in 17.1 seconds. These timings describe the recorded request; they are not a general latency benchmark.

Open the output files

The example saves the answer, source list, and timeline separately so you can inspect each without reconstructing the stream:

artifacts/trace-run/
  question.txt
  command.txt                 the command as run, no secrets
  events.sanitized.jsonl      the lifecycle timeline
  report.md                   the answer
  sources.json                cited pages in returned order
  review-sheet.csv            one row per sentence, ready for review
  trace-diagram.md            the timeline as a diagram, plus its limits
  run-manifest.json           versions, UTC times, terminal state, caveats

Start with report.md, sources.json, and review-sheet.csv. The report contains the answer’s numbered citation markers. The source list preserves their positions. The review sheet joins sentences to those entries and leaves evidence and scoring fields for you to complete.

The sequence diagram shows where that review fits:

View the diagram in the original post.

Preserve the citation mapping

sources.json numbers cited pages from 1 in the order the API returns them. Each record keeps the source’s id, url, title, claims, and source_queries, along with the CLI’s validation flags. The sample returned a title for every entry; it did not populate the optional relevance or reliability fields.

The report’s [n] markers refer to the source at position n. Sorting the list or removing an entry changes the targets of subsequent markers. The CLI preserves source order so each citation number continues to point to the correct page.

The first run returned three entries, but entries 1 and 2 point to the same Ollama documentation page, with and without .md. The CLI retains both entries and flags the repeated source.

Its duplicate detector compares URL patterns, including scheme, www., trailing slashes, and .md suffixes. Those comparisons flag possible matches; they do not inspect page contents. The sheet’s “[2] likely same page as [1]” label records that heuristic. In this example, both URLs resolve to the same documentation page.

The CLI also checks URLs before rendering them as links. It excludes loopback and private addresses, non-HTTP schemes, and URLs containing embedded credentials, and strips credentials from stored URLs.

Fast mode returns an empty claims array for each cited page. Use the report’s numbered markers to follow its statements to their source entries. Balanced mode populates the claim lists, as the Research guide documents. Those lists record the pipeline’s extracted claims; the review still checks them against the page.

If the response contains no cited pages, the CLI writes [] to sources.json and records review_needed_no_sources in the manifest.

Check a sentence against its source

The review sheet starts with one row per report sentence. It fills the citation mapping and reserves the evidence fields for the reviewer:

Fields	Who fills them
`claim_id`, `answer_text`, `citation_ids`, `source_url`	The CLI
`passage`, `source_date_or_version`, `retrieved_at_utc`, `support`, `reason`	You, during source review

Helper columns include cited page IDs, automatic flags, and any claims supplied by the API. The CLI leaves support blank. Split a sentence into separate rows when its clauses make claims that need different evidence.

In the recorded answer, one sentence describes results containing title, url, and content “of a relevant web page” and cites entries [1][2]. Both entries lead to the same documentation. The relevant passage defines the field as:

content (string): relevant content snippet from the web page

The answer omits “snippet”. Correct that clause to say the result contains a relevant content snippet from the page. An application that needs the full page must fetch it rather than treat the search snippet as complete page content.

Record the passage, the source URL, when you inspected it, and why the wording needs correction. The citation review method defines the support values: 2 for supported at the stated scope, 1 for partial support or a missing qualifier, 0 for unsupported or contradicted after inspection, and U for uninspected or inaccessible evidence.

The original review also identified these specific issues:

Answer detail	Source evidence	Correction
An API key requires a free Ollama account, but the sentence has no inline citation	“A free Ollama account is required.”	Attach the supporting documentation to that claim.
“MCP (Multi-Context Processor)”	The documentation describes a Python MCP server; MCP stands for Model Context Protocol.	Replace the incorrect expansion with “Model Context Protocol”.
Built-in search applies to “specific” cloud models	“It works with any model on Ollama’s cloud.”	Do not turn recommended models into a restriction on supported models.

Keep missing-citation and duplicate-source flags separate from factual support scores. A repeated URL describes the source list; it is not another factual claim. A sentence without a marker needs an evidence check before you classify it as unsupported.

The question also asks what each approach returns. Check that required element separately for every approach; correcting one sentence does not establish whole-answer coverage. Write down those required elements before reading a new answer.

cited-research-review preserves completed review entries and edited rows. It will not regenerate over them unless you pass --force.

Read the event timeline

events.sanitized.jsonl records one event per line using an allowlist of fields. The CLI redacts URLs, email addresses, and credential-like strings from status messages, and omits payload fields such as planned queries.

This event comes from the second recorded run:

{"elapsed_ms": 3091, "event": "iteration:end", "is_last": true, "iteration": 1.0, "known_event": true, "message": "Iteration 1 complete (fast mode)", "received_at_utc": "2026-09-29T18:58:40.193Z", "seq": 7, "stop_reason": "max_iterations", "timestamp_raw": 1790708320152.0, "timestamp_type": "float"}

Use seq to preserve arrival order. Several events share a server timestamp, including start and planning:start, so timestamps alone cannot distinguish their sequence.

The client measures elapsed_ms from just before it sends the request. That value records the wait your application experienced, not the server’s internal execution time. The CLI also keeps timestamp_raw and its type. The sample carries a float in Unix epoch milliseconds; the current Research guide documents epoch milliseconds, and the API reference defines the field as a number.

The first recorded run produced this timeline:

seq	Event	Elapsed
1	`start`	496 ms
2	`planning:start`	496 ms
3	`planning:end`	1,565 ms
4	`iteration:start`	1,566 ms
5	`searching:start`	1,566 ms
6	`searching:end`	6,843 ms
7	`iteration:end`	6,843 ms
8	`writing:start`	6,843 ms
9	`writing:end`	17,083 ms
10	`complete`	17,084 ms

In this run, the client observed roughly 1.1 seconds between the planning events, 5.3 seconds between the searching events, and 10.2 seconds between the writing events. Use these events to update a progress indicator while the request runs.

The stream exposes request phases, not individual page-fetch timings or internal reasoning. The complete event supplies the final report and cited-page list; it does not expose the passages the service read.

Handle failures without duplicating requests

Except when TABSTACK_API_KEY is missing, the CLI records a terminal state in the manifest and returns a corresponding exit code:

Exit	Meaning
0	`complete` arrived and every file was written
2	The stream sent an `error` event
3	The request was rejected before streaming began (401, 429, 5xx)
4	Connection or transport failure before the stream opened
5	`TABSTACK_API_KEY` is not set; no request was made
6	The stream closed before `complete` or `error`
7	No event arrived for `--silence-timeout` seconds
8	A second `complete` or `error` arrived after `complete`
9	The connection dropped or timed out after the stream opened
10	`complete` arrived without a report
11	Any other unexpected failure; the files are still written

Disable automatic retries

The CLI disables SDK retries and adds no retry loop. A connection failure can occur after the server accepts a request. Retrying automatically could create another research task and incur another charge. Inspect the failure before deciding whether to send a new request.

The first recorded run predates this change and records sdk_max_retries: 2. The second records 0. Check the manifest when comparing saved runs so you know which retry behavior applied.

Measure stream silence

The Research API has no overall server-side timeout. The CLI resets its silence timer on every event, including the wait for the first event. A request can continue as long as events arrive within the configured interval.

A silence timeout stops the client from waiting; it does not establish that the server cancelled the task. The CLI reports that distinction on exit.

The example includes offline fixtures for clean completion, absent sources, streamed errors, premature closure, duplicate terminal events, HTTP and transport failures, silence timeouts, and a missing API key. These tests exercise client handling without sending Research requests.

Use the trace in your application

Use a managed Research call when your application needs the finished answer and its source record without operating the research loop. Keep orchestration in your own application when you need to control individual searches, fetches, and prompts, or change direction between steps. A search API fits workflows that need links and snippets for the application’s own research process.

Your application sets the acceptance rules for the returned answer. Check decision-driving claims against their sources and choose how to handle missing evidence. The trace supplies the request record and citation mapping; the review supplies the support decisions.

Research runs on hosted infrastructure and sends content and instructions to contracted model providers. Use public, non-sensitive questions for this example. Read What “private by default” means for a managed web API and the Data Handling documentation for the processing path and settings, with the Privacy Notice as the canonical policy.

The trace does not record billing usage. Check your console for the request’s cost.

Reproduce and review the example

The original sample records Python 3.12.13, uv 0.11.28, and macOS arm64. Its reproducibility references are commit 7fe5f20 and the manifest’s 82b9be9, the same tree before a commit-message change. Keep those recorded versions and commit references when comparing the historical artifacts with a new run.

Run the traced Python example with one public question your application answers. Open report.md beside review-sheet.csv, follow one citation to its passage, and record your decision. Review the remaining material claims and required answer elements before using the answer downstream.

Use What makes a citation useful in an AI-generated answer? for the review rubric. The Research guide and API reference define the stream and response fields.

What makes a citation useful in an AI-generated answer?

Mon, 28 Sep 2026 14:00:00 GMT

A citation should let you check the claim beside it. If an answer says a library supports Python 3.12, the linked documentation needs to support that version claim. A link to the library’s homepage leaves you to find the evidence yourself.

We use the same standard for API limits, prices, policy terms, and commands. Find the relevant passage, preserve its qualifiers, and check that it applies to the version or date in the answer.

Check the claim against the passage

A material claim changes what someone decides or builds. A supported version, a request parameter, or a data-retention policy qualifies. A sentence connecting two sections usually does not need a citation.

Start with the exact claim. Open its cited page and find the passage that supports it. Check the passage’s scope before using the claim: an enterprise-only feature does not apply to every plan, and a release note describes the version it names.

One sentence can contain several claims. Separate them when they need different evidence.

Review one sentence from a Research answer

The Python Research example includes a recorded fast-mode run that asked:

What are the current ways to add web search to an Ollama-based application, and what output does each approach return?

The saved report contains this sentence:

The Ollama Web Search API, released on September 24, 2025, requires an API key and performs a POST request to https://ollama.com/api/web_search with a query and optional max_results (default 5, maximum 10) [1][2][3].

The source record maps those markers to these entries:

Marker	Returned URL	Page
`[1]`	`https://docs.ollama.com/capabilities/web-search.md`	Ollama’s Web Search documentation
`[2]`	`http://docs.ollama.com/capabilities/web-search`	The same documentation page
`[3]`	`https://ollama.com/blog/web-search`	Ollama’s announcement

Entries [1] and [2] point to the same documentation page. The sentence cites two distinct pages, both published by Ollama. Count the pages separately from the citation markers.

The documentation states that the API requires a key, accepts a POST request at the named endpoint, and takes a required query. It also defines max_results as optional, with a default of 5 and a maximum of 10.

That supports the request details. The documentation does not establish the release date, so leave the date out until you verify it against a dated source.

Here is the revised sentence:

Ollama’s Web Search API requires an API key and accepts a POST request to https://ollama.com/api/web_search with a required query and an optional max_results, which defaults to 5 and has a maximum of 10 [1].

We retained the supported details and attached their source. We removed the date rather than carrying an unchecked claim into the revision.

The report’s opening sentence lists four approaches without attaching citations. That creates a different review task: locate the evidence for each approach. A missing citation tells you the answer has not shown its support; it does not establish that the claim is false.

Check support, scope, and freshness

For each material claim, answer these questions:

Check	Question	Example failure
Resolution	Can you open the cited page and find the relevant passage?	A documentation homepage links to the parameter’s actual page but does not describe it.
Support	Does the passage support the claim?	The answer names a feature the cited page never describes.
Scope	Did the answer preserve the source’s qualifiers?	The answer describes a paid-plan feature as available on every plan.
Time	Does the source apply to the date or version in the question?	An older release note supports behavior that has since changed.

These checks catch different errors. An accessible page can describe the wrong version. A current page can support part of a sentence while leaving another part unchecked.

If current primary sources disagree, identify the conflicting statements and check their versions and scope. Narrow the answer to what the evidence establishes, or leave the disputed point unresolved.

Check whether the answer covers the question

Claim support and answer coverage require separate checks.

The Ollama question asks for both the available approaches and the output each approach returns. An answer that names the approaches but omits their outputs remains incomplete, even if every named approach has a valid citation.

Write down the required answer elements before reviewing the output. Then mark each element as answered, partially answered, or missing. This keeps a polished paragraph from hiding an unanswered part of the question.

Keep a review record

Record one row per material claim. Split a sentence into multiple rows when its clauses need different sources or decisions.

Field	What to record
`claim_id`	A stable identifier for the claim.
`answer_text`	The exact sentence or clause, including its qualifiers.
`citation_ids`	The attached citation markers; leave empty when none exist.
`source_url`	The page you inspected.
`passage`	A short supporting or contradicting excerpt.
`source_date_or_version`	The relevant date or version; record unknown when the page does not establish it.
`retrieved_at_utc`	When you opened the source.
`support`	`2`: supported at the stated scope; `1`: partially supported or needs a qualifier; `0`: unsupported or contradicted after inspection; `U`: not inspected or could not inspect.
`reason`	The specific evidence gap or reason for the decision.

Keep citation problems separate from factual decisions. Flag missing markers and duplicate source entries without treating them as proof that a claim is wrong.

Report supported, partial, and unsupported claims over the claims you inspected. Report U separately. Measure coverage against the required answer elements, not the number of sentences. If you inspected no claims, report the support rate as unavailable.

Save the passage and retrieval time so the record shows what you checked if the page later changes. This example demonstrates a review method; it does not measure citation accuracy across Tabstack Research.

Use the source record from Tabstack Research

Tabstack Research handles source selection and synthesis inside one call, then returns a Markdown report and citation metadata on the complete event.

The source list uses citedPages in the wire format and cited_pages in the Python SDK. Treat a missing list as empty. In fast mode, each source’s claims array is empty; use the report’s numbered citation markers to trace statements to their source entries. Balanced mode populates the claim lists.

The traced Python example preserves source order so each citation number continues to point to the correct page. It generates a review sheet with the answer text, citation markers, and source URLs. You then add the supporting passages and review decisions.

Your application sets the acceptance rules. Check decision-driving claims before using them downstream, and choose an appropriate review process for lower-risk answers. Automated checks can flag missing markers, out-of-range references, inaccessible links, and matching URL patterns. A verifier still needs to assess whether the passage supports the claim.

Review one answer

Choose a public question your application answers today. List what a complete answer must contain, then check one material claim against its cited passage. Record the evidence and decision before moving to the next claim.

Use the traced Research example to prepare the review sheet and keep the source mapping intact.

Where model-operated web research breaks

Fri, 25 Sep 2026 14:00:00 GMT

Search returned ten results, the interface listed them, and the model answered as though it had never heard of the internet. That failure has at least four possible homes, and the one thing you can be sure of is that it isn’t in the search provider.

Web research fails after search succeeds more often than it fails at search. Discovery, retrieval, extraction, context assembly, the tool loop, synthesis and citation are separate boundaries with separate failure modes, and a visible success at one of them tells you nothing about the next. What follows is nine classes, each with a signature you can observe, the evidence that separates it from its neighbours, and the smallest test that catches it.

Every public example below is a single report in a named version and configuration. They show the classes are real and distinct from each other. They say nothing about how often any of them happens, and I’m not using them to make a claim about any project’s current state.

View the diagram in the original post.

When you write up an incident, name two boundaries: the last one you can prove worked, and the first one you can prove didn’t.

1. Scope and planning

The answer is about the right topic and doesn’t do the job. A comparison names the products and never says what each one returns. A question about current behaviour comes back built from undated pages. A three-part question gets its easiest part answered well.

This is the only class that happens before anything runs. Nobody wrote down what a complete answer contains, so the loop stops when it finds relevant text rather than sufficient evidence, and whoever reviews the output invents the criteria after reading it. By then the answer sets the standard it’s judged against, which is exactly backwards.

Freeze the required elements first, then score each one as answered, partly answered or missing. A well-written answer earns no points for coverage it doesn’t have.

2. Discovery

Search comes back with too few candidates, the same domain five times, stale pages, or results that cover one subquestion and ignore the rest. Open WebUI discussion #18302 shows the symptom plainly: on v0.6.32 the reporter saw one or two sources fetched for most queries, and short, thin answers as a result.

The cause is usually one broad query standing in for a multi-part question, or a loop that stops after the first relevant page instead of searching again while coverage is still short.

Keep every query you issued, the ranked results with their domains, and which required element each candidate might support. Discovery passes when every required element has a plausible candidate. A non-empty result list is not the bar.

3. Retrieval

The URLs are visible and the pages never reach the next stage. The dangerous version is the one where the interface lists the URLs, everyone assumes the pages were read, and the model answers from what it already knew.

This class is quiet by design, because fetch failures are usually caught and skipped. In the thread on Open WebUI #25038, a commenter traced the “No sources found” behaviour on v0.9.5 to a keyword argument passed twice in the page loader. It raised a TypeError on every URL before any network request, and because the loader runs with continue_on_failure=True, the only trace was a warning per skipped fetch. Search worked. The UI listed results. Nothing was read.

Open WebUI’s troubleshooting guide describes a quieter relative: Wikipedia, Cloudflare-protected sites and other large publishers filtering the default Python user agent, so pages come back empty or as 403s.

Per fetch, keep the requested and final URL, the status or browser outcome, content type, size, redirect chain, duration and a truncation flag. Then test it properly: pick a public page containing a unique sentence, fetch it through your production path, and check the sentence survives into stored content. A fetch counter going up proves nothing at all.

4. Extraction and normalisation

The page was fetched and the useful part is missing, mangled, or buried under navigation.

Ollama issue #12690 shows what happens when nothing selects. On v0.12.6, a web search through Qwen3 in the desktop app returned 102,207 tokens for a simple question, because it passed the full contents of five pages, one of which was a Wikipedia article of roughly 100,000 tokens on its own. The same question through gpt-oss:20b-cloud came back at 768 tokens. For scale, Ollama’s web search docs suggest a context of at least around 32,000 tokens, so one unfiltered page can be three times the whole budget.

Keep hashes of the raw and normalised content, both lengths, the extraction method and version, and where truncation happened. Test with one expected passage and one known block of boilerplate: the normalised output keeps the passage, loses the boilerplate, and fits a budget you declared before the run.

5. Context assembly

Search and retrieval both succeed and the model behaves as though it has no evidence at all. This is where I’d put the most monitoring, because every check upstream of it passes.

Open WebUI produced two versions of it in consecutive releases. In #25038, on v0.9.5, results appeared in the interface, then source processing ended with “No sources found” and a traceback from the web search handler, where a JSONResponse was being indexed as if it were a dictionary. In #25585, after an upgrade to v0.9.6, the logs recorded nine items added to the web search collection, no errors anywhere, and a model answering as though no search had run. Commenters traced that one to source resolution: the function had no branch for web search items, and after an access-control change the fallback treated the collection name as untrusted and dropped it silently. Status history showed zero sources retrieved. A fix was proposed in pull request #25600. #25038 has since been closed, with the closing comment saying the reported problems were fixed by 0.10.2 given correct configuration, though a later comment describes similar behaviour on v0.11.3.

Both reports are tied to their versions and setups. What makes them worth reading is that a search provider health check would have passed throughout, and in #25585 the logs actively reported success.

So keep source IDs at every hop: candidates, retrieved, placed in context, present in the final model input. The test is a sentinel. Put one unique, harmless fact into a controlled source and check that the exact fact reaches the model input and can be returned with its source ID. Never infer delivery from a “search complete” event in the UI.

6. Tool loop and state

The model calls search, then loses the question, loops on the same tool, or stops with an empty reply.

Ollama issue #15895 is a clean specimen. On v0.22.0, Gemma 4 reasoned for a while, called web search, and once the tool result arrived it produced an empty reply, having lost both the prompt and the chat context. A follow-up on the thread put it down to truncation rather than compaction, with the user’s own message falling outside the window, and another comment noted Qwen3 wasn’t affected in the same setup.

Keep the ordered message sequence with roles, tool-call IDs matched to their results, context size before and after each call, the stop reason and the final message length. Then test the message adapter away from live search: replay a fixture with the question, one tool call and one small result, and check the model can restate the question, use the result and produce a non-empty answer.

7. Synthesis, coverage and uncertainty

The sources arrived and the answer still misses required elements, blends facts that conflict, or sounds more certain than the evidence allows.

Most systems have no coverage check between “context assembled” and “start writing”, and a writing prompt that rewards finishing over admitting a gap will always find a way to finish. Two sources describing different versions get treated as interchangeable, and the first plausible story wins.

Build a fixture with two sources that explicitly disagree and one required element with no evidence at all. A passing answer scopes the conflict and says the evidence is missing. A failing one resolves the disagreement by inventing a reason to prefer one side.

8. Citation integrity

There are links, there are numbered references, and a cited page doesn’t support the claim sitting next to it. The variants: important claims with no citation, URL variants of one page counted as separate sources, and a source list with no mapping back to claims.

Our own Week 1 sample is a fair example, and I’d rather use it than pretend managed output is immune. The committed run used Tabstack Research in fast mode with SDK 2.8.5 on 15 September 2026. It completed cleanly and cited seven pages. Three of those seven were the same Ollama documentation page under different URLs: once over http, once over https, once with a .md suffix. Every claims array came back empty, and the optional relevance and reliability fields weren’t there. The Research guide’s worked example shows populated claims; this run didn’t return them.

None of that means the answer was wrong. It means three separate measurements were being read as one: whether citations are present, how many distinct sources there are, and whether each claim is supported.

Check every material claim against its cited page and score the support as direct, partial, absent or inaccessible. Report correctness and completeness separately, and don’t award a pass because the links are there.

9. Terminal and operational

The interface searches forever, a stream closes before any terminal event, an HTTP failure gets handled as a task failure, or missing output becomes a successful empty answer.

Most of these live in the consumer. A handler that only listens for success turns every failed run into silence. HTTP errors and in-stream errors sharing a handler lose the information about which phase died. A fixed total timeout kills healthy long runs. Retries can repeat work and cost, and they can be on without anyone choosing them: the Tabstack Python SDK retries a request twice by default on connection errors and on 408, 409, 429 and 5xx responses.

Replay three fixtures at minimum: a successful complete, an in-stream error, and a stream that ends with neither. Each should produce a distinct result, and both failures should exit non-zero.

The nine, on one page

Class	Signature	Minimum evidence	Smallest test
Scope and planning	Relevant answer omits required parts	Frozen question and required elements	Score coverage against criteria written first
Discovery	Too few candidates, or a narrow source set	Queries and result list	Run fixed queries and inspect candidates
Retrieval	URLs exist, pages absent, blocked or truncated	Status, redirects, size, truncation	Fetch a known page through the production path
Extraction	Page fetched, evidence missing or malformed	Raw and normalised content hashes	Compare an expected passage with the output
Context assembly	Fetch succeeds, model gets no usable sources	Source IDs at each hop	Inject a sentinel fact, inspect model input
Tool loop and state	Question lost, tools repeated, empty stop	Ordered messages and tool events	Replay a frozen conversation fixture
Synthesis and coverage	Answer incomplete or overconfident	Required elements and the answer	Score coverage and uncertainty handling
Citation integrity	Links don’t support claims, sources duplicated	Claim-source pairs, normalised URLs	Check each material claim against its citation
Terminal and operational	Stream closes, times out, or errors with no result	Event sequence, timestamps, terminal status	Replay complete, error and premature-close fixtures

They cascade

Two of the reports above fit together into a chain worth recognising. #12690 shows a single search returning six figures of tokens. #15895 shows what truncation does to the user’s message once the context overflows. They come from different setups, so read this as how the classes connect rather than as one incident:

View the diagram in the original post.

The visible symptom sits four steps from the cause, and the symptom points at the model. Diagnose from the last boundary with positive evidence instead.

Triage order

Did the application define the required result? If not, freeze the acceptance criteria before touching anything else.
Did search return plausible candidates for every required element?
Were the selected pages actually retrieved? Check status, redirects and truncation.
Did normalisation keep the evidence? Compare content before and after extraction.
Did that evidence reach the model input? Check source IDs and a sentinel.
Did the tool conversation keep the question and the message order?
Did the answer cover the required elements and state its uncertainty?
Do the citations support the material claims?
Did the run reach an explicit terminal state? A non-empty UI is not evidence of success.

The order exists to stop you swapping the model when the evidence never reached it. That’s the most expensive wrong move available, and it’s the first one most teams make.

Moving the loop out of the model

A managed Research call takes query planning, retrieval, gap checks, synthesis and citation assembly out of your code. Classes 2 to 6 stop being yours to instrument, which is a real reduction in surface area. Failure still happens, in fewer places you control.

Class 1 stays with you, because you still own the question. Classes 7 and 8 stay with you as review work, and the Week 1 sample above is the argument for taking that seriously. Class 9 moves into your consumer: request validation, HTTP errors, in-stream errors, a stream that closes early, optional citation metadata that isn’t there.

I’d use search or model-operated tools when you want links, want control over each step, or the loop itself is the product. I’d use a managed call when the application needs a finished cited result and you’d rather not have the model operating every intermediate stage. Either way, a complete event proves the job terminated and nothing more.

Write the next one up properly

When the next web-enabled answer fails, record the environment and exact versions, the frozen test question or a sanitised hash of it, the answer elements you expected, the last boundary known to have worked, the first known to have failed, sanitised logs or fixtures, whether it reproduces, and whether the workaround changes the architecture or just hides the symptom. Then give it one primary class and note the downstream effects.

A useful incident report lets another engineer reproduce the boundary. A bad one shows them the answer and asks them to guess.

Never trained on. Private by default. Built by Mozilla.

Add current, cited answers to your model application without touching the model

Wed, 23 Sep 2026 14:00:00 GMT

Your application already runs a model. Someone asks it about something that happened last month, and it answers confidently from training data that ended long before. The fix people reach for first is a bigger model, or a search tool bolted onto the one they have. Both of those change the thing that currently works.

There’s a cheaper move. Add a second path at the application layer, send the current question to a managed Research call, and return the result through the response shape your application already uses. The model keeps its prompt, its settings and its context window. It just stops being the component responsible for knowing what happened last month.

Here’s what that looks like in Python, with the parts that bite.

Don’t let the model decide

The first design decision is routing, and I’d make it boring on purpose. The application takes an explicit mode:

model-web-app chat --prompt "Explain our local cache policy"
model-web-app research --question "What are the current ways to add web search to an Ollama-based application?"

Letting a model classify each request as chat or research is the version everyone wants to build, and it’s the version you can’t test deterministically. Every failure then has two possible causes, and the first thing you’ll do when the integration misbehaves is argue about whether the classifier was right. Get the integration working with an explicit mode, prove the model path is untouched, then add classification as its own change with its own tests.

View the diagram in the original post.

The contract belongs to you

The rest of your application should never see an SDK event object. Give it something you own:

from dataclasses import dataclass, field
from typing import Literal

@dataclass(frozen=True)
class SourceRecord:
    id: str
    url: str
    title: str | None = None
    claims: list[str] = field(default_factory=list)
    source_queries: list[str] = field(default_factory=list)

@dataclass(frozen=True)
class ApplicationResponse:
    kind: Literal["chat", "research"]
    content_markdown: str
    sources: list[SourceRecord]
    status: Literal["complete", "partial"]
    provider: str
    retrieved_at_utc: str | None = None

Two fields in there are doing quiet work.

sources stays ordered. Tabstack documents citedPages as ordered by first citation in the report, and the report’s inline [1], [2] markers refer to that order. Sort it or deduplicate it in the adapter and you’ve broken the only claim-to-source mapping you get in fast mode.

status has a value other than complete so a partial result has somewhere to live. This integration never produces one, because a Research call ends in either complete or error. It’s there so that when you add your own retrieval later, or when a call gives you a report with half its sources unreachable, the answer to “what do we return?” isn’t “a confident-looking answer with a gap in it”.

The adapter

This is the whole of it:

import os
from typing import Literal

import httpx
import tabstack
from tabstack import Tabstack

def run_research(
    question: str,
    mode: Literal["fast", "balanced"] = "fast",
    nocache: bool = True,
) -> ApplicationResponse:
    if not os.environ.get("TABSTACK_API_KEY"):
        raise ConfigurationError("TABSTACK_API_KEY is not set")

    with Tabstack(max_retries=0) as client:
        try:
            with client.agent.research(query=question, mode=mode, nocache=nocache) as stream:
                for event in stream:
                    if event.event == "error":
                        raise ResearchTaskError(
                            message=getattr(event.data.error, "message", None) or "unknown error",
                            activity=event.data.activity,
                        )
                    if event.event == "complete":
                        return to_application_response(event.data)
                    report_progress(event)
        except tabstack.APIStatusError as err:
            raise ResearchHTTPError(status=err.status_code) from None
        except tabstack.APIConnectionError:
            raise ResearchTransportError("could not reach Tabstack") from None
        except httpx.TransportError as err:
            raise ResearchTransportError(f"stream interrupted: {type(err).__name__}") from None

    raise ResearchProtocolError("stream ended without complete or error")

Four lines there aren’t obvious.

The environment check happens before the client is built. Tabstack() raises a TabstackError from its constructor when neither api_key nor TABSTACK_API_KEY is set, and a missing environment variable deserves its own exit code and its own message rather than arriving as a generic SDK error.

except httpx.TransportError is there because the SDK’s coverage stops at the stream boundary. Anything that fails before the stream opens comes back as an SDK exception: AuthenticationError, RateLimitError and friends under APIStatusError, or APIConnectionError and its APITimeoutError subclass. Once you’re iterating, the stream iterator in 2.8.5 passes transport failures straight through, so a read timeout or a dropped connection mid-stream arrives as a raw httpx exception. Catch only the SDK types and a stream that dies halfway takes your process with it.

The error event’s payload carries message, name and an optional stack. Keep the message and the activity, drop the stack. Provider stack traces in your logs are noise at best, and a leak at worst. The defensive read on message is there because the guide warns the error object can arrive unpopulated despite being typed as required.

from None keeps the SDK’s exception chain out of your tracebacks, which is the cheapest way to stop a key or a header ending up in a log aggregator.

The final raise is the case people forget. If the stream ends without complete and without error, the loop just finishes and the function returns None unless you make that an error. A silent None becomes an empty answer three layers up, and nobody can tell it apart from a model that had nothing to say.

Timeouts, and the one everybody gets wrong

Research has no server-side timeout on the call as a whole. A fast query usually comes back in under a minute. A broad balanced query consults more sources and can run for around four minutes. So don’t wrap it in a total deadline. A fixed 90-second timeout will kill healthy long runs and you’ll conclude the API is flaky when the problem is your own stopwatch.

What you want is silence detection, and you already have some. The SDK’s default is httpx.Timeout(connect=5.0, read=60, write=60, pool=60), and httpx applies that read timeout to each read, so on a stream it fires when nothing has arrived for sixty seconds. That’s the behaviour you want, just with a number you chose deliberately rather than inherited. If you add your own silence detection on top, reset it on every event.

One parameter that is not a total deadline, despite reading like one: fetch_timeout caps how long Research spends fetching a single page.

Retries, which are probably already on

    with Tabstack(max_retries=0) as client:

That max_retries=0 is deliberate. The SDK retries twice by default, on connection errors and on 408, 409, 429 and 5xx responses. The committed Week 1 sample run recorded application_retries: 0 in its manifest alongside sdk_max_retries: 2, which is a neat illustration of how “we don’t retry” can be untrue in a codebase nobody has lied in.

A replayed Research call can repeat billed work, and there’s no idempotency key to make it safe. Turn retries off until somebody has decided which failures are safe to replay and what the ceiling is, then turn them back on with that decision written down.

What actually comes back

The Week 1 example repository has a committed run of this shape: fast mode, SDK 2.8.5, 15 September 2026. It completed in 20.3 seconds with the first event arriving after 1.06 seconds, analysed seven pages, cited seven, and returned a 1,649-character report. Ten events, each once, in order: start, planning, one iteration with a search inside it, writing, complete.

The interesting part is what the citation metadata did and didn’t contain.

Three of the seven cited pages were the same Ollama documentation page under different URLs: once over http, once over https, and once with a .md suffix. Your distinct source count is not your cited page count, and normalising scheme, repeated slashes, trailing slashes and file suffixes before counting is a ten-line function that stops you reporting seven sources when you have five.

Every claims array came back empty. The field is typed as always present and it was present, carrying nothing, because fast mode skips the analysis phases that populate it. The guide’s worked example shows it populated. That’s the gap between a documented shape and a live response, and it’s why the adapter maps claims into the contract without ever depending on its contents. In that mode the report’s inline [n] markers, read against the source order, are the claim-to-source link.

relevance, reliability and summary were all absent, as the SDK docstrings say they are in fast mode. title was present on every page. And complete.data.timestamp arrived as epoch milliseconds while the guide’s example shows an ISO string, so set retrieved_at_utc from your own clock and store the raw value alongside it.

None of that makes the output unusable. It does mean you build against the response you get rather than the example you read, and that you read citedPages and gapEvaluations defensively, since both are optional.

What to log

Log enough to find the boundary that failed, and nothing that belongs to the person who asked the question. The event names in order, with timestamps. Time to first event and to terminal state. The terminal status and which phase it failed in. Counts of pages analysed, cited pages raw, and distinct URLs after normalising. Whether the model adapter was called, which for this path should always be false. The SDK version, its max_retries and its effective timeout.

Keep out API keys, authorisation headers, cookies, whole environment dumps, and model reasoning.

When this is the wrong shape

If what you need is links, use a search API and skip all of this. If the web layer is what makes your product different from the next one, own the loop yourself and accept the boundaries that come with it. If you need the model to make decisions mid-research based on what it finds, you want a tool loop, not one call.

This shape fits when your application needs a finished, cited answer to a current question, and the model you run is good at everything except knowing what happened last month. You get to keep the model, and the parts you still own are the question, the review and the decision about whether the answer is good enough to act on.

A complete event means the job finished. That’s all it means.

Never trained on. Private by default. Built by Mozilla.

The seven stages of web research for a model you run

Mon, 21 Sep 2026 14:00:00 GMT

A web research system you can trust with a current question does seven separate jobs. It defines what a complete answer looks like, discovers candidate sources, retrieves them, extracts evidence, checks that evidence for gaps, reconciles conflicts, and writes a cited result.

Splitting the work up this way won’t make an answer correct on its own. What it does is give every failure an address, so when an answer comes back wrong you can point at the stage that broke instead of blaming “the model”.

This is for anyone running their own model inside a product, an assistant, an agent or an internal workflow, where users ask about things the training data can’t know.

The loop

View the diagram in the original post.

The arrow from Evaluate back to Discover is the part most implementations leave out, and it’s why I call this a loop and not a pipeline. When one piece of the answer has no evidence behind it, the system has three honest options: search again, narrow the claim, or hand back an incomplete result. Writing confident prose over the hole isn’t on that list.

Write the answer contract before you search

I’ll use one question the whole way through, because it’s the sort of thing people really do ask these systems:

Which web search approaches can an Ollama-based application use today, and what does each one return?

Before any searching, write down what a usable answer has to contain:

{
  "question": "Which web search approaches can an Ollama-based application use today, and what does each one return?",
  "required_elements": [
    "approach name",
    "where the web workflow runs",
    "what the application receives",
    "whether the model must still build the answer",
    "a current source for each material product claim"
  ],
  "output": {
    "answer_format": "markdown",
    "citations": "inline numbered references",
    "source_record": "machine-readable ordered list",
    "incomplete_evidence": "must be stated"
  }
}

This feels like overhead the first time. It’s also the only thing that lets you decide afterwards whether a fluent answer was a complete one. A lot of “relevant but useless” answers come from nobody deciding up front what useful meant.

Stage 1: Define the question and what done looks like

“Web search approaches” is ambiguous. It could mean a raw search API, search plus page fetching, a tool loop the model drives itself, or a managed research call. Each reading is legitimate and each produces a different answer, so the first stage turns the request into a research objective, a set of subquestions, the required elements above, a freshness boundary and an output shape. “Today” in the question is doing real work here: it means current documentation, not a blog post from two years ago.

The test I’d apply is whether a reviewer could mark the answer complete or incomplete using only what this stage produced. If they’d have to invent criteria after reading the answer, the stage isn’t finished.

When the ambiguity would materially change the work, stop and ask. A silent guess at this point poisons everything after it, because every later stage will carefully execute the wrong job.

Stage 2: Discover candidate sources

Discovery finds pages that might contain evidence. Nothing is established as true yet.

A multi-part question needs more than one query. For the Ollama example I’d want one aimed at Ollama’s own search and fetch documentation, one at how Open WebUI wires search into a chat, and one at a research API’s response contract. A single broad query tends to return the loudest pages, and they’re rarely the authoritative ones.

Keep the provenance as you go: which query surfaced each page, and which subquestion it’s supposed to answer. Throw that away and any later debugging turns into guesswork.

This stage fails in a few recognisable ways. A required element has no candidate at all, the only candidates are derivative (the blog post summarising the docs rather than the docs), the results are duplicates of each other, or everything sits outside the freshness boundary.

Stage 3: Retrieve readable source content

A search result is a lead, and the page it points at is the evidence. Ollama’s web search API is a tidy illustration of the split: each result is a title, a URL and a content snippet, and a separate web fetch endpoint returns the page’s title, main content and links. The interface is honest about what search gives you.

Retrieval fetches or renders the chosen pages and turns them into something the next stage can inspect. Record the status, any redirects, a timestamp, the content size, and whether anything was truncated.

Truncation is the quiet failure. Open WebUI’s web search troubleshooting guide notes that web pages often run to 4,000 to 8,000 tokens or more, and that a model running at Ollama’s 4,096-token default context misses most of that content. Nothing errors when it happens. The model answers from part of a page and sounds exactly as sure of itself as it would with the whole thing.

A successful search followed by a failed fetch is a partial run with a known boundary, and it should be reported as one.

Stage 4: Extract claims with their context

A page can be perfectly readable and still useless as evidence. This stage pulls out the statements that bear on the question and keeps the qualifications attached to them.

At minimum I’d keep this much per claim:

{
  "claim": "Ollama's web search API returns a title, URL and content snippet for each result.",
  "source_url": "https://docs.ollama.com/capabilities/web-search",
  "source_title": "Web search",
  "supporting_excerpt": "content (string): relevant content snippet from the web page",
  "retrieved_at": "2026-09-21T00:00:00Z",
  "scope": "documented response fields for the web_search endpoint only",
  "supports_required_element": "what the application receives"
}

The scope field is the one that earns its place. “The Ollama web search API returns snippets” is true. “Ollama-based applications only get snippets” is false, because the same page documents the fetch endpoint that returns page content. Extraction is where a qualified statement gets quietly promoted to an absolute one if nothing stops it.

It’s also where citations start meaning something. A list of URLs at the bottom of an answer is a bibliography. A record of which claim came from which page is something you can check.

Stage 5: Evaluate coverage, source quality and gaps

This is the stage that separates research from summarising whatever came back first. Compare the evidence against the contract you wrote at the start:

Required element	Evidence found	Source quality	Fresh enough	Decision
Approach name	Yes	Official	Yes	Keep
Where workflow runs	Partial	Official plus inference	Yes	Narrow wording
Output returned	Yes	Official docs	Yes	Keep
Model still builds answer	No direct source	Architectural inference	N/A	Label as analysis, or search again
Current source	Yes	Official	Yes	Keep

The row worth looking at is “model still builds answer”. You’re unlikely to find a provider page that states it outright, because it’s a conclusion drawn from what each interface returns. That’s fine, provided the answer labels it as analysis. What isn’t fine is attaching a citation to a page that doesn’t say it.

If a material gap remains, go back to discovery with a narrower query. Don’t pad the context with more loosely related text and hope the volume covers the gap. It won’t, and it makes the next stage harder.

Stage 6: Reconcile conflicts and uncertainty

Sources disagree for boring reasons. They describe different versions, plans, dates, regions or definitions, and usually neither one is wrong. Reconciliation keeps the conflict visible and works out whether it can be resolved:

Compare publication and update dates.
Check that both sources describe the same thing at the same version.
Prefer the source closest to the claim. For product behaviour that’s usually the current official docs.
Separate a factual disagreement from two sources using the same word differently.
Narrow the claim when the evidence only supports a limited version of it.
State the uncertainty when nothing authoritative settles it.

This is the stage I’d trust automation with least. Comparing dates and preferring official documentation is mechanical. Deciding whether two pages are describing the same object, or two objects with the same name, is judgement, and a system that quietly picks the convenient source will look fine in every demo you run. If you’re only adding human review in one place, put it here.

The failure mode is exactly that quiet pick, or its cousin: two incompatible claims blended into one sentence that neither source supports.

Stage 7: Synthesise the answer and keep the source record

Only now does the system write the thing the application asked for, and the result should carry more than prose:

{
  "status": "complete",
  "answer": "The approaches differ mainly in what they return and who operates the work after search. ... [1]",
  "sources": [
    {
      "id": "src_01",
      "url": "https://docs.ollama.com/capabilities/web-search",
      "title": "Web search",
      "claims": [
        "Ollama's web search API returns a title, URL and content snippet for each result."
      ]
    }
  ],
  "unresolved": [],
  "retrieved_at": "2026-09-21T00:00:00Z"
}

Two fields matter more than they look. status needs a value other than complete, otherwise a partial run has nowhere to go but into a confident answer. And an empty unresolved array should be something the system concluded, not the default because nobody wired it up.

Synthesis fails as a fluent answer with gaps in its support, citations that point at nothing in particular, no way to express a partial result, or output your application can’t parse.

The seven contracts on one page

Stage	Takes	Produces	Check	Failure state
Define	User question, application requirements	Objective, subquestions, required elements, freshness boundary, output contract	Could a reviewer judge completeness from this alone?	Question left broad, undefined or unverifiable
Discover	Objective and subquestions	Candidate URLs with the query and subquestion behind each	Does every required element have a plausible candidate?	No source, derivative sources only, duplicates, stale candidates
Retrieve	Selected URLs	Readable content plus status, redirects, timestamps, size, truncation flag	Did we get the part of the page that holds the claim?	Blocked, timed out, unrendered, redirected elsewhere, empty or silently truncated
Extract	Retrieved content and answer criteria	Claim-to-source records with excerpt and scope	Does the excerpt support the claim at the same scope?	Snippet-only evidence, overbroad claims, lost qualifications, no trace back to the page
Evaluate	Claim records and required elements	Coverage matrix, source quality, duplicates, gap list	Is each element supported, qualified or marked unavailable?	Relevant pages that don’t cover the actual question
Reconcile	Supported claims and conflicting evidence	Resolved claims, scoped readings, unresolved items	Can a reviewer see why one source won?	Convenient source chosen quietly, incompatible claims blended
Synthesise	Reconciled evidence and output contract	Answer, citations, source record, limitations	Does every material claim have inspectable support?	Unsupported fluency, orphaned citations, no partial state, unparseable output

The handoffs are where systems break

A visible success at one boundary tells you nothing about the next one.

Visible event	What it proves	What it doesn’t prove
Search returned URLs	Discovery worked	The pages were read, or reached the model
Pages were fetched	Retrieval worked	The relevant claims were extracted
Context was assembled	Data reached a context layer	The model used it correctly
Answer contains links	Citation strings exist	The links support the claims beside them
API emitted complete	The job terminated successfully	The answer is correct or complete

Two public Open WebUI issues from this year show the gap between the first and third rows. In #25038, on v0.9.5 with SearXNG, results came back and appeared in the interface, then source processing ended with “No sources found” and the model claimed web search wasn’t available. In #25585, after an upgrade from v0.9.5 to v0.9.6, the logs recorded nine items added to the web search collection while the model answered as though no search had happened.

These are individual, version-specific reports and I’m not using them as evidence of how often this happens. What they do show is that “search worked” and “the model used the search” are two different facts, and you need a separate check for each.

What should your application log?

Enough to find the failed boundary, without logging secrets or private source content. I’d capture:

request ID and UTC timestamps
a hash of the question, or an approved public test question
the research objective and required answer elements
every search query issued
counts of candidate, fetched, failed, distinct and cited URLs
redirects, fetch status and truncation flags
context size and the source IDs passed to synthesis
terminal status, and the stage it failed at if it failed
where the answer and source record were written
dependency and model versions
duration, and usage figures where the provider reports them

Leave out API keys, authorisation headers, cookies, environment dumps, model reasoning and anything confidential the user sent.

Where should the loop live?

It helps to separate the tools first, because they get lumped together. Search returns links and snippets. Fetch or extract reads a URL you already have and gives you clean content or specific fields. Research takes a question and returns a synthesised answer from sources. Automation performs interactions on a public website when the result needs someone to click through to it.

Approach	Best when	Your application owns
Search API	Links or snippets are the result you want	Any reading, synthesis and citation
Model-operated tools	Your model drives tools reliably and you want the control	Tool orchestration, context, evaluation, retries
DIY research loop	The web layer is what differentiates your product	All seven stages and every boundary between them
Managed Research API	You need a cited result without operating the loop	Question design, acceptance, review, downstream use

My default is not to operate all seven stages yourself unless owning the loop is the product, or you need control over a specific stage that nothing managed gives you. Moving the loop to a provider changes who operates it. Deciding whether the output is good enough for your job stays with you either way.

What a managed call shows you of the seven stages

Tabstack Research is the one I work on, so here’s how its output lines up against the stages. Per the Research guide, you send a question and it always streams Server-Sent Events: a start, a planning phase, one or more search iterations, a writing phase, then a single complete event carrying the report and its metadata.

Stage	Where it shows up
Define	researchObjective, researchQuestions and researchPlan in metadata, always present
Discover	executedQueries, plus sourceQueries on each cited page
Retrieve	totalPagesAnalyzed, with per-page fetch time bounded by the fetch_timeout parameter
Extract	claims on each entry in citedPages
Evaluate	gapEvaluations when populated, plus isLast and stopReason on each iteration:end event
Reconcile	No dedicated field. You see the outcome in the report, not the working
Synthesise	report, as Markdown

The reconcile row is the honest gap, and it’s the reason the review step stays with you for anything that matters. citedPages and gapEvaluations are both optional, so read them defensively and treat a missing citedPages as an empty list.

The operational details are where a first integration usually trips. HTTP errors such as a bad key or a rate limit are raised before the stream opens. Task failures arrive as an error event inside the stream, so a consumer that only listens for complete gets nothing at all from a failed run. You also need to handle a stream that closes without either. There’s no server-side timeout on the request as a whole: fast mode typically finishes in under a minute, while balanced consults more sources, needs a paid plan, and can run for up to about four minutes on the broadest questions. A fixed total timeout will kill healthy long runs, so reset a timer on every event and fail on silence instead.

Map your current research loop

Take one current question your application has to answer and fill in these seven rows before you change any tools:

Stage	Current owner	Input	Output	Check	Failure state	Logged?
Define
Discover
Retrieve
Extract
Evaluate
Reconcile
Synthesise

Any blank cell in the owner column is work somebody assumed somebody else was doing. That’s the next thing to instrument, or the next thing to hand off.

Never trained on. Private by default. Built by Mozilla.

Search + fetch + model versus a managed Research call: the evaluation plan

Fri, 18 Sep 2026 14:00:00 GMT

We are comparing two complete ways of answering current public-web questions. One is a pinned search + fetch + model pipeline that the application runs itself. The other is a single managed Research call. The questions, the system configurations, the hypotheses, the metrics, the failure rules and the analysis plan all went into Git before either system produced an output anyone looked at. This article publishes that method. It does not publish a winner, because the scored run has not happened yet.

It is written for people running their own model inside a product, an assistant or an internal workflow, who have to decide where the research loop lives.

The protocol went in first

View the diagram in the original post.

Protocol v1 is committed at 116d90191c6d900a97779ac0fc07dc9ffcfc229a. The application-operated system was frozen at 22c1613a8e2546acb429c818d61a5327706004ed. The protocol, question set, rubric, result schema and harness are public in the repository’s evals directory. FULL_EVALUATION_RUN=false.

The question

For current public-web questions, what differences appear between a pinned search + fetch + model pipeline the application operates, and one managed Research call?

That compares two complete configured systems, and it is worth being blunt about the limit this creates. If the outputs differ, we cannot point at search, page retrieval, model choice, orchestration, source availability or the managed service and say which one caused it. Isolating a component needs a different experiment, and a controlled one.

The decision underneath is architectural. Does your application own source discovery, retrieval, context assembly, synthesis and citation formatting, or does it call a managed web API and take back the finished result?

The input and output are held constant

Every run receives one frozen question and the same output instruction:

Answer the question directly.
Support material factual claims with citations to public source URLs.
Distinguish sourced facts from interpretation.
State when evidence is incomplete or conflicting.
End with a Sources section listing only sources used in the answer.

Each run must produce a record with:

{
  "question_id": "Q01",
  "system_id": "redacted during blind review",
  "started_at_utc": "2026-09-15T20:31:39.716Z",
  "completed_at_utc": "2026-09-15T20:31:56.383Z",
  "duration_ms": 16666,
  "terminal_status": "complete",
  "answer_path": "answer.md",
  "sources_path": "sources.json",
  "event_log_path": "events.sanitized.jsonl",
  "usage_receipt_path": "usage-receipt.json",
  "pilot_only": true,
  "scoring_status": "unscored"
}

That is a pilot record, included here to show the harness contract. It is not a scored result.

Both systems were frozen before the test

We are comparing full paths, not an abstract category against a product.

System A: application-operated search + fetch + model

System A uses a pinned web-search API, HTTP page retrieval with redirects and a 30-second fetch timeout, standard-library HTML-to-text conversion, and a dated flagship general model behind a chat-completions interface. The planner and writer prompts are committed. There are no application-level retries. Its fixed resource limits are:

max_search_iterations: 3
max_queries_per_iteration: 3
max_results_per_query: 5
max_pages_fetched_total: 12
max_chars_per_page: 12000
max_total_source_chars: 60000
total_wall_clock_timeout_seconds: 300
application_retries: 0

Model choice is the part of System A most open to challenge, so here is the reasoning behind it. We picked a configuration meant to represent a serious per-query deployment. We did not pick a smaller tier that would quietly weaken the application-operated baseline, and we did not pick a higher-cost specialist tier that a team would never run on every query. The complete provider and version record stays in the committed system configuration.

System B: Managed Research

System B uses:

endpoint: /research
mode: fast
nocache: true
fetch_timeout: default
application_retries: 0
python_sdk: tabstack==2.8.5

The application makes one /research call and receives the streamed report and the cited-page metadata.

Both configurations are part of the public record. If either one changes after anyone has inspected outputs, the protocol version has to change and the full test restarts.

The question set includes work that may not need a research loop

The frozen set contains 12 questions:

Question type	Count	Why it is included
Simple source discovery	3	Test jobs where a list of official sources may be enough
Multi-source synthesis	5	Test questions that require combining several pieces of evidence
Freshness-sensitive	2	Test whether sources and claims meet an explicit date requirement
Conflict or incomplete evidence	2	Test how each path handles uncertainty and unresolved evidence

Each question carries two to five required answer elements, written before execution. The set deliberately includes jobs where a list of official sources might be all anyone needs. A comparison built only around the work one system prefers would not answer the architecture question honestly.

The complete set is in evals/questions.jsonl.

The hypotheses are directional, not conclusions

Five, registered before the run:

The managed path will have higher median citation completeness, because citations are part of its output contract.
The managed path will have higher median question coverage on multi-source and conflicting-evidence questions.
The application-operated path may have lower median wall-clock time on simple questions.
Cost has no pre-registered direction.
On pure source-discovery questions, the application-operated path will be no worse on usefulness within this small sample.

Any of these can be wrong. Publishing them beforehand is what makes being wrong visible.

Separate metrics, no composite score

A single number would conceal the tradeoffs we are trying to see, so every metric is reported on its own.

Answer correctness. Per material factual claim: 2 supported and accurate, 1 partly supported, imprecise, or missing a necessary qualification, 0 contradicted or unsupported. If a qualified evaluator or primary-source adjudication is unavailable, correctness stays unavailable rather than being guessed.

Citation correctness. Per claim-citation pair: 2 the cited source directly supports the claim, 1 the source is relevant but support is partial or indirect, 0 the source does not support the claim or cannot be inspected.

Citation completeness. Cited material factual claims divided by material factual claims. Opinions, transitions and interpretation that is clearly marked as interpretation do not require citations.

Question coverage. Per pre-listed required element: 2 fully answered, 1 partly answered, 0 missing.

Source quality. Per cited source: 3 primary or official source for the claim, 2 strong independent secondary source, 1 weak, derivative, undated or unclear, 0 irrelevant or unusable. We also record source-domain diversity, without assuming that more domains mean better evidence.

Freshness. Where a question defines a date window: 2 source and claim meet it, 1 the date is unclear and no contradiction was found, 0 the evidence is outside the window or stale.

Uncertainty handling. Whether the answer names material gaps and conflicts and scopes its conclusion, instead of turning incomplete evidence into certainty.

Latency. Client-observed wall-clock milliseconds for every attempt, and first-event latency where the interface exposes it. Failures stay in the record. One run is one observation, not typical latency.

Cost. Authoritative usage and billing evidence only. For the application-operated path, the harness records search calls and model input and output tokens, and those records get combined with the providers’ published rates on the run date. The managed path is the awkward one, because the API does not return usage. Tabstack request telemetry recorded one action for the fast-mode calls made while validating the harness, so the cost of a managed call is recorded action count multiplied by the published action rate, and the customer-visible receipt is a console balance change. Published pricing allows a Research call to spend a variable number of actions. One action on a validation call establishes nothing about what the next question will cost.

Terminal status. Every run gets exactly one: complete, partial, unanswered, HTTP failure, transport failure, task failure, or timeout. Provider failures, empty answers and malformed citations stay in the denominator.

The committed rubric and results template define the full field list.

Scoring happens before anyone knows which system wrote what

The harness copies each final answer under a random identifier and strips provider names from the wrapper metadata. Evaluators score answer quality before they see system identity, event logs, latency or cost. The engineer who built the systems will not be the sole scorer.

The blind is incomplete, and we are not going to pretend otherwise. The two systems can produce visibly different answer structures, and an evaluator can infer identity from formatting with every label removed. That limitation ships alongside the results.

Failures stay in the denominator

The protocol does not allow a cleaner result set to be manufactured after the run.

Do not stop because one system appears better.
Stop the whole study for security, runaway spend, a provider outage that breaks comparability, corrupted logging, or a protocol violation.
Exclude a run only when the harness fails before the provider receives the request.
Keep provider errors, timeouts, empty answers, and malformed citations as results.
Never rerun only the system that appears to lose.
If the scoring rules change after outputs are visible, create protocol v2 and rerun the full set.

Operational failure is part of the architecture being evaluated. A path that times out is telling you something about itself, and deleting the row throws that away.

The pilot tested the harness, not the systems

We ran one source-discovery question through both configured paths to check that the harness could send the frozen input, run each complete system, preserve answers and sources, record timing and usage fields, mark the rows pilot_only=true, and package answer text for blind review.

Both paths reached a terminal complete state. Their outputs, logs and operational records are committed under evals/runs/20260915T203139-pilot.

Nobody has scored those answers. They are not ranked, and the rows will not enter the full evaluation denominator. The only thing the pilot concludes is that the harness runs end to end with both frozen configurations.

What this first evaluation will not establish

The planned 12-question run is exploratory. One run per system per question cannot establish universal superiority, stable provider performance, statistical significance, typical latency or cost, reliability across workloads, or which individual component caused an observed difference.

It can show the outputs, costs, timing, failures and evaluation scores that two disclosed configurations produced on a frozen question set. That is enough to identify tradeoffs and design a stronger follow-up. It is not enough to declare a category winner.

Most teams should do something smaller than this

If you only need to know whether one workflow meets a product requirement, run an acceptance test instead. Define the required claims, the citation behaviour, the latency ceiling, the cost ceiling and the failure handling, then test that one workflow against them. It answers the question you actually have, and it takes a fraction of the setup.

If you need causality, run a component-level experiment. Hold the search provider, the fetched pages, the prompt and the model constant, then change one component at a time.

If you need estimates of variability or stable provider behaviour, you need repeated runs over a larger stratified set, with the sample, the analysis and the stopping rules registered before execution.

And when the deciding factor is ownership, privacy, deployment boundary or maintainability, a benchmark is the wrong instrument. Do the qualitative architecture review and skip the scores.

Data flow

Both paths send a public-web question and retrieved content through external services in the frozen configurations. The application-operated path sends data to its chosen search and model providers. The managed path sends the question and retrieved page content to Tabstack, and third-party models may process it under Tabstack’s contracts. Review every provider’s data terms before using confidential inputs. For Tabstack, start with the Privacy Notice.

Never trained on. Private by default.

Submit a workflow or evaluation question

The evaluation is stronger if its questions reflect jobs people are actually trying to ship. Propose a current public-web workflow, a question, and the answer elements that would make the result useful.

Open the evaluation directory and propose a question

Build a cited live-web answer in Python with one Research call

Wed, 16 Sep 2026 14:00:00 GMT

Clone the example, set one environment variable, and run one command to send a current question to Tabstack Research. The call streams progress, then writes a Markdown report and a machine-readable list of cited pages. The committed sample shows the full path completed in a recorded Python environment, with every artifact included so you can reproduce it.

Who this is for: People running their own model inside a product, assistant, or internal workflow that needs a current, cited answer from the public web.

Run the example

Start with the complete implementation in the cited-research example repo rather than assembling a partial snippet:

git clone https://github.com/Mozilla-Ocho/tabstack-cited-research-python.git
cd tabstack-cited-research-python
uv sync --frozen
export TABSTACK_API_KEY=...

uv run cited-research \
  --query "What are the current ways to add web search to an Ollama-based application, and what output does each approach return?" \
  --mode fast \
  --nocache \
  --output artifacts/sample-run

The key comes from the environment. The CLI has no –api-key flag and does not write the key to its output files.

The repository pins tabstack==2.8.5 in uv.lock. The sample was run with Python 3.12.13 and uv 0.11.6 on macOS Darwin 25.6.0, arm64. The project declares Python 3.9 as its floor, but this sample was not executed on Python 3.9.

What the program does

View the diagram in the original post.

Your application sends the question to /research. The endpoint returns a Server-Sent Events stream. The program displays selected progress events, stops on completion, and saves the report and cited-page metadata as separate files.

The application remains responsible for reviewing the result and deciding how to use it. A completed Research call gives you a report and a source record to review; how you use the answer stays in your application.

The Python call

The repository contains the tested CLI, sanitization, file persistence, and exit-code handling. This standalone version shows the same core API path and can be run as a script after installing tabstack==2.8.5 and setting TABSTACK_API_KEY:

import json
import os
from pathlib import Path

import tabstack
from tabstack import Tabstack

QUERY = (
    "What are the current ways to add web search to an Ollama-based "
    "application, and what output does each approach return?"
)
OUTPUT = Path("artifacts/my-run")
PROGRESS_EVENTS = {
    "start",
    "planning:start",
    "planning:end",
    "iteration:start",
    "iteration:end",
    "searching:start",
    "searching:end",
    "writing:start",
    "writing:end",
}

if not os.environ.get("TABSTACK_API_KEY"):
    raise SystemExit("TABSTACK_API_KEY is not set")

OUTPUT.mkdir(parents=True, exist_ok=True)
final = None

with Tabstack() as client:
    stream = client.agent.research(
        query=QUERY,
        mode="fast",
        nocache=True,
    )

    for event in stream:
        if event.event in PROGRESS_EVENTS:
            print(event.event, event.data.message)

        if event.event == "error":
            raise RuntimeError(event.data.error.message)

        if event.event == "complete":
            final = event
            break

if final is None:
    raise RuntimeError("Research stream ended without a complete event")

(OUTPUT / "report.md").write_text(
    final.data.report.rstrip() + "\n",
    encoding="utf-8",
)

sources = []
for source in final.data.metadata.cited_pages or []:
    sources.append(
        {
            "id": source.id,
            "url": source.url,
            "title": source.title,
            "claims": source.claims,
            "source_queries": source.source_queries,
            "relevance": source.relevance,
            "reliability": source.reliability,
        }
    )

(OUTPUT / "sources.json").write_text(
    json.dumps(sources, indent=2, ensure_ascii=False) + "\n",
    encoding="utf-8",
)

print(f"report -> {OUTPUT / 'report.md'}")
print(f"sources ({len(sources)}) -> {OUTPUT / 'sources.json'}")
print(f"tabstack SDK -> {tabstack.__version__}")

Python exposes SDK fields in snake case, including cited_pages, source_queries, and total_pages_analyzed. The JSON wire format uses camel case.

How the stream terminates

/research always returns a Server-Sent Events stream. The sample run received these events, in this order, once each:

start
planning:start
planning:end
iteration:start
searching:start
searching:end
iteration:end
writing:start
writing:end
complete

complete is the successful terminal event. Research does not send a later done event. A task-level failure arrives as an error event inside the stream. That is why the loop must handle both error and complete. HTTP rejections such as 401, 400, or 429 happen before the stream opens and surface as SDK exceptions instead.

Complete input

The publication run used this question:

What are the current ways to add web search to an Ollama-based application, and what output does each approach return?

It used mode=“fast” and nocache=True. The input, exact command, sanitized events, output, and run manifest are committed in artifacts/sample-run.

Complete output

The CLI writes a durable output bundle:

artifacts/sample-run/
├── question.txt
├── command.txt
├── events.sanitized.jsonl
├── report.md
├── sources.json
├── run-manifest.json
├── stdout.txt
├── stderr.txt
└── terminal.png

The complete returned report is in report.md. The ordered citation metadata is in sources.json. They remain separate so an application can render the answer while retaining a machine-readable source record.

The program printed this progress before writing the files:

start  Starting research
planning:start  Planning research strategy
planning:end  Planning complete: 6 queries
iteration:start  iteration 1/1  Starting iteration 1 of 1
searching:start  iteration 1  Searching with 6 queries
searching:end  iteration 1  8 new urls  Found 8 URLs
iteration:end  iteration 1  Iteration 1 complete (fast mode)
writing:start  Writing report
writing:end  Report draft complete
complete  report -> artifacts/sample-run/report.md
sources (7) -> artifacts/sample-run/sources.json

The run in numbers

The run started at 2026-09-15 19:00:04.556 UTC and completed 20.3 seconds later, with the first progress event arriving after 1.06 seconds. The stream ended on complete and the process exited 0. Tabstack analyzed seven pages, cited seven, and returned a 1,649-character report with inline numbered citations.

Request telemetry recorded one Research action for the whole call; at the published fast-mode rate that is 250 credits for a planned, multi-source, cited answer. The API response itself carries no usage field, so treat that as the implied cost.

One run is one data point. Use it to see the shape of a fast-mode result, not to size latency or cost for your workload.

What fast mode returns, and how to read it

Fast mode is built for speed: one iteration, no judge pass, sources returned as they were found. That shows up in the output in four specific ways, and each has a straightforward handling pattern.

Cited pages are counted by URL, not by canonical page

Three of the seven entries pointed at the same Ollama documentation page under different URLs (http versus https, and a .md suffix). Normalize scheme and suffix before counting distinct sources, or display the report’s inline citation numbers, which already do that work for the reader.

Claim lists arrive empty in fast mode

Each cited page carries a claims array, and in this mode the array is []. The report’s inline [n] markers are the claim-to-source link: they map in order to cited_pages. The example keeps the claims field in sources.json because it is part of the SDK contract and deeper modes can populate it.

Optional metadata stays optional

Title was present on every source. Relevance, reliability, and summary were not; those come from the analysis phases that fast mode skips. Read them defensively and your code works unchanged across modes.

Source selection is broad by design

Two of the seven sources were tutorials about building your own page-reading tool, and the report used them to describe a fourth implementation path. Fast mode returns what the search surfaced; whether that path belongs in the answer is a judgment your application or reviewer makes. We have not scored this answer, so the article presents it as returned output.

Failure handling

The repository gives configuration, HTTP, and task failures different exit codes:

Path	Observed or tested behavior
TABSTACK_API_KEY missing	Exit 5, message on stderr, no request made
Invalid key	HTTP 401 before the stream opened, exit 3
Streamed error event	Synthetic fixture produced exit 2 and printed the returned message and phase
Stream closed without complete	Synthetic fixture produced exit 2

Only the invalid-key case sent a failing request to production. The streamed failure cases use synthetic fixtures rather than trying to break production.

The full implementation also sanitizes its public event log. It retains event names, timestamps, iteration counters, short status messages, and completion counts. It excludes API keys, authorization headers, cookies, full source text, stack traces, environment dumps, and model reasoning.

Timeouts, retries, and cleanup

The SDK applies a 600-second timeout to /research streams. The example adds no shorter total timeout, so it does not turn a longer healthy run into an application timeout.

The SDK retries certain transport failures twice by default, including 408, 409, 429, and server errors. The example adds zero application-level retries. Repeating a request can repeat work and billing, so retry policy should be an explicit product decision.

The Tabstack client runs inside a context manager. The response closes when the block exits, including when the stream returns an error.

A clean uv sync –frozen passed 17 tests. Ruff and Pyright also passed with no reported errors. The quality gate ran on Python 3.12.13, not every declared Python version.

When this approach is sufficient

Use one managed Research call when your application starts with a question and needs a cited answer from live sources without operating the search, retrieval, and synthesis loop itself.

Use a search interface instead when links and excerpts are the desired output, or when your own model already operates the remaining loop in a way that meets your requirements.

Build the loop in your application when source selection, retrieval policy, model behavior, or citation assembly is core to your product and you need control over each component.

Use known-page extraction when you already have the URL and need page text or fields in a defined shape. Do not use a research workflow for work that a simpler output contract already solves.

Data flow

A Research call sends your question and the pages it retrieves through Tabstack’s pipeline, including a contracted model provider. Nothing you send is used to train a model. Request payloads are kept for 90 days while detailed data collection is on, which is the default for new organizations, and an organization admin can turn it off. The Data Handling page spells out what each endpoint processes and where.

Never trained on. Private by default.

Run the Python example

The repository contains the lockfile, implementation, fixtures, failure evidence, sample output, and evaluation harness:

Run the cited Research example on GitHub

Then continue with:

Search gives your model sources. What turns them into a cited answer?

Mon, 14 Sep 2026 14:00:00 GMT

A web search API can return useful titles, URLs, and excerpts. That is enough when your application needs search results or your model can reliably do the remaining work. A cited answer requires a research loop after search: define the question, select and read sources, identify gaps, reconcile evidence, write the answer, and preserve the claim-to-source connection.

Who this is for: People running their own model inside a product, assistant, agent, or internal workflow that needs current answers from the public web.

In this guide, you will learn:

what search returns and what it leaves to your system;
the seven stages between a question and a cited answer;
the difference between search, fetch, extraction, research, and automation;
when search alone is sufficient;
what a research-ready output should contain.

Search is access. A cited answer is a pipeline.

Search and research solve different parts of the same job. A search endpoint usually begins with a query and returns candidates: titles, URLs, snippets, or excerpts. Those results help a person or program decide what to read next. For example, the current Ollama Web Search API documents a response containing title, url, and a relevant content snippet for each result. Brave’s Web Search API similarly returns result objects with a title, URL, description, and optional extra snippets.

That output may be exactly right. A link finder, source browser, retrieval component, or model with a dependable tool loop can use it directly. But if the application needs an answer it can show, store, or act on, somebody still owns the work after search:

decide what the question actually requires;
choose which results are worth reading;
retrieve the relevant pages;
extract the useful evidence;
check whether the evidence is sufficient;
resolve conflicts and qualify uncertainty;
write the answer and attach the sources.

A cited answer is therefore not a formatting option on a list of search results. It is the output of a research system.

The seven-stage research loop

View the diagram in the original post.

The arrows matter more than the boxes. Research is not always one pass from left to right. If the evidence does not answer part of the question, the system needs another query, another source, or a narrower claim. That loop is what separates research from “summarize the first result.”

1. Define the question and success criteria

The system first turns the user’s request into an answerable research objective. Consider this question:

What are the main approaches to browser automation for AI agents, and how do they differ?

A useful answer needs more than definitions. It should establish the comparison dimensions, such as browser control method, deployment model, supported interaction, and who owns the infrastructure. It may also need to distinguish open-source libraries, hosted browser infrastructure, and higher-level task APIs. Without a clear objective, a system can collect relevant pages and still produce an incomplete answer.

2. Discover candidate sources

Search finds pages that may contain evidence. One broad query is rarely enough for a multi-part question, so the system may create narrower searches for each comparison dimension. Discovery should preserve where each candidate came from and why it may be relevant. At this stage, a search result is a lead, not yet evidence for a final claim.

3. Retrieve and read the pages

Snippets are useful for triage, but they are not a substitute for the source page. They can omit qualifications, dates, tables, footnotes, or the paragraph that changes the meaning of a claim. The system therefore fetches the selected pages and converts them into content it can inspect. Some pages can be read through a direct HTTP request. Others require rendering before the meaningful content is available. Failed, blocked, truncated, or outdated pages need to be recorded rather than silently treated as supporting evidence.

4. Extract claims with their context

Reading a page is not the same as identifying the evidence that answers the question. The system must isolate the relevant statement, retain its source URL, and preserve enough context to avoid turning a qualified claim into an absolute one. For a comparison, this may mean extracting:

the specific capability being documented;
the conditions or plan on which it is available;
the publication or update date;
the exact source page;
the claim the source supports.

The goal is a claim-to-source record, not a pile of page text.

5. Evaluate coverage, source quality, and gaps

The system checks the collected evidence against the original question. Which parts are answered? Which are partial? Which have no useful source yet?

This stage should also ask whether the sources are appropriate for the claim. Official documentation may be the best source for current API behavior. A maintainer issue may be better evidence for a known failure mode. A third-party comparison can provide a useful lead, but it should not quietly become proof of a competitor’s current limits.

If a material gap remains, the loop returns to discovery with a more specific query.

6. Reconcile conflicts and qualify uncertainty

Sources disagree for ordinary reasons: they cover different dates, product tiers, regions, definitions, or versions. The system should not choose whichever sentence is easiest to quote. Reconciliation means comparing the scope of each claim, preferring the most direct and current evidence, and stating uncertainty when the sources do not support a single conclusion. When the evidence remains incomplete, the answer should say so.

7. Synthesize the answer and preserve citations

Only after the evidence is collected and evaluated should the system write the final answer. The result should separate what the sources establish from the system’s interpretation. Citations should let a reader inspect the evidence behind a material claim. A bibliography at the bottom is useful, but it is weaker than a record that connects individual claims to the pages used to support them.

What goes in and what should come out?

Here is a complete input for a research system:

{
  "query": "What are the main approaches to browser automation for AI agents, and how do they differ?",
  "mode": "fast"
}

A useful research output is not just prose. It should contain the answer and enough source metadata to inspect how the answer was built. Tabstack’s current /research response represents that contract with a report plus citation metadata:

{
  "report": "# Browser automation for AI agents\n\nThe main approaches differ in who controls the browser, where it runs, and how much orchestration the application owns. [...]",
  "metadata": {
    "citedPages": [
      {
        "id": "pg_a1b2c3",
        "url": "https://source.example/official-documentation",
        "title": "Official documentation",
        "claims": [
          "A specific statement used in the report."
        ],
        "sourceQueries": [
          "browser automation approaches for AI agents"
        ]
      }
    ],
    "totalPagesAnalyzed": 12
  },
  "message": "Research complete",
  "timestamp": 1789420248496
}

Example qualification: This response is illustrative and combines the documented /research response fields with placeholder content. It is not the output of a live research run, and the page count is not a benchmark or product claim.

The output contract matters because different consumers need different artifacts.

Output taxonomy: choose the artifact your application needs

Output	What the application receives	What still needs to happen	Good fit
Search results	Ranked titles, URLs, and excerpts	Select, read, compare, synthesize, and cite	Discovery, link finding, model-operated research
Clean page text	Readable content from one known URL	Interpret the page and combine it with other sources	RAG ingestion, reading a page, downstream analysis
Matching JSON	Fields returned in a defined schema	Validate business rules and decide how to use the record	Known-page extraction, enrichment, pipelines
Cited answer	Synthesized answer plus source references	Review evidence and apply domain judgment	Current questions that span multiple sources
Completed public web task	Progress and the result of browser interaction	Confirm the result and handle downstream side effects	Navigation, clicks, forms, and multi-step public workflows

These outputs are not quality levels. They are different contracts. Returning a cited report when an application needs ten links adds unnecessary work. Returning ten links when the application needs an answer transfers the research loop into the model or application.

What is the difference between search, fetch, extraction, research, and automation?

What is web search?

Web search returns candidate sources relevant to a query. A typical result includes a title, URL, and excerpt. Search is sufficient when the list of sources is the desired output or when another component reliably owns the rest of the research loop.

What is fetch?

Fetch retrieves the content at a known URL. It answers “read this page,” not “answer this open question.” Fetching may be direct or may require rendering, but it does not by itself establish which other pages are needed or reconcile evidence across them.

What is extraction?

Extraction turns a known page into a cleaner or more structured representation. That may be readable Markdown or JSON matching a supplied schema. Extraction is the right abstraction when the source is known and the application knows what fields or content it needs.

What is research?

Research begins with a question and produces a synthesized answer from one or more sources. It includes source discovery, reading, gap detection, reconciliation, and citation. A research API is useful when the application needs the finished answer rather than material for its model to process.

What is automation?

Automation completes a task that requires interaction with a website. It applies when the system must navigate, click, fill forms, or move through several steps before the requested result exists. Reading or researching a public page does not automatically require browser automation.

When is search alone enough?

Search alone is often the simpler and better choice when:

the user wants links, not an answer;
your model already performs the tool loop reliably;
the question is simple and one source is enough;
you need full control over ranking, retrieval, or synthesis;
the research loop is part of your product’s core differentiation;
you are building a source-discovery interface rather than an answer feature.

A managed research loop is more useful when:

the application must return a current, cited answer;
the question spans several sources or sub-questions;
incomplete coverage needs to trigger another search;
citations must remain connected to specific claims;
you do not want the model to spend its context and tool calls operating the loop;
your team would otherwise maintain search, retrieval, synthesis, and citation orchestration.

Neither path removes the need to evaluate the result. A citation makes a claim inspectable. It does not guarantee that the source is authoritative, that every claim is supported, or that the answer is complete.

The architecture choice is about ownership

Approach	Application receives	Who owns source selection and reading?	Who owns gap detection and reconciliation?	Who owns citation assembly?
Search API	Results and excerpts	Your model or application	Your model or application	Your model or application
Search + fetch tools	Results plus page content	Your model or application	Your model or application	Your model or application
DIY research loop	Whatever contract you build	Your team’s orchestration	Your team’s orchestration	Your team’s orchestration
Managed Research API	Synthesized answer and source metadata	The API inside the call	The API inside the call	The API inside the call

A managed API can reduce the orchestration in your codebase, but it does not transfer responsibility for the result. Your team still owns question design, acceptance criteria, domain-specific review, and how the application uses the answer.

The decision is not “build versus buy” in the abstract. It is where this loop should live, how much control you need, and which output your application can consume safely.

A concrete managed Research call

Tabstack is one implementation of the managed path. The application sends a question to /research; the endpoint streams progress with Server-Sent Events, then emits a complete event containing the report and metadata for cited pages.

import Tabstack from "@tabstack/sdk";

const client = new Tabstack();

const stream = await client.agent.research({
  query: "What are the main approaches to browser automation for AI agents, and how do they differ?",
  mode: "fast",
});

for await (const event of stream) {
  if (event.event === "error") {
    throw new Error(event.data.error.message);
  }

  if (event.event === "complete") {
    console.log(event.data.report);

    for (const source of event.data.metadata.citedPages ?? []) {
      console.log(`${source.title ?? "Untitled source"}: ${source.url}`);
    }
  }
}

Environment: Node.js 20 or later; @tabstack/sdk 2.8.6; TABSTACK_API_KEY set in the environment.

Last technically validated: September 14, 2026, against the current TypeScript SDK source and published Research guide.

Validation boundary: The snippet’s types and API surface were checked against SDK commit 366182e084a7d5fd372cde47e8bc2e96e63f5e16. A live fast-mode request was run on September 14, 2026 against the production API; the complete event matched the shape shown above, including the cited page fields.

Failure modes a cited-answer system must expose

A relevant result is not necessarily a usable source

A search result may point to an outdated, derivative, blocked, or incomplete page. Record source failures and continue rather than treating every candidate as evidence.

Snippets can remove the qualification that matters

A snippet may mention a price, limit, feature, or policy without its date or plan. Read the source before using the statement in an answer.

More sources do not automatically improve the answer

Ten pages repeating the same secondary claim do not provide ten independent confirmations. Source authority, independence, recency, and directness matter.

Citations can be present but poorly matched

A link beside a paragraph does not prove that it supports every sentence in the paragraph. Preserve the specific claims associated with each source and test citation correctness separately from citation presence.

The evidence can remain incomplete

Some questions have no public answer. Others have conflicting or rapidly changing evidence. The system needs an explicit partial or unanswered state rather than filling gaps with plausible prose.

The stream can fail before completion

For Tabstack /research, HTTP-level failures can occur before the stream opens, while task-level failures arrive as error events inside the stream. A consumer that only listens for completeness can mistake a failed run for an empty answer.

Trust and data flow

A managed research call sends the question and relevant processing data outside your application. Treat that as an architecture decision, not a footnote. Review which providers process inputs and outputs, what telemetry is retained, whether history can be stored, and how personal or confidential information should be handled.

For Tabstack, the approved summary is: Never trained on. Private by default. Built by Mozilla.

The current Tabstack Privacy Notice explains that Mozilla operates the service, that inputs and outputs may be processed through third-party models for applicable services, and that users should be careful about placing confidential or sensitive information in inputs. Review the notice for the full data flow rather than interpreting the summary as a promise that nothing is processed or retained.

What should you evaluate before choosing a research path?

Use one real question from your product, not a generic demo. Then inspect:

Answer correctness: Are the material claims accurate?
Citation correctness: Does each cited page support the associated claim?
Citation completeness: Are important factual claims cited?
Source quality: Are the sources direct, current, and appropriate?
Coverage: Did the answer address every part of the question?
Failure behavior: Does the system expose missing evidence, blocked pages, and task errors?
Operational fit: Can your application handle streaming, retries, latency, and cost for the job?

This is the practical difference between adding search and shipping a current-answer feature. The first gives your system access to sources. The second requires a research loop with an output contract, evidence handling, and failure states.

Evaluate a real Research question

Choose one current question your model or application must answer today. Run it through your existing search-and-fetch path and through a managed Research call. Compare the outputs at the claim level, including missing claims and failed sources, rather than choosing the answer that sounds better.

Make a Tabstack Research call or review the Research guide before you design the evaluation.

Give a LlamaIndex.TS agent live web access

Thu, 06 Aug 2026 05:00:00 GMT

LlamaIndex is built around grounding answers in real data. Usually that data is yours: documents you have indexed, embedded, and made searchable.

The awkward half is everything that is not yours and not static. Competitor pricing. A vendor’s changelog. What was published this morning. You cannot index your way to that, because by the time you have, it has moved.

This tutorial adds the live half. Five tools that read the web on demand, sitting alongside whatever you have already indexed. My honest view is that most RAG systems need less of this than they think and more of it than they have.

What you need

A Node project (the package ships dual ESM and CJS builds)
llamaindex 0.12.x (the peer dependency is ^0.12.0, so it is pinned to the 0.12 line)
A Tabstack API key from console.tabstack.ai

npm install @tabstack/llamaindex llamaindex zod @llamaindex/workflow @llamaindex/openai

export TABSTACK_API_KEY="your-key-here"

llamaindex and zod are peer dependencies, so your app’s single instance of each is used. Zod 3 and Zod 4 both work here, since LlamaIndex’s tool() is Zod-native and accepts either. Nothing to think about on upgrade.

@llamaindex/workflow and @llamaindex/openai are LlamaIndex’s own packages rather than anything to do with Tabstack. The agent() helper lives in the first and the model provider in the second, and neither is re-exported from the llamaindex root. Workflow does arrive as a transitive dependency under npm’s hoisted layout, so you can get away with omitting it there, but declare it anyway or the snippet breaks the moment you move to pnpm or Yarn PnP. Swap @llamaindex/openai for whichever provider you actually use.

Step 1: an agent that reads the web

import { tabstackTools } from "@tabstack/llamaindex";
import { agent } from "@llamaindex/workflow";
import { openai } from "@llamaindex/openai";
import { Settings } from "llamaindex";

Settings.llm = openai({ model: "gpt-4o" });

const researcher = agent({ tools: tabstackTools });

const result = await researcher.run(
  "What are Vercel's current pricing plans? Cite your sources.",
);

console.log(result.data);

tabstackTools is a pre-built array, which is the shape agent() expects. The tools resolve TABSTACK_API_KEY lazily on first call, so importing the package never requires a key to be set.

That is a working agent with live web access in about ten lines, and there is nothing running on your side. No Playwright, no browser binary, no headless Chrome to keep patched.

Step 2: the five tools

Export	Tool name	What it does
`extractStructuredDataTool`	`extract_structured_data`	Pull specific fields from a URL into a JSON shape you define
`extractPageContentTool`	`extract_page_content`	Fetch a page as clean markdown
`researchQuestionTool`	`research_question`	Synthesised answer with cited sources across multiple pages
`generateStructuredDataTool`	`generate_structured_data`	Fetch a page, then AI-transform it into derived or reshaped JSON
`automateBrowserTaskTool`	`automate_browser_task`	Run a multi-step, natural language browser task

Import individually for a subset, or use the array for all of them:

import { researchQuestionTool, extractPageContentTool } from "@tabstack/llamaindex";
import { agent } from "@llamaindex/workflow";

const researcher = agent({ tools: [researchQuestionTool, extractPageContentTool] });

Worth doing. Every tool you pass is another option the model weighs on every step, and another way for it to pick wrong. An agent that reads and researches does not need browser automation in its toolbox.

Two of these are especially worth knowing well.

extract_page_content returns clean markdown. That is the format you want when the output is heading somewhere else, whether that is a summarisation step, a prompt, or an ingestion pipeline. It is not HTML you then have to strip.

research_question returns a synthesised answer with the sources behind it. This is the one that is genuinely hard to build yourself, and the one to reach for when the answer is not on any single page you can name.

Step 3: extract or generate

The inputs, which the model fills in:

extract_structured_data: url, json_schema_json, a JSON encoded JSON Schema string
extract_page_content: url
research_question: query
generate_structured_data: url, instructions, json_schema_json
automate_browser_task: task, plus optional url, guardrails, data, country, max_iterations, max_validation_attempts

The distinction people get wrong is between the two structured tools. Extract pulls out what is already written on the page. Generate fetches the page and produces something derived from it, like a category, a sentiment, or a one-line summary nobody wrote down.

If the field exists in the HTML, extract it. If you are asking for judgement, generate it. Reaching for generate when extract would do costs you time and adds a layer where the answer can drift away from what the page actually said.

Inputs are validated against the core Zod schema before the SDK call, since LlamaIndex passes the schema straight through as the tool’s parameters. Malformed model output is caught before a request goes out rather than after the API rejects it.

Step 4: the flags that matter in production

The model can pass these itself, and they are sent to Tabstack only when present, so leaving them out keeps the defaults.

On extract_structured_data, extract_page_content, and generate_structured_data:

effort: "min", "standard", or "max". Use "max" for JavaScript-heavy pages, where it does full server-side browser rendering.
nocache: bypass the cache.
country: an ISO 3166-1 alpha-2 code for geotargeted fetches.

On research_question: mode as "fast" or "balanced", plus nocache.

If a page comes back thin or empty, it is nearly always a client-rendered app and effort: "max" is the fix. That single argument is the whole workaround, and it costs you a keyword rather than a browser dependency.

nocache is the one to remember for this audience specifically. If you are pulling live data precisely because it changes, a cached read defeats the purpose. Set it on anything you are monitoring.

Step 5: failures

When a tool call fails it throws TabstackToolError, carrying a normalised message plus an HTTP status for API errors, and LlamaIndex surfaces it in the tool result. The agent sees a readable failure and can retry or explain rather than the run falling over.

One catch covers transport problems, parsing problems, and API problems. status carries the HTTP status for API failures and is undefined for everything else, which is usually enough to decide between retrying and giving up.

One behaviour to know before shipping: automate_browser_task runs non-interactively. It never pauses for human-in-the-loop form input, so it cannot block an agent waiting on a person. It returns the final answer plus the data it extracted and the pages it visited. If you genuinely need a human in the middle of a browser session, that is the interactive mode flow in the API rather than this tool.

Step 6: custom keys and clients

For a key per tenant, a custom base URL, or reuse of an SDK client you already have:

import { createTabstackLlamaindexTools } from "@tabstack/llamaindex";

const tools = createTabstackLlamaindexTools({ apiKey: process.env.MY_KEY });

Same array shape, same tool names, so it drops in exactly like the default export.

Where to take it next

The obvious move is combining these tools with your own. An agent holding both a query engine over your documents and research_question over the live web can answer questions neither source covers alone, which is usually the question someone actually wanted to ask.

extract_page_content is also worth a look as an ingestion step rather than an agent tool. Clean markdown is a far better starting point for chunking than raw HTML, and you get it without running a browser or maintaining a parser.

The tool names are identical in @tabstack/langchain, @tabstack/ai, @tabstack/eve, and the Python langchain-tabstack, so prompts carry over if this workload moves.

The package is on npm as @tabstack/llamaindex, and the full API reference is at docs.tabstack.ai.

Ready to build? Sign up for 10,000 free credits, no card required, and query your first live page alongside your index. Full API in the docs.

Add web tools to an OpenAI Agents SDK agent with Tabstack

Wed, 05 Aug 2026 05:00:00 GMT

The OpenAI Agents SDK has a clean model. You build an Agent, you give it tools, you call run. This tutorial gives it the tools nearly every agent eventually needs: read a page, pull typed fields from it, research across sources, and drive a browser when the data sits behind an interaction.

It also spends a section on strict mode, because that is the one place where this adapter had to make a real decision. Why are these tools not strict, when the SDK pushes you so hard towards it? The answer is more interesting than it sounds, and knowing it will save you from filing a bug that is not a bug.

What you need

A Node project (the package ships dual ESM and CJS builds)
@openai/agents v0.13 or later
Zod 4
A Tabstack API key from console.tabstack.ai

npm install @tabstack/openai-agents @openai/agents zod

export TABSTACK_API_KEY="your-key-here"

Zod 4 is required, not preferred. The Agents SDK depends on it, and the adapter uses Zod 4’s native z.toJSONSchema to advertise each tool’s parameters. If you are still on Zod 3, that upgrade comes first.

Step 1: an agent with web access

import { Agent, run } from "@openai/agents";
import { tabstackTools } from "@tabstack/openai-agents";

const agent = new Agent({
  name: "Research assistant",
  instructions:
    "You are a research assistant with web intelligence tools. Use research_question for open " +
    "questions that need multiple sources, extract_page_content to read a specific URL as " +
    "markdown, and the extract tools to pull structured fields from a page. Always cite sources.",
  tools: tabstackTools,
});

const result = await run(agent, "What are Vercel's pricing plans, with sources?");
console.log(result.finalOutput);

tabstackTools is an array of ready-to-use tools, which is exactly what Agent wants for tools. No mapping step, no wrapper.

The tools resolve TABSTACK_API_KEY lazily on first call, so importing the package never requires a key to be set. Your test suite and your build steps stay clean.

Step 2: the five tools

Tool name	Export	What it does
`extract_structured_data`	`extractStructuredDataTool`	Pull specific fields from a URL into a JSON shape you define
`extract_page_content`	`extractPageContentTool`	Fetch a page as clean markdown
`research_question`	`researchQuestionTool`	Synthesised answer with cited sources across multiple pages
`generate_structured_data`	`generateStructuredDataTool`	Fetch a page, then AI-transform it into derived or reshaped JSON
`automate_browser_task`	`automateBrowserTaskTool`	Run a multi-step, natural language browser task

Import them individually when you want a subset. I think building your own array is worth the thirty seconds it costs you. Every tool you pass is another option the model weighs on every step, and another way for it to choose wrong. An agent that only reads and researches should be handed two tools, not five.

The inputs, which the model fills in:

extract_structured_data: url, json_schema_json, a JSON encoded JSON Schema string
extract_page_content: url
research_question: query
generate_structured_data: url, instructions, json_schema_json
automate_browser_task: task, plus optional url, guardrails, data, country, max_iterations, max_validation_attempts

Extract versus generate is the distinction that trips people up. Extract pulls out what is already written on the page. Generate fetches the page and produces something derived from it, like a category or a summary nobody wrote down. If the field exists in the HTML, extract it. If you are asking for judgement, generate it.

Step 3: why these tools are not strict

Open the tool definitions and you will find they are registered with strict: false. That is deliberate, and it is worth understanding rather than treating as a wart.

The Agents SDK forces strict JSON Schema mode whenever a tool’s parameters is a Zod schema. Passing strict: false alongside a Zod schema throws, so it is not a knob you can simply turn. And strict mode cannot represent two constructs the shared core schemas rely on:

Optional fields. Strict mode requires every property to appear in required, which rules out .optional(). All those production flags in the next section are optional by design, because sending them when you do not mean them would override Tabstack’s defaults.

Open objects. automate_browser_task accepts a data object for form-filling context, whose keys you cannot know ahead of time. Strict mode forbids the schema-valued additionalProperties that an open record needs.

So the adapter converts each core Zod schema to a JSON Schema and registers the tools with strict: false. Two things you do not lose in the trade. The model still sees full field descriptions, so tool selection and argument quality are unaffected. And every call is validated against the core Zod schema inside execute before the request goes out, so malformed model output still fails fast with a clear error rather than reaching the API.

The shared core schemas are never modified to accommodate this. That is what keeps behaviour identical across every framework and both languages.

Step 4: the flags that matter in production

The model can pass these itself, and they are only sent to Tabstack when present, so omitting them keeps the defaults.

On extract_structured_data, extract_page_content, and generate_structured_data:

effort: "min", "standard", or "max". Use "max" for JavaScript-heavy pages, where it does full server-side browser rendering.
nocache: bypass the cache.
country: an ISO 3166-1 alpha-2 code for geotargeted fetches.

On research_question: mode as "fast" or "balanced", plus nocache.

If a page comes back thin or empty, it is almost always a client-rendered app and effort: "max" is the fix. That one argument replaces installing Playwright, shipping a browser binary, and keeping both patched. Nothing runs on your side.

The tidiest way to apply these is through instructions rather than code:

If a page returns little or no content, retry once with effort set to max. Use fast mode for research while iterating.

Step 5: failures

When a tool call fails it throws TabstackToolError, carrying a normalised message plus an HTTP status for API errors. The Agents SDK catches tool errors and surfaces them to the model as a tool result, so a failing call does not abort the run by default. The model sees the failure, and can retry with different arguments or explain the problem.

That is usually what you want. A dead link in step two of a five-step research task should not kill the run.

One behaviour to know before shipping: automate_browser_task runs non-interactively. It never pauses for human-in-the-loop form input, so it cannot block waiting on a person. It returns the final answer plus the data it extracted and the pages it visited. If you need a human in the middle of a browser session, that is the interactive mode flow in the API rather than this tool.

Step 6: custom keys and clients

For a key per tenant, a custom base URL, or reuse of an SDK client you already have in scope:

import { createTabstackOpenAIAgentsTools } from "@tabstack/openai-agents";

const tools = createTabstackOpenAIAgentsTools({ apiKey: process.env.MY_KEY });

Same array shape, same tool names, so it drops into tools exactly like the default export.

Where to take it next

Narrow the toolset per agent. If you are using handoffs, a specialist agent should carry only the tools its speciality needs. A pricing researcher wants extract_structured_data. A general assistant wants research_question. Very few agents want all five.

Show the citations. research_question returns sources alongside its answer, and surfacing them is what separates an assistant people trust from one they quietly verify.

The tool names are identical in @tabstack/langchain, @tabstack/ai, @tabstack/eve, and the Python langchain-tabstack, so your instructions carry over if this workload moves.

The package is on npm as @tabstack/openai-agents, and the full API reference is at docs.tabstack.ai.

Ready to build? Sign up for 10,000 free credits, no card required, and hand your agent its first typed extraction. Full API in the docs.

Give a Claude Agent SDK app web access

Tue, 04 Aug 2026 05:00:00 GMT

Every other integration in this series hands your framework an array or an object of tools. This one is different, because the Claude Agent SDK expects tools to arrive as an MCP server.

That sounds like more work, and I assumed it would be. It is actually less, because the package ships a server already built. You import it, name it in your query() options, and Claude has web access. There is no separate process to run and no MCP config file to maintain.

What you need

A Node project (the package ships dual ESM and CJS builds)
A Tabstack API key from console.tabstack.ai
An Anthropic API key

npm install @tabstack/claude-agent @anthropic-ai/claude-agent-sdk zod

export TABSTACK_API_KEY="your-tabstack-key"
export ANTHROPIC_API_KEY="your-anthropic-key"

One version constraint to note. The Claude Agent SDK depends on Zod 4, so this package’s zod peer dependency is ^4.0.0. Unlike the eve and LlamaIndex adapters, Zod 3 is not an option here. If your app is still on Zod 3, that upgrade is a prerequisite rather than a nice-to-have.

Step 1: wire in the server

import { query } from "@anthropic-ai/claude-agent-sdk";
import { tabstackServer, tabstackAllowedTools, tabstackMcpServerName } from "@tabstack/claude-agent";

for await (const message of query({
  prompt: "What are Vercel's current pricing plans, with sources?",
  options: {
    mcpServers: { [tabstackMcpServerName]: tabstackServer },
    allowedTools: tabstackAllowedTools,
  },
})) {
  if (message.type === "result" && message.subtype === "success") {
    console.log(message.result);
  }
}

Three imports, and Claude can read the web.

Note the for await. query() returns a lazy async generator rather than a promise, so nothing runs until you iterate it. Writing await query({ ... }) typechecks fine and does absolutely nothing, which is a quiet half hour to lose.

tabstackServer is a ready-built in-process MCP server holding all five tools. In-process is the important word. It runs inside your Node process, so there is no subprocess to spawn, no port to bind, and no server lifecycle to manage. If your only previous experience of MCP was editing a JSON config and restarting something, this is a pleasant surprise.

tabstackAllowedTools pre-approves every Tabstack tool. The Claude Agent SDK gates tool use by permission, so without this the tools exist but Claude cannot call them. It is the line people forget, and the symptom is an agent that talks about wanting to look something up and then does not.

tabstackMcpServerName is the string "tabstack". Use the export rather than typing the literal, because the name becomes part of how Claude addresses each tool.

The server resolves TABSTACK_API_KEY lazily on first tool call, so importing the package never requires a key to be set.

Step 2: understand what Claude actually sees

Claude does not see research_question. It sees mcp__tabstack__research_question.

MCP tools are namespaced by their server, so every tool arrives fully qualified as mcp__tabstack__<tool_name>. This matters in two places. When you read a transcript and want to know which tool ran, that prefix is what you are scanning for. And when you write instructions referring to a tool by name, the unqualified name is what Claude maps onto, but the qualified name is what appears in permission prompts and logs.

Underneath the prefix, the names are the same everywhere:

Tool name	What it does
`extract_structured_data`	Pull specific fields from a URL into a JSON shape you define
`extract_page_content`	Fetch a page as clean markdown
`research_question`	Synthesised answer with cited sources across multiple pages
`generate_structured_data`	Fetch a page, then AI-transform it into derived or reshaped JSON
`automate_browser_task`	Run a multi-step, natural language browser task

Those names are shared with the LangChain, Vercel AI SDK, eve, Mastra, and Python packages, so a prompt written for one carries over.

The distinction worth internalising is extract versus generate. Extract pulls out what is already written on the page. Generate fetches the page and produces something derived from it, like a category or a summary nobody wrote. If the field exists in the HTML, extract it. If you are asking for judgement, generate it.

Step 3: the inputs the model fills in

You will read these in transcripts:

extract_structured_data: url, json_schema_json, a JSON encoded JSON Schema string
extract_page_content: url
research_question: query
generate_structured_data: url, instructions, json_schema_json
automate_browser_task: task, plus optional url, guardrails, data, country, max_iterations, max_validation_attempts

Each tool’s input schema is the core Zod schema’s raw shape, which means the Claude Agent SDK validates Claude’s arguments before the handler runs. Malformed tool calls fail immediately with a clear message rather than travelling to the API and coming back rejected.

There are optional inputs too, and each is accepted by a different subset of the tools:

effort, as "min", "standard", or "max", on the three fetch-based tools
nocache to bypass the cache, on the three fetch-based tools and research_question
country, an ISO 3166-1 alpha-2 code, for geotargeted fetches, on the three fetch-based tools and automate_browser_task
mode, as "fast" or "balanced", on research_question only

The first three are forwarded only when Claude supplies them. mode is the exception: it defaults to "balanced" and is always sent.

If a page comes back thin or empty, it is nearly always a client-rendered app and effort: "max" is the fix. Full rendering happens server side, so there is no Playwright in your project and no browser binary in your image. Put it in your system prompt and let Claude apply it:

If a page returns little or no content, retry once with effort set to max. Always cite the sources you used.

Step 4: custom keys, and assembling it yourself

For a key per tenant, a custom base URL, or reuse of an SDK client you already have:

import { query } from "@anthropic-ai/claude-agent-sdk";
import {
  createTabstackClaudeAgentServer,
  tabstackAllowedTools,
  tabstackMcpServerName,
} from "@tabstack/claude-agent";

const server = createTabstackClaudeAgentServer({ apiKey: process.env.MY_KEY });

for await (const message of query({
  prompt: "...",
  options: {
    mcpServers: { [tabstackMcpServerName]: server },
    allowedTools: tabstackAllowedTools,
  },
})) {
  if (message.type === "result" && message.subtype === "success") {
    console.log(message.result);
  }
}

If you would rather assemble the server yourself, perhaps because you want Tabstack’s tools alongside your own in a single server, createTabstackClaudeAgentTools(config) returns the raw tool array to pass into your own createSdkMcpServer({ name, version, tools }). That is also the route to take when you want a subset. Build the array, keep the tools you need, and hand the rest of your own tools in beside them.

Step 5: failures

This adapter handles errors differently from the others, and the difference is deliberate.

When a tool call fails, the handler normalises it to a TabstackToolError with a clean message, plus an HTTP status for API errors, then returns it as an MCP error result with isError: true. What Claude sees is the message; the status stays on the error object for your own logging rather than travelling into the transcript. Either way Claude reads a useful message rather than a raw exception, which means it can retry with different arguments or explain the problem to the user. A dead link does not end the run.

That is the right default for an agent loop. You are not catching these in your own code in the common case, because the model is the one that needs to react.

One behaviour to know before shipping: automate_browser_task runs non-interactively. It never pauses for human-in-the-loop form input, so it cannot block a run waiting on someone. It returns the final answer plus the data it extracted and the pages it visited. If you genuinely need a human in the middle of a browser session, that is the interactive mode flow in the API rather than this tool.

Where to take it next

Trim allowedTools. tabstackAllowedTools approves all five, which is the fastest way to get started and, I would argue, rarely what you want in production. Narrowing it to the two or three your agent actually needs reduces both cost and the chance of Claude choosing a tool you did not intend.

Combine it with your own tools. Because this is an in-process MCP server, adding Tabstack does not preclude anything else. Your own createSdkMcpServer output and tabstackServer sit side by side under different names in mcpServers.

The package is on npm as @tabstack/claude-agent, and the full API reference is at docs.tabstack.ai.

Ready to build? Sign up for 10,000 free credits, no card required, and hand your Claude agent its first live page. Full API in the docs.

Build a web-connected Mastra agent with Tabstack

Mon, 03 Aug 2026 05:00:00 GMT

Mastra gives you a clean Agent abstraction and expects you to bring the tools. This tutorial brings the ones nearly every agent ends up needing: read a page, pull typed fields off it, research a question across several sources, and drive a browser when the data is behind an interaction.

It is one import and one line in your agent definition. That is the entire integration, so most of what follows is about the decisions you make afterwards rather than the wiring.

What you need

A Node project (the package ships dual ESM and CJS builds)
Mastra v1 or later
A Tabstack API key from console.tabstack.ai

npm install @tabstack/mastra @mastra/core zod

export TABSTACK_API_KEY="your-key-here"

@mastra/core and zod are peer dependencies, so your app’s single instance of each is shared. That is the detail that keeps you out of the genuinely miserable bug where two copies of Zod disagree about whether your schema is a schema.

Step 1: a web-connected agent

import { Agent } from "@mastra/core/agent";
import { tabstackTools } from "@tabstack/mastra";

const agent = new Agent({
  id: "web-researcher",
  name: "Web Researcher",
  instructions: "Answer questions with current information, and always cite your sources.",
  model: "anthropic/claude-sonnet-4-6",
  tools: tabstackTools,
});

const result = await agent.generate("What are Vercel's pricing plans, with sources?");
console.log(result.text);

That is a working agent with web access. tabstackTools reads TABSTACK_API_KEY from the environment, and the client is created lazily, so importing the package never requires a key to be set.

tabstackTools is a named object keyed by tool name, which is exactly the shape Mastra’s tools expects. No adapter, no mapping step.

Notice what the instructions are doing. They describe the job, not the tools. Each tool ships with a description written for the model to read, so you do not need to explain in your prompt what research_question is for. Keep your instructions about the outcome you want and let the descriptions do the routing. It keeps prompts short, and it means description improvements reach you on upgrade rather than requiring a prompt rewrite.

Step 2: know your five

Tool	What it does
`extract_structured_data`	Pull specific fields from a URL into a JSON shape you define
`extract_page_content`	Fetch a page as clean markdown
`research_question`	Synthesised answer with cited sources across multiple pages
`generate_structured_data`	Fetch a page, then AI-transform it into derived or reshaped JSON
`automate_browser_task`	Run a multi-step, natural language browser task

The model fills in the arguments, but knowing the shapes helps when you are reading a trace and wondering why it chose badly:

extract_structured_data: url, json_schema_json, a JSON encoded JSON Schema string
extract_page_content: url
research_question: query
generate_structured_data: url, instructions, json_schema_json
automate_browser_task: task, plus optional url, guardrails, data, country, max_iterations, max_validation_attempts

The one distinction worth internalising is extract versus generate. Extract pulls out what is already written on the page. Generate fetches the page and produces something derived from it, like a category or a one-line summary nobody wrote. If the field exists in the HTML, extract it. If you are asking for judgement, generate it. Reaching for generate when extract would do costs you time and adds a layer where the answer can drift.

Step 3: give it fewer tools

Passing all five is the easy default, and often the wrong one. Every tool you expose is another option the model weighs on every step, and another way for it to pick wrong. On smaller models, trimming the toolset is one of the highest-leverage changes available to you.

Each tool is exported individually, and the package ships a toolNames map so you do not hand-write the string keys:

import { Agent } from "@mastra/core/agent";
import { extractPageContentTool, researchQuestionTool, toolNames } from "@tabstack/mastra";

const agent = new Agent({
  id: "web-researcher",
  name: "Web Researcher",
  instructions: "Summarize pages and research questions.",
  model: "anthropic/claude-sonnet-4-6",
  tools: {
    [toolNames.researchQuestion]: researchQuestionTool,
    [toolNames.extractPageContent]: extractPageContentTool,
  },
});

Use toolNames rather than typing "research_question" by hand. Those strings are the contract shared across every Tabstack integration, including the Python package, and a typo produces a tool your prompt references and your agent does not have.

Step 4: the flags that matter in production

The model can pass these itself, and they are only sent to Tabstack when present, so leaving them alone keeps the defaults.

On extract_structured_data, extract_page_content, and generate_structured_data:

effort: "min", "standard", or "max". Use "max" for JavaScript-heavy pages, where it does full server-side browser rendering.
nocache: bypass the cache.
country: an ISO 3166-1 alpha-2 code, such as "US", for geotargeted fetches.

On research_question: mode as "fast" or "balanced", plus nocache.

If a page comes back thin or empty, it is nearly always a client-rendered app and effort: "max" is the fix. That single argument replaces installing Playwright, shipping a browser binary in your container, and keeping both patched. There is nothing running on your side at all.

nocache matters when you are watching something that moves, like prices or availability. country matters when the page varies by region, which for pricing pages is most of the time.

The clean way to apply these is in your agent’s instructions rather than in code:

If a page returns little or no content, retry once with effort set to max. Use fast mode for research while iterating.

Step 5: failures

When a tool call fails it throws TabstackToolError, carrying a normalised message plus an HTTP status for API errors. Mastra surfaces it in the tool result, so the model sees a readable failure and can retry or explain rather than the run collapsing.

If you invoke a tool’s execute directly, catch it yourself:

import { TabstackToolError, tabstackTools, toolNames } from "@tabstack/mastra";

try {
  await tabstackTools[toolNames.extractPageContent].execute?.(
    { url: "https://example.com" },
    {} as never,
  );
} catch (err) {
  if (err instanceof TabstackToolError) {
    console.error(`Tabstack failed (${err.status ?? "no status"}): ${err.message}`);
  }
}

One error type instead of separate handling for transport, parsing, and API problems. status carries the HTTP status for API failures and is undefined for everything else, which is usually enough to decide between retrying and giving up.

One behaviour to know before you ship: automate_browser_task runs non-interactively. It never pauses for human-in-the-loop form input, so it cannot block your agent waiting on someone. It returns the final answer along with the data it extracted and the pages it visited. If you need a human in the middle of a browser session, that is the interactive mode flow in the API rather than this tool.

Step 6: custom keys and clients

For a key per tenant, a custom base URL, or reuse of an SDK client you already have in scope:

import { createTabstackMastraTools } from "@tabstack/mastra";

const tools = createTabstackMastraTools({ apiKey: process.env.MY_KEY });

Same object shape, same tool names, so it drops into tools exactly like the default export.

Where to take it next

If I were starting a Mastra agent from scratch today, I would wire in research_question and extract_page_content and nothing else, then add the structured tools when a concrete use case asks for them. Most agents never need all five.

Render the citations. research_question returns sources alongside the answer, and showing them is what separates an assistant people trust from one they quietly double-check.

If part of this workload moves elsewhere, the tool names and inputs are identical in @tabstack/langchain, @tabstack/ai, and the Python langchain-tabstack. Your instructions carry over, because the model sees the same tools.

The package is on npm as @tabstack/mastra, and the full API reference is at docs.tabstack.ai.

Ready to build? Sign up for 10,000 free credits, no card required, and hand your Mastra agent its first live page. Full API in the docs.

Give an eve agent web access with Tabstack

Fri, 31 Jul 2026 05:00:00 GMT

eve does something unusual with tools. It does not ask you to register them, assemble them into an array, or pass them into a constructor. It reads the filenames in agent/tools/. Add a file, the model can call it. Delete the file, it cannot.

I like this more than I expected to. It makes this the shortest integration tutorial in the series: giving your agent the ability to read and research the web is going to take five one-line files.

What you need

An existing eve project (the package ships dual ESM and CJS builds)
eve v0.24 or later
A Tabstack API key from console.tabstack.ai

npm install @tabstack/eve eve zod

export TABSTACK_API_KEY="your-key-here"

eve and zod are peer dependencies, so your app’s single instance of each is what gets used. Zod 3 and Zod 4 both work. On Zod 4 the adapter uses the built-in z.toJSONSchema, and on Zod 3 it falls back to zod-to-json-schema. You do not have to care which, but it is worth knowing nothing will break when you upgrade.

Step 1: add a tool

Create one file under agent/tools/, named after the tool, and re-export the Tabstack tool as its default export.

// agent/tools/research_question.ts
export { researchQuestionTool as default } from "@tabstack/eve";

That is the whole integration. Your agent can now answer open questions using multiple web sources, with citations.

The filename is load-bearing. eve derives tool identity from the path, so research_question.ts is what makes the tool research_question. It has to match exactly, and not only because eve requires it. That name is identical in the LangChain, Vercel AI SDK, Mastra, and Python packages, so a prompt written against one works against all of them. Rename the file to web_search.ts and you have quietly opted out of that.

Step 2: add the rest

Same pattern, one file each:

// agent/tools/extract_page_content.ts
export { extractPageContentTool as default } from "@tabstack/eve";

// agent/tools/extract_structured_data.ts
export { extractStructuredDataTool as default } from "@tabstack/eve";

// agent/tools/generate_structured_data.ts
export { generateStructuredDataTool as default } from "@tabstack/eve";

// agent/tools/automate_browser_task.ts
export { automateBrowserTaskTool as default } from "@tabstack/eve";

Here is what each one buys you:

File name	Export	What it does
`extract_structured_data.ts`	`extractStructuredDataTool`	Pull specific fields from a URL into a JSON shape you define
`extract_page_content.ts`	`extractPageContentTool`	Fetch a page as clean markdown
`research_question.ts`	`researchQuestionTool`	Synthesised answer with cited sources across multiple pages
`generate_structured_data.ts`	`generateStructuredDataTool`	Fetch a page, then AI-transform it into derived JSON
`automate_browser_task.ts`	`automateBrowserTaskTool`	Run a multi-step, natural language browser task

The tools resolve TABSTACK_API_KEY lazily on first call, so importing the package never requires a key to be set. Your build steps and tests stay happy without secrets.

Step 3: choose your toolset by choosing your files

This is where eve’s convention pays off. In every other framework, narrowing the toolset means editing the array or object you pass to the agent. Here you just do not create the file.

So ask yourself what your agent actually needs before you create all five. If it only reads and researches, create two files and stop. The model never sees automate_browser_task, so it can never decide to try it. Your toolset is visible in a directory listing, which is a more honest description of your agent’s capabilities than most codebases manage.

That restraint is worth exercising. Every tool you expose is another option the model weighs on every step, and another way for it to choose wrong. Smaller models feel this most.

One tradeoff worth naming. Unlike the other Tabstack adapters, this package has no aggregate tabstackTools export. Given the file convention there is nowhere sensible to put one, so I think it is the right call, but if you are moving code over from the LangChain or Vercel AI SDK packages it is the difference you will hit first.

Step 4: know what the model is filling in

You never write these arguments yourself, but you will read them in traces when something goes sideways:

extract_structured_data: url, json_schema_json, a JSON encoded JSON Schema string
extract_page_content: url
research_question: query
generate_structured_data: url, instructions, json_schema_json
automate_browser_task: task, plus optional url, guardrails, data, country, max_iterations, max_validation_attempts

The distinction people get wrong is extract versus generate. Extract pulls out what is already written on the page. Generate fetches the page and produces something derived from it, like a category or a summary nobody wrote down. If the field exists in the HTML, extract it. If you are asking for judgement, generate it.

There are also optional inputs the model can pass for finer control, sent to Tabstack only when present:

effort, as "min", "standard", or "max", on the three fetch-based tools. "max" does full server-side browser rendering.
nocache to bypass the cache
country, an ISO 3166-1 alpha-2 code, for geotargeted fetches
mode, as "fast" or "balanced", on research_question

If a page comes back thin or empty, it is almost always a client-rendered app and effort: "max" is the fix. That single argument is the entire replacement for running a browser. No Playwright, no binaries in your image, nothing to keep patched.

You can steer this from your agent’s instructions rather than in code:

If a page returns little or no content, retry once with effort set to max. Prefer research_question for open questions, and always cite the sources you used.

Step 5: use a custom key or client

The default tools read TABSTACK_API_KEY from the environment. When you need a key per tenant, a custom base URL, or you want to reuse an SDK client you already have, build the tools explicitly and re-export the one you want:

// agent/tools/research_question.ts
import { createTabstackEveTools } from "@tabstack/eve";

const tools = createTabstackEveTools({ apiKey: process.env.MY_KEY });

export default tools.research_question;

The file convention still applies. The filename decides the tool name, whatever you did inside.

Step 6: failures

When a tool call fails it throws TabstackToolError, carrying a normalised message plus an HTTP status for API errors, and eve surfaces it in the tool result. The model sees a readable failure rather than a stack trace, which means it can retry or explain rather than falling over.

Inputs are also validated against the core Zod schema before the SDK call runs. When a model produces malformed arguments, it fails fast with a clear message instead of sending a broken request and waiting for the API to reject it.

One behaviour to know up front: automate_browser_task runs non-interactively. It never pauses for human-in-the-loop form input, so it cannot block your agent waiting for someone to fill in a field. It returns the final answer along with the data it extracted and the pages it visited. If you genuinely need a human in the middle of a browser session, that is the interactive mode flow in the API rather than this tool.

Where to take it next

Start with two files. research_question.ts and extract_page_content.ts cover most of what a general assistant needs, and you can add the structured tools when a real use case asks for them.

When you add automate_browser_task, put a guardrails instruction in your agent’s system prompt. It is a plain English constraint on what a run may do, and on anything pointed at a live site it is the difference between a read-only browse and an agent cheerfully clicking a button you did not want clicked.

The package is on npm as @tabstack/eve, and the full API reference is at docs.tabstack.ai.

Ready to build? Sign up for 10,000 free credits, no card required, and hand your eve agent its first live page. Full API in the docs.

Stream a research assistant with the Vercel AI SDK and Tabstack

Thu, 30 Jul 2026 05:00:00 GMT

Web research is slow. Reading four pages and synthesising an answer takes seconds, sometimes tens of seconds, and there is no clever trick that makes it instant.

What you can do is stop making people stare at a spinner while it happens. This tutorial builds a research assistant on the Vercel AI SDK that streams its answer as it forms, using Tabstack for the web access. The whole thing is about thirty lines.

What you need

A Node project (the package ships dual ESM and CJS builds)
A Tabstack API key from console.tabstack.ai
An OpenAI key, or any other provider the AI SDK supports

npm install @tabstack/ai ai zod @ai-sdk/openai

export TABSTACK_API_KEY="your-key-here"
export OPENAI_API_KEY="your-openai-key"

ai and zod are peer dependencies, so your app keeps one instance of each. The package works with AI SDK v5 and later, and is v6 ready.

Step 1: give the model tools and get out of the way

The AI SDK expects tools to be an object keyed by tool name. That is exactly what tabstackTools is, so it passes straight through.

import { openai } from "@ai-sdk/openai";
import { generateText, stepCountIs } from "ai";
import { tabstackTools } from "@tabstack/ai";

const { text } = await generateText({
  model: openai("gpt-4o"),
  tools: tabstackTools,
  stopWhen: stepCountIs(5),
  prompt: "What are Vercel's pricing plans and how do they compare?",
});

console.log(text);

That is a working web-connected agent. tabstackTools reads TABSTACK_API_KEY from the environment, so there is no client to construct.

The one line people leave out is stopWhen. Without it the SDK makes a single call, the model asks for a tool, and you get back a response containing a tool call and no text. stepCountIs(5) tells it to keep going: call the tool, take the result, and carry on until it has an actual answer. Five is a reasonable ceiling for research work. Raise it if your agent legitimately needs to chain several lookups, lower it if you want a hard cost bound.

Step 2: stream it

generateText waits for everything. For anything user-facing, swap to streamText and start showing output immediately.

import { openai } from "@ai-sdk/openai";
import { streamText, stepCountIs } from "ai";
import { tabstackTools } from "@tabstack/ai";

const result = streamText({
  model: openai("gpt-4o"),
  tools: {
    research_question: tabstackTools.research_question,
    extract_page_content: tabstackTools.extract_page_content,
  },
  stopWhen: stepCountIs(5),
  prompt: "Summarize the latest on quantum error correction, with sources.",
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

Note what happened to tools. Because it is a plain object keyed by tool name, narrowing the toolset is just picking keys. Here the assistant only researches and reads, so it gets two tools instead of five.

That is worth doing deliberately. Every tool you pass is another option the model weighs on every step, and another way for it to pick wrong. On smaller and cheaper models, cutting the toolset to what the job needs is one of the highest-leverage changes you can make.

One honest note about streaming with tools, and it is the thing I would fix first in any demo. The text stream goes quiet while a tool runs, because the model is waiting on a network call it cannot narrate. Users read that silence as a hang. Surface tool activity in your UI, even if it is just “searching the web”, so the pause has an explanation.

Step 3: know your five tools

Tool	What it does
`extract_structured_data`	Pull specific fields from a URL into a JSON shape you define
`extract_page_content`	Fetch a page as clean markdown
`research_question`	A synthesised answer with cited sources across multiple pages
`generate_structured_data`	Fetch a page, then AI-transform it into derived or reshaped JSON
`automate_browser_task`	Run a multi-step, natural language browser task

The model fills in the inputs, but knowing the shapes helps when you are debugging why it chose badly:

extract_structured_data: url, json_schema_json, a JSON encoded JSON Schema string
extract_page_content: url
research_question: query
generate_structured_data: url, instructions, json_schema_json
automate_browser_task: task, plus optional url, guardrails, data, country, max_iterations, max_validation_attempts

The distinction that matters most is extract versus generate. Extract pulls out what is already written on the page. Generate fetches the page and produces something derived from it, like a category or a summary nobody wrote. If the field exists in the HTML, extract it. If you are asking for judgement, generate it.

Step 4: the flags that matter in production

The model can pass these itself, and they are only sent to Tabstack when present, so defaults stay intact when they are omitted.

On extract_structured_data, extract_page_content, and generate_structured_data:

effort: "min", "standard", or "max". Reach for "max" on JavaScript-heavy pages, where it does full server-side browser rendering.
nocache: bypass the cache when you need a fresh read.
country: an ISO 3166-1 alpha-2 code for geotargeted fetches.

On research_question: mode as "fast" or "balanced", plus nocache.

If a page comes back suspiciously empty, it is nearly always a client-rendered app and effort: "max" is the fix. That is the whole workaround. No Playwright, no browser binary in your container image, no keeping either patched.

You can steer these from the system prompt rather than in code, which is often the cleaner move:

const result = streamText({
  model: openai("gpt-4o"),
  tools: tabstackTools,
  stopWhen: stepCountIs(5),
  system:
    "You research questions using web tools. " +
    "If a page returns little or no content, retry once with effort set to max. " +
    "Prefer research_question for open questions and cite every source you use.",
  prompt: userQuestion,
});

Step 5: failures

When a tool call fails it throws TabstackToolError, carrying a readable message and, for API failures, an HTTP status. Inside an agent loop the AI SDK surfaces this in the tool result, so the model can see the failure and react. If you are calling a tool’s execute directly, catch it yourself:

import { TabstackToolError, tabstackTools } from "@tabstack/ai";

try {
  await tabstackTools.extract_page_content.execute(
    { url: "https://example.com" },
    { toolCallId: "1", messages: [] },
  );
} catch (err) {
  if (err instanceof TabstackToolError) {
    console.error(`Tabstack failed (${err.status ?? "no status"}): ${err.message}`);
  }
}

One error type instead of separate handling for transport, parsing, and API problems.

Step 6: configuration

The defaults read TABSTACK_API_KEY from the environment, which is what you want in most apps. When you need a key per tenant, a custom base URL, or you already have an SDK client in scope:

import { createTabstackAiTools } from "@tabstack/ai";

const tools = createTabstackAiTools({ apiKey: process.env.MY_KEY });

The client is created lazily, so importing the package never requires a key to be set. Your module imports stay safe in tests and in build steps that do not have secrets.

Where to take it next

Render the citations. research_question returns sources alongside the answer, and showing them is what separates an assistant people trust from one they double-check.

Narrow the toolset per route. A pricing page reader needs extract_structured_data and nothing else. A general assistant needs the lot. There is no rule that every surface in your app gets the same five.

If part of this workload moves to Python or another framework, the tool names and inputs are identical in @tabstack/langchain and langchain-tabstack. Your prompts carry over, because the model sees the same tools.

The package is on npm as @tabstack/ai, and the full API reference is at docs.tabstack.ai.

Ready to build? Sign up for 10,000 free credits, no card required, and stream your first cited answer. Full API in the docs.

Build a competitor brief agent with LangChain.js and Tabstack

Wed, 29 Jul 2026 05:00:00 GMT

The Python tutorial built a research agent. This one builds something with a shape you can hand to a colleague: a competitor brief. Point it at a company, get back their pricing, their positioning, and what people are saying about them, with sources.

Same five tools as the Python package, same names, same inputs. If you have read the Python version, the only thing that changes here is the syntax, which I still find slightly surprising every time I move between them.

What you need

A Node project (the package ships dual ESM and CJS builds)
A Tabstack API key from console.tabstack.ai
An OpenAI key, or any other model LangChain.js supports

npm install @tabstack/langchain @langchain/core zod langchain @langchain/openai

export TABSTACK_API_KEY="your-key-here"
export OPENAI_API_KEY="your-openai-key"

@langchain/core and zod are peer dependencies rather than dependencies. That is intentional. Your app keeps a single instance of each, which avoids the genuinely miserable class of bug where two copies of Zod disagree about whether your schema is a schema.

Step 1: pull structured data off a page

Start with the piece the whole brief hangs on. You know the pricing page URL, and you know what you want from it, so describe the shape and ask for it.

import { extractStructuredDataTool } from "@tabstack/langchain";

const pricing = await extractStructuredDataTool.invoke({
  url: "https://example.com/pricing",
  json_schema_json: JSON.stringify({
    type: "object",
    properties: {
      plans: {
        type: "array",
        items: {
          type: "object",
          properties: {
            name: { type: "string", description: "Plan name" },
            price: { type: "number", description: "Monthly price in USD" },
            features: {
              type: "array",
              items: { type: "string" },
              description: "Included features",
            },
          },
        },
      },
    },
  }),
});

for (const plan of pricing.plans) {
  console.log(`${plan.name}: $${plan.price}`);
}

Two details worth pausing on.

The schema goes in as json_schema_json, a JSON encoded string rather than an object. That looks fussy until you remember who else calls this tool. When an agent invokes it, the model has to produce that argument, and a string is a shape models get right far more often than nested objects.

If you write one thing carefully in this whole tutorial, make it the field descriptions. The description on each field is not decoration. It is the instruction that tells the extractor which number on the page you meant when you said price. A pricing page has annual figures, discounted figures, and per-seat figures on it. The description is how you disambiguate. Vague or missing descriptions are the most common reason extraction results disappoint.

Unlike the Python package, the TypeScript tools return plain objects. No JSON.parse step.

Step 2: research what is not on any single page

Pricing lives on a pricing page. Positioning and reputation do not live anywhere in particular, which is what research_question is for.

import { researchQuestionTool } from "@tabstack/langchain";

const { answer, sources } = await researchQuestionTool.invoke({
  query: "How do developers describe Example Corp's developer experience?",
});

console.log(answer);
for (const source of sources) {
  console.log(`- ${source.title}: ${source.url}`);
}

sources comes back as { title, url } objects, which drops straight into a citation list. This is the part that makes a brief defensible rather than just confident. When someone asks where a claim came from, you have the page.

While iterating, pass mode: "fast". Move to "balanced" when you care more about quality than latency.

Step 3: generate what the page never said

Extraction pulls out what exists. Sometimes you want something derived, like a category or a one-line summary that nobody wrote down. That is generate_structured_data: fetch the page, then transform it.

import { generateStructuredDataTool } from "@tabstack/langchain";

const positioning = await generateStructuredDataTool.invoke({
  url: "https://example.com",
  instructions:
    "Identify the primary audience this homepage is written for, " +
    "the main problem it claims to solve, and its strongest differentiator.",
  json_schema_json: JSON.stringify({
    type: "object",
    properties: {
      audience: { type: "string" },
      problem: { type: "string" },
      differentiator: { type: "string" },
    },
  }),
});

The line between the two tools is simple. If the field is written on the page, extract it. If you are asking for judgement, generate it. Reaching for generate when extract would do costs you time and adds a layer where the answer can drift.

Step 4: assemble the agent

Three tool calls, three pieces of a brief. You could orchestrate that yourself with three awaits, and for a fixed report you probably should. But the moment the shape of the question varies, let the model choose.

import { createAgent } from "langchain";
import { tabstackTools } from "@tabstack/langchain";

const agent = createAgent({
  model: "openai:gpt-4o",
  tools: tabstackTools,
  systemPrompt:
    "You produce competitor briefs. " +
    "Use extract_structured_data for facts on a known page such as pricing, " +
    "extract_page_content when you need to read a page in full, " +
    "generate_structured_data when you need a judgement the page does not state outright, " +
    "and research_question for anything spread across multiple sources. " +
    "Always cite the sources you used.",
});

const result = await agent.invoke({
  messages: [
    {
      role: "user",
      content:
        "Build me a brief on Example Corp. Their pricing is at https://example.com/pricing.",
    },
  ],
});

console.log(result.messages.at(-1)?.content);

tabstackTools is all five, ready to pass straight in:

Tool	When the model reaches for it
`extract_structured_data`	Specific fields from a URL you already know
`extract_page_content`	The whole page as clean markdown
`research_question`	An open question needing several sources
`generate_structured_data`	Fetch a page, then transform it into derived JSON
`automate_browser_task`	A multi-step browser task described in plain English

You do not have to pass all five. tabstackTools is an array here, so filter it or import individual tools and build your own array if the agent needs two. Fewer tools means fewer chances to pick the wrong one, which matters more the smaller your model is.

One inconsistency to watch, because it caught me out. The aggregate export shape is not the same across the Tabstack adapters. Here and in the OpenAI Agents and LlamaIndex packages it is an array. In @tabstack/ai and @tabstack/mastra it is an object keyed by tool name. That is not sloppiness, it is each package matching what its framework expects, but if you are wiring up two of these in one codebase, check before you spread.

Notice what the system prompt is doing. It is not describing the tools, since each package ships descriptions written for the model to read. It is describing the job. That division keeps your prompts short and means tool description improvements reach you on upgrade rather than requiring a prompt rewrite.

Step 5: when the data is behind an interaction

Some things are not on a page until you do something. A search box, a region selector, a “load more” button. That is automate_browser_task:

import { automateBrowserTaskTool } from "@tabstack/langchain";

const result = await automateBrowserTaskTool.invoke({
  task: "Find the top 3 trending repositories and their star counts",
  url: "https://github.com/trending",
  guardrails: "browse and extract only, do not submit forms",
});

console.log(result.answer, result.pagesVisited, result.durationMs);

You get back answer, success, iterations, actions, durationMs, extracted, and pagesVisited. The last few are what make failures diagnosable. When the answer is wrong, actions and pagesVisited usually tell you why in about ten seconds.

Set guardrails. It is a plain English constraint on what the run is allowed to do, and on anything pointed at a live site it is the difference between a read-only browse and an agent cheerfully clicking a button you did not want clicked.

One thing to know up front: this tool runs non-interactively. It never pauses for human input, so it cannot block your agent loop waiting for someone to fill in a form. If you genuinely need a human in the middle of a browser session, that is the interactive mode flow in the API rather than this tool.

Step 6: flags and failures

Four optional inputs cover nearly everything. They are only sent when you pass them, so omitting them keeps Tabstack’s defaults.

// Render a JS-heavy SPA, fresh, as if from the UK:
await extractPageContentTool.invoke({
  url: "https://example.com/app",
  effort: "max",
  nocache: true,
  country: "GB",
});

effort takes "min", "standard", or "max". If a page comes back suspiciously empty, it is almost always a JavaScript-rendered app and "max" is the fix. That single keyword replaces installing Playwright, shipping a browser binary, and keeping both patched.

nocache matters when you are watching something that moves, like prices. country matters when the page varies by region, which for pricing pages is most of the time.

Every tool throws one error type:

import { TabstackToolError, extractPageContentTool } from "@tabstack/langchain";

try {
  await extractPageContentTool.invoke({ url: "https://example.com" });
} catch (err) {
  if (err instanceof TabstackToolError) {
    console.error(`Tabstack failed (${err.status ?? "no status"}): ${err.message}`);
  }
}

One catch instead of separate handling for transport, parsing, and API errors. status carries the HTTP status for API failures and is undefined for everything else, which is usually enough to decide between retrying and giving up.

Configuration, if you need it

The default tools read TABSTACK_API_KEY from the environment. If you need a different key per tenant, a custom base URL, or you already have an SDK client you want reused:

import { createTabstackLangchainTools } from "@tabstack/langchain";

const tools = createTabstackLangchainTools({ apiKey: process.env.MY_KEY });

Where to take it next

Turn the brief into a scheduled job. nocache: true plus a diff against last week’s run is a change monitor, and it is maybe twenty more lines.

If you are moving this to a streaming UI, the same five tools exist in @tabstack/ai for the Vercel AI SDK, under identical names. Your system prompt carries over unchanged.

The package is on npm as @tabstack/langchain, and the full API reference is at docs.tabstack.ai.

Ready to build? Sign up for 10,000 free credits, no card required, and generate your first competitor brief. Full API in the docs.

Build a research agent that cites its sources with LangChain and Tabstack

Tue, 28 Jul 2026 05:00:00 GMT

Most agent tutorials stop at the point where the model says something plausible. I want to go one step further, because the interesting part is never the answer. It is the list of URLs underneath it.

We are going to build a small research agent in Python. By the end you will have an agent that can read a specific page, extract typed fields from it, and answer open questions using multiple sources with citations attached. Roughly forty lines of code, no browser to install.

If you have written LangChain before, this will feel familiar. If you have not, everything you need is here.

What you need

Python 3.9 or later
A Tabstack API key from console.tabstack.ai
An OpenAI key, or any other chat model LangChain supports

pip install langchain-tabstack langchain langchain-openai

export TABSTACK_API_KEY="your-key-here"
export OPENAI_API_KEY="your-openai-key"

langchain-tabstack pulls in the official Tabstack SDK for you. There is nothing else to install, and specifically there is no Playwright and no browser binary. Page rendering happens on Tabstack’s side.

Step 1: call a tool on its own

Before wiring anything into an agent, prove the plumbing works. Every tool in the package is exported individually and can be invoked directly, which makes debugging much easier later.

Create research.py:

import json
from langchain_tabstack import research_question_tool

raw = research_question_tool.invoke(
    {"query": "What are the latest developments in quantum error correction?"}
)

result = json.loads(raw)

print(result["answer"])
print()
for source in result["sources"]:
    print(f"- {source['title']}: {source['url']}")

Run it:

python research.py

You get a synthesised answer, then the pages it was built from. That second part is the bit worth pausing on. The tool did not just fetch a search results page and hand the model a wall of text. It read across multiple sources and returned the answer alongside the specific pages behind it, so you can put those in front of a user.

Two things about the return shape. It is a JSON string, not a dict, so json.loads is doing real work. That is the LangChain Python convention for tool output rather than a Tabstack quirk. And sources is a list of {"title": ..., "url": ...} objects, which drops straight into a citation list in whatever you are building.

The tool read TABSTACK_API_KEY from the environment on its own. The client is created lazily, so importing the package without a key set does not blow up, which keeps your imports safe in tests.

Step 2: extract typed data from a known page

Research is for open questions. When you already know which page holds the data, use extract_structured_data instead. You describe the shape you want, and you get that shape back.

import json
from langchain_tabstack import extract_structured_data_tool

schema = {
    "type": "object",
    "properties": {
        "stories": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "title": {"type": "string"},
                    "points": {"type": "number", "description": "Story points"},
                },
            },
        }
    },
}

raw = extract_structured_data_tool.invoke(
    {
        "url": "https://news.ycombinator.com",
        "json_schema_json": json.dumps(schema),
    }
)

for story in json.loads(raw)["stories"]:
    print(story["points"], story["title"])

Note json_schema_json. The schema goes in as a JSON encoded string rather than a dict, which looks odd for about ten seconds until you remember who else calls this tool. When an agent invokes it, the model has to produce that argument, and a string is a shape models handle reliably.

Write real descriptions on your fields. "description": "Story points" is not decoration, it is the instruction that tells the extractor which number on the page you actually meant. Schemas with vague or missing descriptions are the single most common cause of disappointing extraction results.

Step 3: hand the whole toolset to an agent

Now the part you came for. Instead of deciding which tool to call, let the model decide.

from langchain.agents import create_agent
from langchain_tabstack import TABSTACK_TOOLS

agent = create_agent(
    "openai:gpt-4o",
    tools=TABSTACK_TOOLS,
    system_prompt=(
        "You are a research assistant with web intelligence tools. "
        "Use extract_structured_data for specific fields from a URL, "
        "extract_page_content for full page text, and research_question "
        "for multi-source research. Always cite the sources you used."
    ),
)

result = agent.invoke(
    {"messages": [{"role": "user", "content": "What are Vercel's pricing plans?"}]}
)

print(result["messages"][-1].content)

One version note, because it is the first thing that will bite you. create_agent is the LangChain 1.x replacement for AgentExecutor plus create_tool_calling_agent. If you are still on 0.x you will get an import error here, and the fix is upgrading LangChain rather than anything to do with Tabstack.

TABSTACK_TOOLS is all five tools, ready to pass straight in:

Tool	When the model reaches for it
`extract_structured_data`	Specific fields from a URL you already know
`extract_page_content`	The whole page as clean markdown
`research_question`	An open question needing several sources
`generate_structured_data`	Fetch a page, then AI-transform it into derived JSON
`automate_browser_task`	A multi-step browser task described in plain English

You do not have to pass all five. TABSTACK_TOOLS is a list, so slice it or import individual tools if your agent only needs two. Fewer tools means fewer chances for the model to pick the wrong one, and on smaller models that matters.

Ask it something that needs more than one page and watch which tool it selects. That decision is driven by the tool descriptions shipped in the package, which is why they are written for the model to read rather than for you.

Step 4: the flags you will need in production

Four optional inputs cover almost everything you will hit. They are only sent when you pass them, so leaving them out keeps Tabstack’s defaults.

from langchain_tabstack import extract_page_content_tool

markdown = extract_page_content_tool.invoke(
    {
        "url": "https://example.com/app",
        "effort": "max",      # full server-side browser rendering
        "nocache": True,      # skip the cache, fetch fresh
        "country": "GB",      # fetch as if from the UK
    }
)

effort takes "min", "standard", or "max". If a page comes back suspiciously empty, it is almost always a JavaScript-rendered app and "max" is the fix. This is the flag that replaces an entire Playwright setup, and it costs you one keyword argument instead of a browser binary in your Dockerfile.

nocache matters when you are monitoring something that changes, like prices or stock. country matters when the page you want varies by region, which for pricing pages is most of the time.

research_question takes mode as "fast" or "balanced" instead of effort. Start with "fast" while you are iterating, move to "balanced" when quality matters more than latency.

Step 5: handle failures like an adult

Pages go away. Sites rate limit. Networks wobble. Every tool in the package raises one error type:

from langchain_tabstack import TabstackToolError, extract_page_content_tool

try:
    markdown = extract_page_content_tool.invoke({"url": "https://example.com"})
except TabstackToolError as err:
    print(f"Tabstack failed ({err.status or 'no status'}): {err}")

One except clause instead of catching transport errors, parse errors, and API errors separately. err.status carries the HTTP status for API failures and is None for everything else, which is usually enough to decide between retrying and giving up.

Step 6: go async

If you are fetching several pages, do not do it one at a time. Every tool supports ainvoke natively, backed by an async Tabstack client, so it runs in your event loop rather than bouncing through a thread pool:

import asyncio
from langchain_tabstack import extract_page_content_tool

async def fetch_all(urls):
    return await asyncio.gather(
        *(extract_page_content_tool.ainvoke({"url": url}) for url in urls)
    )

pages = asyncio.run(
    fetch_all(
        [
            "https://example.com/one",
            "https://example.com/two",
            "https://example.com/three",
        ]
    )
)

Three pages in the time of the slowest one. This is the difference between an agent that feels responsive and one people abandon.

Where to take it next

You have the whole surface now. The obvious next moves:

Swap research_question output into your UI as a real citation list rather than printing it. The sources array is already the right shape for that.

Try generate_structured_data when the field you want is not written on the page. Extraction pulls out what exists. Generation produces what does not, like a category or a one-line summary, in a schema you define.

If you move this agent to TypeScript later, the tool names and inputs are identical in @tabstack/langchain and @tabstack/ai. Your prompts carry over unchanged, because the model sees exactly the same tools.

The full API reference lives at docs.tabstack.ai, and the package is on PyPI as langchain-tabstack.

Ready to build? Sign up for 10,000 free credits, no card required, and give your agent its first cited answer. Full API in the docs.

Turn any page into clean markdown or typed JSON, cheaply

Mon, 20 Jul 2026 05:00:00 GMT

If I had to bet on which Tabstack endpoint you will call most often, I would put my money on /extract without thinking twice. It is not the one people get excited about in the demo. Research and automate get the “wow.” Extract is the one that ends up in production, running thousands of times a day, doing the unglamorous work that everything else is built on top of.

Here is the thing about extract: it is deterministic, it is cheap, and it does exactly one job per call. Those are not limitations. They are the reason it is the right default for most web data work. Before you reach for an agent, ask whether the answer is just “fetch this page and give me the clean version.” A lot of the time, it is.

Extract has two methods. They solve two different problems, so let me take them one at a time.

extract.markdown: the page, minus the noise

Every web page is wrapped in stuff you do not want. Navigation, ads, cookie banners, sidebars, footers, the same boilerplate on every page of the site. When all you need is the actual content, extract.markdown fetches the URL, strips the cruft, and hands you clean markdown.

import Tabstack from '@tabstack/sdk';

const client = new Tabstack();

const result = await client.extract.markdown({
  url: 'https://example.com/blog/article',
});

console.log(result.content);

By default, the page’s metadata (title, author, description, and so on) comes embedded as YAML frontmatter at the top of content. That is handy if you are feeding a static site generator that already expects frontmatter. For most programmatic use, though, you want the metadata as a structured object, not a string you have to parse. Add metadata: true:

const result = await client.extract.markdown({
  url: 'https://example.com/blog/article',
  metadata: true,
});

console.log(result.content);          // pure markdown, no frontmatter
console.log(result.metadata?.title);  // "Example Article Title"

Now content is clean markdown and metadata is a typed object with title, author, publisher, image, site_name, type, and friends. No YAML parsing, which means no broken parse the day a title contains a colon or a quote.

Where does this earn its keep? Mostly two places. The first is anything that feeds an LLM: RAG pipelines, summarization jobs, content classification. Models work better on clean markdown than on a soup of HTML, and you are not paying for tokens you will throw away. The second is content systems: read-it-later apps, aggregators, archives, anything that wants a stable, storable version of a page instead of the live HTML.

A couple of parameters worth knowing:

effort controls how hard the renderer works, with values min, standard (the default), and max. Static pages are fine on standard. Pages that build their content with JavaScript may need max. Pages that are already plain HTML can drop to min. This is a direct cost and speed lever, so it is worth tuning per source rather than leaving on default everywhere.
nocache defaults to false. Extract caches for a short window, which is a win for blog posts and docs. For breaking news or anything that changes minute to minute, set nocache: true to force a fresh fetch.
geo_target takes a country code ({ country: 'US' }) when the page serves different content by region.

extract.json: the shape you asked for, every time

extract.markdown gives you the whole page. Often you do not want the whole page. You want three fields out of it, typed, ready to drop into your code. That is extract.json.

You pass a URL and a JSON schema. You get back data in exactly that shape.

const result = await client.extract.json({
  url: 'https://news.ycombinator.com',
  json_schema: {
    type: 'object',
    properties: {
      stories: {
        type: 'array',
        items: {
          type: 'object',
          properties: {
            title: { type: 'string' },
            points: { type: 'number' },
          },
        },
      },
    },
  },
});

// { stories: [ { title: "...", points: 342 }, ... ] }

The response matches your schema. Not “close to” your schema. You get strings where you asked for strings and numbers where you asked for numbers, instead of everything arriving as text that you then coerce by hand. There is no HTML parsing on your side, no brittle selectors that break the next time the site ships a redesign.

Two behaviors worth internalizing:

First, a missing field does not throw. A single field the extractor cannot fill, on one page in a batch of a thousand, will not blow up the whole run, which is exactly what you want in production. What comes back for that field is not guaranteed to be null, though: depending on the type, you may get null or a best-effort value the extractor inferred. So validate the values you get back, do not just check for null, and mark a field required when you want the extractor to work harder to find it.

Second, your schema descriptions are doing real work. Adding a description to a property gives the extractor context about what to look for. Start with a simple schema, run it, and add descriptions to the fields that come back wrong. It is the highest-leverage tuning you can do.

json_schema: {
  type: 'object',
  properties: {
    price: {
      type: 'number',
      description: 'The current sale price in USD, not the original or crossed-out price.',
    },
  },
}

That one description is the difference between extracting the right number and the struck-through one.

Where extract is the wrong tool

This is the honest part, and it is the same boundary I drew for generate. Extract pulls what is on the page. It does not form opinions. If the field you want is “is this pricing aggressive” or “summarize this in two sentences,” that is interpretation, and it belongs to /generate/json, which puts a model in the loop on purpose.

The reason to keep that line clean is cost and determinism. Extract is the cheaper, repeatable workhorse. Reach for it whenever the data is literally on the page, which is more often than you would guess. Save the AI endpoints for the jobs that actually need judgment.

If you are deciding between extract and research: extract is for when you have the URL. Research is for when you have a question and do not yet know which page holds the answer. Different problems, different endpoints.

Getting started

npm install @tabstack/sdk
# or
pip install tabstack

The SDK reads TABSTACK_API_KEY from your environment, so a first call is two lines:

from tabstack import Tabstack

client = Tabstack()
result = client.extract.markdown(url="https://example.com/blog/article")
print(result.content)

Start with extract.markdown to confirm your key works and see what a clean page looks like. Then move to extract.json the moment you know which fields you actually want. Most of the web data problems you think need a pipeline are one extract call and a schema. Ready to build? Sign up for 10,000 free credits, no card required, and make your first extract call. Full API in the docs.

Tabstack on Stripe Projects

Wed, 15 Jul 2026 05:00:00 GMT

At Tabstack, we build the advanced web-browsing, automation, and research toolchains that power autonomous AI agents. Yet, every agent project eventually hits a setup wall: your agent needs an API key. That means leaving whatever you were building, going to another dashboard, creating an account, adding billing, copying a credential, and wiring it into your environment. For a human developer, this is friction. For an AI agent trying to bootstrap its own tooling, it is a hard stop.

Stripe Projects resolves this exact architectural bottleneck.

Stripe Projects: Commercial Routing for Machines

Stripe Projects is a new workflow in the Stripe CLI. Currently in developer preview, it lets developers provision third-party services directly from within a Stripe project. Once provisioned, credentials inject automatically into the project environment, and billing flows natively through Stripe. Think of it as a marketplace where an AI agent can walk in, select the required tools, and exit with functional credentials and an established billing relationship.

When you add a service through Stripe Projects, Stripe Projects coordinates account creation or linking with Tabstack, provisions the resource, and drops the credentials directly into your .env file. Your agent moves from zero to a working API key with just a few terminal commands.

Terminal-First Workspace Setup

Starting today, Tabstack is participating in the Stripe Projects developer preview. As a co-design partner on the underlying Agentic Provisioning Protocol, we worked closely with Stripe to ensure that provisioning and credential handoff work reliably and deterministically across providers.

With Stripe Projects, you can initialize your app workspace and attach the Tabstack toolchain directly from the terminal:

# Initialize the project environment
stripe projects init my-app

# Provision the Tabstack resource
stripe projects add tabstack/api

You will then be prompted to select a Tabstack plan for your account (you can also view our full tier breakdown on the Tabstack pricing page). The Stripe CLI triggers an interactive menu directly in your terminal to make your selection:

Plan        Price             Description
trial       Free              10,000 credits included.
individual  Usage based       $0.35 per 1,000 credits.
team        $99/month         500,000 credits included, then $0.30 per 1,000.
pro         $499/month        3,000,000 credits included, then $0.25 per 1,000.

Once you select a plan and confirm, the CLI provisions your Tabstack account while injecting TABSTACK_API_KEY into your project environment. On paid plans, billing details are set up once and shared securely with your SaaS stack through a Shared Payment Token, eliminating separate invoice management. Resources continue to live under your own Tabstack account with standard access and dashboard configurations intact.

Unlocking the Toolchain

Once provisioning is complete, your Tabstack API key gives you or your agent full access to all Tabstack features:

/extract/markdown: Converts any URL into clean Markdown, built for an LLM context window.
/extract/json: Converts web pages into structured JSON matching a defined schema.
/generate/json: Generates structured data from web content using natural language payloads.
/automate: Navigates, clicks, fills forms, and completes multi-step web workflows.
/research: Runs a multi-pass agentic research loop and outputs a synthesized, cited report.

Initialize the SDK once the credential populates your environment variables:

import { Tabstack } from '@tabstack/sdk';

const client = new Tabstack({
  apiKey: process.env.TABSTACK_API_KEY
});

// Target the markdown extraction backend directly
const result = await client.extract.markdown({
  url: 'https://example.com/docs'
});

console.log(result.content);

Lifecycle Management

Beyond provisioning, you can manage your entire Tabstack resource lifecycle directly from the terminal, making it simple to upgrade tiers, rotate credentials, or log in to the Tabstack dashboard without leaving your workflow:

# Rotate your API key
stripe projects rotate tabstack/api

# Open the Tabstack dashboard for your account
stripe projects open tabstack

# Upgrade to a higher plan
stripe projects upgrade tabstack/api

# Remove the service and deprovision the account
stripe projects remove tabstack/api

Designing for Autonomy

This integration is about more than just developer convenience; it is about building infrastructure native to machines.

Manual credentials break autonomous systems. When an AI agent relies on one-off dashboard navigation or hardcoded keys, it stalls. Eliminating this manual setup allows the agent to provision its own browsing infrastructure dynamically as dependencies scale.

When an automated deployment script triggers the workspace setup, a functional key drops in thirty seconds later with billing wired instantly. This unlocks entirely new operational patterns: automated project initialization, self-healing runtimes that swap out broken tools, and dynamic infrastructure scaling based on task complexity.

Tabstack handles the browsing. Stripe handles the billing. The agent handles the execution.

Next Steps

This integration is in developer preview, and we want your feedback. To get started, install the Stripe CLI here, initialize your app workspace, and attach Tabstack to your environment:

stripe projects init my-app
stripe projects add tabstack/api

Review the full API documentation at docs.tabstack.ai.

Introducing Schema Source: Programmatic Blueprinting for Web Data Extraction

Fri, 26 Jun 2026 05:00:00 GMT

We built Tabstack to replace traditional parsing configurations with a declarative model. Instead of telling a scraper how to navigate a page, you define what data you want using a JSON schema, and our /extract/json and /generate/json endpoints handle the execution.

Schema-first extraction bypasses structural web shifts, but introduces a predictable bottleneck: manual schema design.

Manually building a valid, production-grade JSON schema for a complex web page is tedious. It requires inspecting the DOM, identifying data hierarchies, mapping out types, and writing detailed string descriptions so the underlying LLMs understand the context of each field.

To automate this setup phase, we built Schema Source.

What is Schema Source?

Schema Source is an independent API that reverse-engineers a webpage’s visual and semantic layout to instantly generate a structured data schema. Pass a URL via a raw GET request to instantly retrieve a structured JSON blueprint.

For example, if you point it at a content-heavy community page like https://reddit.com/r/nba, Schema Source analyzes the page components and automatically outputs an optimized JSON schema mapping the posts, metrics, and metadata.

You can then pass this schema directly into Tabstack’s extraction pipelines to begin pulling live data immediately.

How Schema Source Fits into the Tabstack Workflow

In a schema-first architecture, object descriptions act as system prompts. If your schema has a field called score, an LLM needs to know if that means a sporting event’s box score, a user upvote count, or a sentiment rating.

Schema Source handles this contextual work by generating explicit metadata alongside the data types.

Step 1: Bootstrapping the Schema via Schema Source

To generate a structural baseline for a target page, append your target URL directly to the Schema Source endpoint:

async function generateSchema(url) {
	const res = await fetch(
	  `https://schema.tabstack.ai/get/${encodeURIComponent(url)}?format=json`,
	);

  const { schema } = await res.json();

  return schema;
}

The API analyzes the target page and returns a clean, production-ready JSON schema tailored to its core data elements:

{
   "type":"object",
   "properties":{
      "posts":{
         "type":"array",
         "description":"List of popular posts on Reddit's front page",
         "items":{
            "type":"object",
            "properties":{
               "title":{
                  "type":"string",
                  "description":"The title of the Reddit post"
               },
               "author":{
                  "type":"string",
                  "description":"The username of the post's author"
               },
               "score":{
                  "type":"number",
                  "description":"The score (upvotes) of the post"
               },
               "comment_count":{
                  "type":"number",
                  "description":"The number of comments on the post"
               },
               "permalink":{
                  "type":"string",
                  "description":"The direct link to the Reddit post discussion page"
               },
               "published_at":{
                  "type":"string",
                  "description":"The timestamp when the post was published"
               }
            },
            "required":[
               "title",
               "author",
               "score",
               "comment_count",
               "permalink",
               "published_at"
            ]
         },
         "maxItems":10
      }
   },
   "required":[
      "posts"
   ]
}

Step 2: Running a Direct Extraction with Tabstack

Deploying the schema into a production pipeline requires zero manual layout configuration. Pass the generated schema directly into the official @tabstack/sdk to execute a literal data extraction.

import Tabstack from '@tabstack/sdk';

const client = new Tabstack({ apiKey: process.env.TABSTACK_API_KEY });

const url = 'https://reddit.com/r/nba';
const json_schema = await generateSchema(url);

async function extractLiteralData() {
  const response = await client.extract.json({
    url,
    json_schema,
  });

  console.log('Extracted Data:', JSON.stringify(response.data, null, 2));
}

Step 3: Layering AI Transformation with generate.json

If your pipeline requires transforming, filtering, or analyzing data rather than just mirroring what is literally on the page, you can use the exact same schema with Tabstack’s /generate/json endpoint.

While /extract/json returns literal data from the DOM, generate allows you to pass custom prompt instructions (like summarization or synthesis) while guaranteeing the engine wraps those modified results perfectly inside your schema structure:

async function extractAndTransformData() {
  const response = await client.generate.json({
    url,
    json_schema,
    instructions: "Filter out any threads that aren't about live game updates. For the remaining posts, rewrite the title to be a strictly objective 5-word summary.",
  });

  console.log('Transformed Data:', JSON.stringify(response.data, null, 2));
}

Decoupling Configuration from Execution

We decoupled Schema Source to isolate structural data discovery from production data ingestion pipelines.

This architecture eliminates the need to manually draft schemas or guess layout hierarchies during the initial development cycle. Teams can use Schema Source programmatically to seed baseline data definitions, then rely entirely on Tabstack’s infrastructure to fetch, parse, and validate those payloads at scale.

Try generating your first schema at schema.tabstack.ai.

Hardening AI Web Agents: How We're Securing Tabstack Against Indirect Prompt Injection

Fri, 12 Jun 2026 05:00:00 GMT

At Mozilla, we believe that building a useful AI ecosystem requires radical transparency, especially when it comes to security.

Recently, security researchers at Brave reached out to us regarding an Indirect Prompt Injection (IPI) vulnerability they identified in Tabstack’s /v1/automate endpoint, which they have since detailed in their public blog post on the flaw. Because Tabstack is built to act as an autonomous web agent that can browse, click, and interact with the live web on behalf of a user, the implications of IPI are a critical design challenge.

The vulnerability has been patched, and the fix was independently verified by the Brave team before their public write-up. We want to share a transparent look at the exploit, how our model handled it, and the architecture we’ve implemented to harden our automation engine against this entire class of attacks.

The Vulnerability: Bypassing the Scope of the Task

The attack discovered by Brave highlights the unique risks associated with “agentic” AI tools. During a controlled test, researchers passed a standard, routine prompt to the /v1/automate endpoint: “Summarize this page.”

However, the target page contained hidden, malicious instructions (rendered in white-on-white text, invisible to a human but fully readable in the page’s text layer ingested by the AI). The injected text instructed the model to ignore its previous task, grab the user’s full conversation history, paste it into an external web form, and hit submit.

Because Large Language Models mix user intent and third-party data into a single, flat text window, Tabstack didn’t view this as a security conflict. Instead, it confidently followed the instructions step-by-step:

Navigated away from the target page to an external domain.
Copied the user’s conversation context.
Submitted the form, actively exfiltrating the data to the researcher’s server.

When we analyzed the agent’s internal reasoning traces, the challenge became even clearer. The model wasn’t “tricked” or confused; it was executing what it genuinely believed to be a legitimate workflow continuation.

How We Fixed It: Moving Beyond a Flat Context

Prompt injection is a dynamic, shifting threat vector across the entire AI industry. While it is functionally impossible to guarantee any LLM is 100% immune to prompt injection, we can severely limit an agent’s ability to act on malicious inputs. That was our north star: assume the model will occasionally be fooled by injected text, and make sure that being fooled cannot translate into exfiltrated data.

Following the disclosure, our engineering team confirmed the gap and shipped a series of changes to our underlying browser automation engine, Mozilla Pilo. Rather than trying to detect malicious prompts (a losing game of pattern-matching), we focused on shrinking the agent’s “blast radius” with structural guardrails that hold regardless of what the page says.

A Structural Action Firewall for Forms

The core of the fix is a new action firewall that sits between the agent’s decisions and the browser’s execution of them. Critically, it does not work by scanning text for “suspicious” instructions. Instead, it classifies every form interaction using DOM field metadata and reference provenance: where a value came from, what kind of field it is targeting, and whether a human ever approved it.

The agent is free to fill operational controls it legitimately needs to do its job (search boxes, date pickers, range sliders, comboboxes).
It is structurally blocked from auto-filling freeform or sensitive fields, and from submitting any form that contains agent-filled data that was never approved, before the action reaches the browser.
Even operational submissions are now restricted to the same host as the current page. An attacker page can label its collector field as a “search box,” but it cannot make the agent submit that data to an attacker-controlled domain. Unknown page hosts fail closed.

When a block fires in non-interactive mode, the CLI prints a remediation footer explaining how to proceed. That footer is user-facing only: the model never sees it, so injected page content cannot instruct the agent to talk the user into disabling the firewall.

External Content Isolation

We also closed the more fundamental gap that made the original exploit possible: the agent treated web-page text and user instructions as one undifferentiated stream.

Web-sourced content returned by the agent’s tools (page extracts, fetched markdown, search results, even the completion validator’s own feedback) is now wrapped in explicit <EXTERNAL-CONTENT> tags at the point the data enters the conversation, with an inline warning on every block and a matching directive in the system prompt. This mirrors the trust-framing we already applied to raw page snapshots, so untrusted page text is consistently delineated from the user’s actual task.

This wrapping is structural, not learned: the boundary is enforced by how we construct the context, not by hoping the model “decides” to distrust the right paragraph. The same change also clips stale external content out of conversation history after a turn, which incidentally shut down a cost-amplification angle of the same attack.

Caller-Controlled Trust Boundaries, Now Per Request

Some legitimate workflows genuinely need the agent to enter data into third-party forms. Rather than weaken the firewall globally, we exposed explicit, opt-in trust controls:

trustedHostnames: A caller-supplied allowlist that bypasses the fill and submit gates, but only when the current page host and every form-action target (including submitter formaction overrides) are all on the list.
unsafeMode: A deliberate, fully-documented global opt-out, with prominent data-risk warnings on every surface.
A caller-provided start URL is treated as consent to interact with that specific site, but planner-chosen or agent-navigated URLs are not, since those are influenced by the model and the page.

Originally configured via local environment variables and core configuration files, we have exposed these parameters directly to our automation engine’s request layer. This allows the Tabstack API to pass trustedHostnames and unsafeMode dynamically inside the request body, establishing strict trust boundaries per request rather than per deployment.

Defense in Depth

Alongside the firewall and content isolation, we tightened how the agent reports completion. A new data-grounding rule requires every value the agent returns to trace back to a page snapshot, tool result, or task input (no filling in answers from the model’s own training), backed by a pre-completion verification checklist. We also wired the completion validator’s conversation history and outcome into the agent loop, and hardened the agent against getting stuck repeating injected actions across changing element references.

The Responsible Disclosure Timeline

We are incredibly grateful to the Brave research team for their clean, professional coordination on this bug. Their outreach allowed us to identify, patch, and verify a critical gap before it could ever affect users in production.

May 13, 2026: Brave reports the IPI vulnerability to the Mozilla Tabstack team.
May 14, 2026: Mozilla engineers review the traces, confirm the vulnerability, and begin engineering defenses.
June 1, 2026: Mozilla deploys the updated Pilo browser engine changes, and the Brave team independently verifies the fix.

Securing autonomous web agents is one of the most complex engineering challenges in AI right now. By treating third-party web content as inherently untrusted, creating structural boundaries, and cutting off the pathways for unauthorized data exfiltration, we are building a safer foundation for agentic workflows. We will continue to harden Tabstack out in the open, and we welcome the global security community’s continued scrutiny.

Upgrading Pilo to Support Human-in-the-Loop Browser Automation

Thu, 16 Apr 2026 05:00:00 GMT

When we open-sourced Pilo, our goal was to provide a robust execution engine for AI web agents capable of navigating the chaos of the modern web. By leveraging the accessibility tree and intelligent reasoning loops, Pilo handles complex browser orchestration reliably. But autonomous execution has a hard limit: missing context.

Today, we are excited to introduce Interactive Mode (Beta) for Pilo. This new feature allows the Pilo agent to pause its execution loop and ask the caller, whether that is a human user or a parent agent, for the specific information it needs to complete a task.

The Problem of Missing Context

In traditional web automation, agents are expected to operate entirely on their own once given a prompt. But what happens when an agent encounters a step that requires specific, localized knowledge?

Imagine giving an agent the prompt: “Sign up for the mozilla newsletter.”

The agent can successfully generate a plan, navigate to the Mozilla website, locate the newsletter signup page, and identify the required input fields. But when it tries to submit the form, it hits a roadblock: it doesn’t know your email address or your subscription preferences. Previously, this would result in a failed task or a hallucinated input. The agent was trapped in a silo, unable to request the missing piece of the puzzle.

Enter Interactive Mode

Interactive Mode transforms Pilo from a purely autonomous executor into a collaborative agent. Instead of failing when it lacks information, Pilo can now dynamically halt its loop and prompt the caller for guidance.

For our first iteration of Interactive Mode, we are focusing specifically on form completion. When Pilo encounters a form requiring personal data, credentials, or preferences it doesn’t possess, it will output a request for input. Once the caller provides the missing information, Pilo directly injects it into the form and resumes the agentic loop to complete the task.

How It Works Across the Stack

Interactive Mode integrates deeply into Pilo’s core execution engine rather than relying on the LLM to decide when to ask for help. When Pilo encounters a form, it uses a “fill gate” mechanism to inspect the required fields. If it detects that user input is needed, Pilo bypasses the LLM entirely to pause task execution and request the data.

Because Pilo can be run in several different environments, we designed the interactive feedback loop to adapt to how you are using it:

Pilo Core: When you are building directly on top of the Pilo library, the core engine expects a simple callback function. When the execution loop pauses, it triggers your callback with the required fields, waiting for your application logic to return the necessary data before resuming.
Pilo CLI: For developers testing locally, the Pilo CLI handles this automatically. When the core requests information, the CLI pauses the terminal output and interactively prompts the user for the missing fields right in the console.
Pilo Server: For remote execution, the Pilo Server emits an interactive:form_data:request event containing the full field data. It utilizes a dedicated WebSocket endpoint (/pilo/run), enabling real-time, bidirectional communication to exchange user data while the task is suspended.

Furthermore, Pilo captures form validation states directly within its ARIA tree snapshots. If a user submits an invalid email, the agent detects the field error in the snapshot and will automatically trigger a re-prompt for the corrected information.

Seeing it in Action with Tabstack

The Tabstack /automate endpoint is built directly on top of the Pilo Server. This means you can test the WebSocket-driven Interactive Mode today using the Tabstack SDK.

Here is what it looks like to catch those interactive requests and supply the missing context:

from tabstack import Tabstack

# Initialize the Tabstack client
client = Tabstack(
    api_key="YOUR_TABSTACK_API_KEY",
)

# Start a task with Interactive Mode enabled
stream = client.agent.automate(
    task="signup for the mozilla newsletter",
    interactive=True,
)

# Listen for events in the stream
for event in stream:
    print(event)

    # Catch the specific event where the agent asks for form data
    if event.event == "interactive:form_data:request":
        request_id = event.data.get("requestId")
        fields = event.data.get("fields", [])

        print(f"\n--- Interactive input requested (requestId: {request_id}) ---")
        
        field_values = []
        
        # Dynamically prompt the user for the missing fields
        for field in fields:
            ref = field.get("ref", "")
            label = field.get("label", "")
            required = field.get("required", False)
            
            prompt = f"  {label}{'*' if required else ''}: "
            value = input(prompt)
            field_values.append({"ref": ref, "value": value})

        # Submit the user's input back to the agent to resume the task
        response = client.agent.automate_input(
            request_id,
            fields=field_values,
        )
        print(f"\n--- Input response: {response} ---\n")

By decoupling the reasoning engine from the data source, Interactive Mode allows you to build more adaptable, resilient pipelines. Whether you are typing in the terminal or wiring Pilo into a complex backend where a master orchestrator agent automatically supplies the missing data, the engine easily adapts.

Just the First Step

It is important to note that Interactive Mode is currently in Beta. This initial release is highly optimized for form-filling scenarios, but this is only the first step in our vision for collaborative agents.

As we continue to test and build out this functionality, Pilo’s interactive capabilities will become much more advanced. Future iterations will support complex scenarios like resolving ambiguous instructions mid-task, requesting visual confirmations before executing destructive actions, and handling multi-step interactive workflows.

We built Pilo to be the most reliable open-source foundation for web agents, and giving those agents the ability to ask questions is a massive step forward in making them truly useful.

Check out the updated documentation on the Mozilla Pilo GitHub repository or try out the SDK at Tabstack.ai today.

Tabstack's New Pricing Plans

Mon, 06 Apr 2026 05:00:00 GMT

Unlocking Flexibility

Since day one, our goal at Tabstack has been to build tools that make your workflow smoother and more efficient. We have loved watching our community grow and build amazing things using our platform. But as your projects have evolved, we realized our pricing model needed to evolve alongside them.

Until now, Tabstack has operated exclusively on prepaid credit packs. While this worked for many, we heard your feedback loud and clear: you want clearer, consistent billing, and you want choices that adapt to the way your team actually works.

Today, we are thrilled to announce a complete overhaul of our pricing structure, designed to give you ultimate control, predictability, and flexibility.

What Is New?

We are moving away from the strict advanced credit model and introducing a tiered system that grows with you, taking you from your very first API call to enterprise deployments.

The New Trial Tier

We want everyone to experience the power of Tabstack without any friction. Every new user will now start with a Trial Tier that includes 10,000 free credits. This allows you to test the waters, build a proof of concept, and see exactly how Tabstack fits into your stack before spending a dime.

The Individual Plan (Pay As You Go)

Got a project with unpredictable volume? The Individual plan is for you. Once you graduate from the Trial Tier, you can opt to simply pay for what you use. You will be billed monthly based strictly on your exact usage. There are no upfront commitments, and you are never paying for credits you do not end up using.

Predictable Monthly Subscriptions

For teams with consistent or high volume needs, we are introducing two new subscription plans. These plans offer a set monthly price, a generous default credit allowance, and a lower cost per credit ratio.

Plan Tier	Monthly Price	Included Monthly Credits	Ideal For
Team	$99 / month	500,000 credits	Growing startups and mid-sized teams with consistent workloads.
Pro	$499 / month	3,000,000 credits	High-volume applications and established companies scaling fast.

Overages and Spend Protection

We know that traffic can spike unexpectedly. If you are on the Team or Pro plan and exceed your monthly credit allowance, your workflow will not break. You will seamlessly transition to overage billing so your services keep running smoothly.

However, we also want to protect you from unexpected bills. You now have the power to configure your own custom usage limits directly from your dashboard. You can set up alerts to notify you when you reach a certain threshold, or set your own hard caps to pause operations automatically.

Additionally, to provide an extra layer of security against massive, surprise bills, we have implemented a platform-level hard cap on all accounts. While you cannot edit this system cap directly, you can easily request an increase from our support team as your project scales.

Important Details for Existing Users

If you are already building with Tabstack, we want to make sure your transition is as smooth as possible. We are providing a 90 day grace period for all existing accounts.

During the next 90 days, you will continue to receive your legacy monthly allowance of 50,000 credits, and you can still purchase credit packs just like before. You can choose to upgrade to the Individual plan or one of the new subscription plans at any time during this window.

When you upgrade, any existing credit balance you have will be carried over to the new system, and those credits will not expire. If you have not transitioned after 90 days, your account will be updated automatically, and your remaining credit balance will still be safely preserved without an expiration date.

Updated Credit Costs

Along with the new plans, we are updating the credit cost per operation to better reflect the scale and processing power of our tools.

If you are an existing user in the 90 day grace period, you will continue to enjoy the old pricing. All new users, existing users who choose to upgrade early, and all users after the 90 day window will be subject to the new pricing structure.

Operation	Old Cost (Credits/Action)	New Cost (Credits/Action)
Extract Markdown	9	10
Extract JSON	50	100
Generate	57	250
Automate	75	500
Research Fast	25	500
Research Balanced	50	1,000

Why We Made the Switch (And Why It Is Good For You)

We did not just change our pricing to add more options; we redesigned it to solve real pain points we noticed in how users were interacting with the platform. Here is how the new system works in your favor:

Clearer, Consistent Billing: You can now budget effectively. Whether you choose the predictability of the Team and Pro tiers or the exactness of the Individual plan, your finance team will thank you.
Goodbye, “Top-Up Anxiety”: Under the old credit pack system, running out of credits meant your operations paused until someone manually logged in and bought more. Now, your workflow remains entirely uninterrupted.
Scale at Your Own Pace: Start for free, scale to the Individual plan as you gain traction, and lock in savings with a subscription tier once your volume is established. You are never forced into a box that does not fit your current size.
No Bill Shock: With customizable alerts and platform safety limits, you get the peace of mind that your budget is protected even if your application goes viral overnight.
Lower Barrier to Entry: The 10k credit Trial Tier allows developers and creators to experiment freely without needing to pull out a company credit card right away.

Ready to Explore the New Plans?

These new plans are live today! Head over to your dashboard to view your current usage, explore the new tiers, and select the plan that makes the most sense for your next big project.

As always, if you have any questions, our support team is ready to help you find the perfect fit. Happy building!

Not Every Page Needs a Browser

Tue, 17 Mar 2026 05:00:00 GMT

The Problem with One-Size-Fits-All

When you point an extraction API at a URL, you typically have no say in how the page gets fetched. A static documentation page and a JavaScript-heavy single-page application go through the same pipeline, with the same overhead and the same latency. You end up paying a time penalty on simple pages because the system has to assume every page might be complex.

The reality is that most pages on the web are straightforward. A news article, a product listing on a server-rendered site, a wiki page: none of these need a full headless browser to extract. But a React dashboard behind client-side rendering absolutely does.

You know your target sites. We should let you tell us.

What Changed

We made two changes to how Tabstack handles page fetching.

First, we rearchitected the default fetch path to be significantly faster and more reliable. The "standard" effort level that every request uses by default is now smarter about how it retrieves content, with better fallback mechanisms and reduced overhead.

Second, we added the effort parameter to the /extract/json, /extract/markdown, and /generate/json endpoints. This gives you explicit control over how hard Tabstack works to fetch a page before extraction begins.

There are three levels:

"min": a lightweight fetch with no fallback. This is the fastest option usually under 1 second, ideal for static or server-rendered pages where you know the content is available without JavaScript execution.
"standard": the default. A balanced approach with enhanced reliability and fallback mechanisms, typically completing in less than 10 seconds. This will handle most pages, including ones with some JavaScript. Start here for any new integration.
"max": full browser rendering. This spins up a headless browser, executes JavaScript, waits for dynamic content to load, and then extracts. It takes 15 to 60 seconds but is necessary for SPAs, pages behind client-side hydration, or sites that load content dynamically. Try this when the other levels are not getting the content you want.

Using Effort

Adding the parameter is straightforward. Here is a basic example that extracts markdown from a JavaScript-heavy page:

import os
from tabstack import Tabstack

client = Tabstack(api_key=os.environ.get("TABSTACK_API_KEY"))

# Full browser rendering for a JS-heavy page
result = client.extract.markdown(
    url="https://example.com/spa-dashboard",
    effort="max"
)

print(result.content)

For a static docs site where speed matters, drop to "min":

result = client.extract.markdown(
    url="https://docs.example.com/api-reference",
    effort="min"
)

Adaptive Effort: Start Fast, Escalate When Needed

The most powerful pattern with the effort parameter is adaptive escalation. Instead of guessing the right level upfront, you start with the fastest option and only escalate when the result is insufficient. This gives you the best possible latency on easy pages while still handling complex ones reliably.

import os
from tabstack import Tabstack

client = Tabstack(api_key=os.environ.get("TABSTACK_API_KEY"))

def extract_with_adaptive_effort(url, min_length=100):
    """Extract markdown, escalating effort until we get a good result."""
    efforts = ["min", "standard", "max"]

    for effort in efforts:
        result = client.extract.markdown(url=url, effort=effort)

        if result.content and len(result.content) >= min_length:
            return result

    return result  # Return whatever we got at max effort

result = extract_with_adaptive_effort("https://example.com/products")
print(result.content)

This pattern works well in production pipelines where you process a mix of sites. Simple pages resolve in a couple of seconds at "min", and only the pages that genuinely need browser rendering incur the additional latency. Your average response time drops significantly while your success rate stays high.

When to Use Each Level

Here is a quick guide for choosing the right effort level:

Use "min" when you are scraping well-known static sites, server-rendered pages, or any URL where you have verified that lightweight fetching returns complete content. Great for high-volume pipelines where speed is critical.
Use "standard" when you are integrating with a new site or processing a mix of URLs. It handles most of the web reliably without the overhead of full rendering.
Use "max" when content is missing from lower effort levels, the target is a known SPA or JavaScript-heavy application, or the page loads data asynchronously after initial render.

Get Started

The effort parameter is available now on all extract and generate endpoints. Update to the latest SDK version to use it, or pass effort directly in your request body if you are calling the API without an SDK.

API Reference: docs.tabstack.ai
Tabstack Dashboard: tabstack.ai

If you are building pipelines that process diverse URLs, try the adaptive effort pattern. Start fast, escalate smart, and let your code decide how hard to try.

Scaling Web Automation with Pilo and Agentic AI

Tue, 24 Feb 2026 06:00:00 GMT

The Chaos of the Modern Web

Building reliable web automation is notoriously difficult. Modern websites are dynamic, heavily reliant on JavaScript, and constantly changing. When you add Large Language Models to the mix to create autonomous web agents, the complexity multiplies. You suddenly have to manage browser orchestration, keep token costs down while maintaining full page context, handle flaky networks, and build complex reasoning loops just to get an AI to reliably extract data or click a button. You end up managing massive amounts of infrastructure just to handle a basic reasoning loop.

We didn’t just stumble upon these problems; we ran into them headfirst while building Tabstack. We needed a reliable engine to power our own /automate endpoint, but nothing off the shelf could handle the chaos of the modern web without constant breakage. So we built our own solution.

We call it Pilo. It became the bedrock of our platform, and we realized that developers everywhere need this same robust, transparent foundation. Today, we are very excited to share Pilo as an open source project, enabling anyone to run this powerful web automation engine completely on their own infrastructure.

You can find the full source code and documentation on Github here.

Giving AI the Steering Wheel

At its core, Pilo moves away from rigid scripts and embraces an agentic loop built around Playwright’s accessibility tree. Instead of telling a program exactly where to click, you give Pilo a natural language goal. Pilo then takes over and makes decisions at every step based on what it actually sees on the screen.

The system operates in a continuous, intelligent loop:

Observe: Pilo captures the current page state using the browser’s accessibility tree. This provides a semantic, stable structure of the page rather than a chaotic mess of raw HTML tags.
Decide: Pilo passes this state to an LLM provider of your choice. The LLM evaluates the context and selects a specific tool to use next.
Act: Pilo executes the chosen action, whether that is navigating to a URL, clicking a button, filling out a form, or extracting structured data.

Pilo handles all the gritty details under the hood. It features layered retries for flaky navigation, automatic context truncation to prevent context window bloat during long tasks, and a robust error recovery philosophy. If an action fails, Pilo does not simply crash. It feeds the error back to the LLM so the agent can adapt and try a new approach.

Anatomy of an Autonomous Action

To understand how Pilo handles the complexities of the web, let’s look under the hood at a task like “find the best pizza restaurants in Seattle”.

1. The Planning Phase: Before launching a browser, Pilo forces the LLM to pause and strategize. It calls a create_plan tool to generate a step-by-step path and, crucially, defines strict success criteria. For a search-heavy task like this, the planner might smartly set the starting URL to about:blank, signaling the agent to immediately trigger its search workflow rather than wasting time navigating to a specific homepage.

2. Layered Navigation: Navigating to a URL is the single most failure-prone operation in browser automation. Pilo wraps navigation in a layered defense system. It starts with timeout escalation, doubling the allowed wait time on each failure. If the network is truly flaky and throws a DNS error, Pilo will automatically kill and restart the entire browser instance to clear any bad state before retrying.

3. Compressing the Matrix: Once the page loads, Pilo needs to show it to the LLM. Passing raw HTML is too expensive and noisy, so Pilo captures the accessibility tree, a semantic map of the page’s structure.

It pipes this tree through a compression engine that maps verbose tags like listitem to li, shortens reference IDs, and deduplicates repetitive text. This process slashes token usage by 60 to 80 percent while keeping every interactive element accessible.

4. The Decision Loop: The LLM reviews the compressed snapshot and selects a tool, such as click. Pilo resolves the reference ID (e.g., e45) to a live Playwright locator and executes the action. If the page shifted and e45 is gone, Pilo catches the InvalidRefException and feeds it back to the LLM as a recoverable error. The agent sees the failure, realizes the page state has changed, and naturally requests a fresh snapshot to try again.

5. Quality Control: When the agent thinks it’s finished and calls done, Pilo doesn’t just take its word for it. It triggers an isolated validation step where a second LLM acts as a grader. It compares the final results against the success criteria defined in step one. If the agent found only two restaurants when the plan demanded five, the validator marks it as “partial” quality and sends the agent back to work with specific feedback on what’s missing.

Seeing is Believing: Pilo in Your Browser

While Pilo is a powerful backend engine, we wanted to make its capabilities completely tangible. Because the event system decouples the core logic from where it runs, Pilo is built to work seamlessly in different contexts, including a browser extension. You can install the extension, give it a natural language prompt, and literally watch the agentic loop drive your browser in real time. It is one thing to read about accessibility trees and LLM decision making; it is entirely another to sit back and watch an AI navigate, click, and extract data across the live web right in front of your eyes. It provides a perfect sandbox to test tasks and understand exactly how Pilo thinks and operates.

The Engine Inside Tabstack

We built Pilo to be completely standalone. You can run it locally, plug in your own API keys, and use it to power your own applications without ever signing up for Tabstack. Our goal is to provide developers with a reliable, open source foundation for web agents.

However, building and scaling this kind of infrastructure in production is incredibly resource intensive. You have to manage compute pools, handle persistent browser sessions, and scale concurrent tasks. This exact challenge is why we built Tabstack in the first place.

Pilo is the core engine that underlies our /automate endpoint. If you want to leverage the power and resilience of Pilo but do not want to manage the underlying infrastructure, browser orchestration, or scaling logistics, Tabstack provides all of this out of the box.

Tabstack Research: Verified Answers from the Open Web

Tue, 03 Feb 2026 06:00:00 GMT

Web browsing is the hidden tax of AI development. When you connect an agent to the open web, you are forced to stop building AI and start managing the overhead of browser orchestration. You end up debugging JS-rendering, rotating proxies, and writing brittle selectors just to turn the chaotic web into clean inputs. This is why we built Tabstack: to turn browsing into a reliable infrastructure layer.

But even with a stable browser fleet, developers face a secondary bottleneck: The Synthesis Gap.

Fetching raw data from a single URL is an infrastructure problem. Answering a complex question like “Compare Slack vs. Teams retention policies” is a state management and reasoning problem. To answer that, an agent must spawn a fleet of parallel searches, navigate dozens of unvetted URLs, filter out 90% marketing noise, and reconcile conflicting data.

Processing this at scale creates a direct conflict between latency and context window density. If you feed your model every raw byte, you burn your token budget on noise. If you truncate too early, you lose the ground truth.

Today we are launching Tabstack Research. It is a high-level research primitive that moves the autonomous reasoning loop into the infrastructure layer. You give us a goal. We execute the coordinated workflow of discovery, extraction, and verification to return a synthesized report backed by source citations.

The Drudgery of Scale

Most developers begin by wrapping a basic scraper in a loop, but they quickly hit an infrastructure wall. Scaling a research task does not just increase your bandwidth requirements. It multiplies your orchestration complexity.

When an agent tackles a non-trivial query, it rarely performs a single lookup. It spawns a fan-out of parallel sub-queries. In this scenario, “ten blue links” quickly explode into 100 unvetted URLs.

Processing this volume in production requires you to solve three high-stakes problems:

The Concurrency Bottleneck: Running 100 parallel headless sessions to handle modern, JS-heavy sites requires massive compute overhead. Managing the memory leaks, zombie processes, and proxy rotation needed to avoid rate limits turns into a full-time DevOps job.
The Token Tax: Consumption is not comprehension. Shoveling 50,000 tokens of raw HTML boilerplate into your model is a recipe for high latency and “lost in the middle” reasoning errors. Without a pre-processing layer, you are paying to process navigation menus and footer links instead of actual data.
The Resolution Logic: You are forced to build custom logic to reconcile conflicting data points across different domains. You end up spending more time on data cleaning and deduplication than on your core agent logic.

The result is a fragile pipeline where engineering cycles are wasted managing a browser fleet instead of refining the product.

An Adaptive Research Loop

We built Tabstack Research to move this orchestration logic out of your application and into our specialized browsing layer. When you send us a request, we initiate a multi-phase agentic loop designed to mimic the recursive nature of human research at machine speed.

By offloading the “discovery and verification” cycle to our infrastructure, you avoid the complexity of building custom state machines to handle search branching and error correction.

Behind the scenes, the system executes a coordinated workflow that treats research as an iterative process rather than a linear fetch:

Planning and Decomposition: The system mimics a researcher’s initial “mental map” by breaking your goal into targeted sub-questions. It identifies that a true comparison requires hitting distinct data silos: official documentation, enterprise pricing tables, and compliance whitepapers.
Parallel Execution: We visit these sites in parallel using our core browsing infrastructure. We perform real-time content extraction, filtering out the DOM noise and marketing fluff that typically bloat a context window.
Gap Evaluation: This is the recursive heart of the system. The system evaluates the collected data against the original intent. If it identifies a missing variable or a conflicting date, it detects the gap and triggers a new iteration to hunt down that specific information.
Verification and Termination: The loop concludes when the system determines the claims are sufficiently verified against the source text or when it reaches the iteration limit for the selected mode. For example, while the system stops early if it has all the required information, Balanced Mode allows for up to three rounds of recursive discovery to ensure the final output is grounded in retrieved evidence rather than model weights.

Anatomy of a Request

To see how this works in production, consider a complex research task: “Compare enterprise data retention policies for Slack vs. Microsoft Teams.”

import os from tabstack import Tabstack

# Initialize the client
tabs = Tabstack(api_key=os.getenv('TABSTACK_API_KEY'))

# Ask a research question
result = await tabs.agent.research(
    query="Compare enterprise data retention policies for Slack vs. Microsoft Teams.",
    mode="balanced",
)

print(result)

If you pass this query to a standard LLM or a basic search-based agent, you will likely get generic advice or a surface-level summary of the marketing pages. Here is how the Tabstack Research loop executes it:

The Planning Phase

The system deconstructs the high-level prompt into a series of technical primitives. It identifies that “retention” is not a single value. It generates sub-queries for “Teams Purview chat retention,” “Slack Enterprise Grid file storage limits,” and “M365 E5 compliance overrides.”

The Discovery Phase

It executes these searches in parallel. It prioritizes official documentation like learn.microsoft.com and slack.com. Critically, our browsing layer ignores the SEO-optimized “Top 10” blog posts from 2023 that no longer reflect the 2026 pricing and policy landscape.

The Recursive Pivot

During extraction, the system hits a common stumbling block. Initial results for Teams mention a 30-day default retention. However, our evaluation step detects a critical gap: Files shared in Teams are actually stored in SharePoint and OneDrive, which often have conflicting retention policies. A standard agent would miss this nuance. Tabstack Research detects the ambiguity and triggers a targeted, second-pass search to clarify the “Conflict Resolution” logic between these interdependent Microsoft services. The engine identifies that while messages appear in Teams, the actual preservation happens in a hidden “SubstrateHolds” folder within the “Exchange Recoverable Items” folder. This is a nuance that dictates how legal holds are actually applied and accessed.

The Verification Phase

The final report is compiled with specific, time-sensitive data points that are verified against the source text to mitigate the risk of model hallucinations:

Service Interdependence: It clarifies that while Teams manages the message metadata, SharePoint manages the physical file retention.
Price Adjustments: It catches upcoming M365 price increases scheduled for July 1, 2026 (e.g., E3 moving from $36 to $39).
Infrastructure Transitions: It identifies technical shifts scheduled for late 2025 and 2026, such as Slack implementing a two-year rolling retention policy for audit logs and Teams migrating private channel messages to group mailboxes.
Retention Granularity: It identifies that Slack offers channel-level control, whereas Microsoft favors a centralized “longest-period-wins” logic.

Every single claim is returned with an inline citation and a direct link to the specific documentation used for the extraction.

Verifiable Claims

The output is not a block of prose. It is a structured synthesis designed for downstream application consumption. We prioritize high-fidelity data over general summaries.

A Note on AI-Generated Research

LLMs can be remarkably confident even when they are incorrect. While Tabstack Research is built to minimize errors through recursive browsing and cross-referencing, no automated system is perfect. This is why we provide thorough, inline citations for every claim. We believe the value of an AI agent is not in replacing human judgment, but in providing the clear, sourced evidence required for you to make an informed decision.

Grounded Attribution

In an era of black box AI, we have moved the burden of proof from the model to the evidence. Every claim in a Tabstack Research report is backed by an inline citation. If the system reports that Slack Enterprise Grid requires a custom quote for the Discovery API, it provides the specific URL and the text fragment used to verify that fact. This allows your application to provide “click to verify” functionality for your end users.

Conflict Resolution

The system does not just aggregate data. It reconciles it. The loop often encounters conflicting information. For example, one source may claim Slack retention is limited to 90 days while another cites indefinite storage for paid tiers. The system resolves this by verifying the context, such as Free tier visibility limits versus Enterprise Grid storage policies. The result is a reconciled data point that prioritizes the most authoritative source.

Structured Data Primitives

Because we strip the DOM noise and marketing boilerplate during the execution phase, the final synthesis is high-density. You receive clean and actionable data points:

Logic Mapping: You learn that Slack offers granular channel level control while Microsoft favors a centralized “longest period wins” conflict logic.
Temporal Accuracy: You get specific price points like Slack’s ~$15 to $45 per user per month (custom pricing) versus Microsoft 365’s $36 for E3 and $60 for E5 list price, which accounts for upcoming 2026 shifts.
Infrastructure Insights: You receive technical specifics that impact compliance. This includes the fact that Teams video clips and embedded images are retained, but code snippets and voice memos are often excluded from standard retention policies.

By the time the data reaches your application, the heavy lifting of discovery and extraction is complete. You are left with a verified asset that is ready to be stored in a database or presented in a UI.

Continuum of Execution: Choosing Your Speed

Research is not a one-size-fits-all operation. Different use cases require different levels of recursion and computational depth. We offer two primary modes to help you manage the balance between latency and comprehensiveness.

Fast Mode: This is optimized for instant answers and populating UI tooltips. It uses lightweight fetches and high-priority search excerpts to return an answer in 10 to 30 seconds. It is ideal for “what is” questions where the answer is likely available in the primary search result or a single documentation page.
Balanced Mode: This triggers our full agentic loop. The system visits, renders, and analyzes pages deeply. It performs the multi-pass gap evaluation we described earlier. Typically delivering a comprehensive report in 1 to 2 minutes, this mode is designed for complex comparisons and multi-source verification.

Built for Trust

In an era of opaque AI, we have taken a different path. As part of the Mozilla ecosystem, Tabstack is built on a foundation of privacy and responsibility. This is not just a marketing claim. It is a technical constraint on how we handle your data.

Zero Training: We do not train AI models on your research queries or the data we collect for you. Your intellectual property remains yours.
Ephemeral by Design: Your research data is treated as ephemeral. It is used to execute the specific task and is then discarded from our active processing memory.
Full Verifiability: By providing full citations and source metadata, we move the burden of proof from the model to the evidence. This allows you to build applications where the AI can be audited in real time.

What You Can Finally Ask

When you stop worrying about how to get the data, you can finally focus on asking the right questions. By moving the orchestration of web browsing into the infrastructure layer, the scope of what your agents can accomplish expands significantly.

You can build agents that handle:

Competitive Intelligence: Compare enterprise pricing and data retention policies for Slack versus Teams.
Regulatory Compliance: Extract mandatory data residency requirements for healthcare SaaS in the European Union.
Due Diligence: Identify strategic supply chain risks in the latest 10-K filings from NVIDIA.
Strategic Analysis: Summarize the major commercial partnerships announced by Salesforce in the last year.

We take care of the hard parts. We handle the browsing, the parsing, and the noise reduction so you can focus on the reasoning that makes your agent unique.

Ready to build? Sign up to get started and start with 10,000 free credits, or explore the technical implementation in our documentation.

Tabstack: Browsing Infrastructure for AI Agents

Wed, 14 Jan 2026 06:00:00 GMT

The Web Access Challenge for Agentic AI

Every AI engineer eventually hits the same infrastructure wall. You build an agent that reasons perfectly in a sandbox, but the moment you connect it to the open web to perform a complex task, you are forced to stop building AI and start managing the massive overhead of browser orchestration.

What starts as a simple script to fetch a URL quickly spirals into a full-time job. You find yourself managing headless browsers just to handle client-side rendering. You spend hours writing brittle selectors to parse raw HTML, only to watch them break the moment a site updates its layout. Instead of getting clean data, you are stuck debugging a chaotic mess of tags and attributes.

This is the hidden tax of autonomous systems. The web is messy, unpredictable, and designed for human eyes rather than software agents. When teams try to bring browsing infrastructure to production themselves, they end up spending more cycles handling session reliability and patching browser fleets than they do improving their agent’s intelligence.

Suddenly, you are not building AI anymore. You are maintaining a fragile, expensive web layer. We built Tabstack to turn that chaotic infrastructure into a solved problem.

Web Browsing as Infrastructure

Today, we are publicly launching Tabstack, the web execution layer for AI systems.

Tabstack is a developer-focused API that enables AI agents to extract, generate, and automate web content. It acts as a single abstraction layer that collapses the complexity of web automation. Tabstack gives your agents the ability to navigate pages, interact with real user flows, and turn the chaotic web into clean, actionable inputs.

We treat browsing as a distinct infrastructure problem. Developers should not have to manually assemble network layers, proxies, and rendering engines just to reliably interact with a website.

Whether you need to render complex single-page applications or navigate multi-step user workflows, Tabstack handles the heavy lifting of orchestration and reliability. We provide the stability needed to operate at scale so you can focus on building the intelligence that drives your agents.

A Continuum of Execution

Under the hood, Tabstack dynamically routes every request to the most efficient extraction method. It does not just blindly fire up a heavy browser for every request. Instead, it intelligently selects the lightest viable method by starting with raw HTTP fetches and escalating to full browser sessions only when edge cases or complex interactions demand it.

This adaptive approach delivers:

Speed: Process simple pages instantly via lightweight fetches.
Reliability: Handle complex single-page apps (SPAs) with a robust automation engine that manages scrolling, rendering, and wait times.
Resilience: Maintain high success rates with intelligent routing and retry logic, without your team needing to manage the underlying infrastructure.

Turning the Web into Data

Tabstack provides a high-level interface for both understanding and acting on web content. By filtering out the noise before it reaches your model, Tabstack prevents you from burning your budget on parsing raw HTML. We ensure every token in your context window is spent on reasoning, not rendering.

Whether you are feeding a RAG (Retrieval-Augmented Generation) pipeline or building an autonomous agent, we have an endpoint for the job:

/markdown: Instantly convert any URL’s page content into clean Markdown, perfect for feeding into LLM context windows.
/json: Transform unstructured web pages into structured JSON objects using a schema you provide. This is ideal for turning product listings or news articles into database-ready records.
/automate: Instruct Tabstack to navigate, click, type, and handle complex interactions to complete multi-step workflows. This allows your agent to execute tasks just like a human user.

Giving Your Agent Eyes

We designed our API to be dead simple. Just import the SDK and tell Tabstack what to do.

Here is how you can automate a complex task, like researching the top posts on Hacker News, in just a few lines of Python:

import os from tabstack import Tabstack

# Initialize the client
tabs = Tabstack(api_key=os.getenv('TABSTACK_API_KEY'))

# Run an autonomous task
result = await tabs.agent.automate(
    url="https://news.ycombinator.com",
    task="Go through 5 pages of the top posts. For each post, determine the website it is from and group all the posts by website. Return the list of the 10 websites with the most posts."
)

print(result)

In the background, Tabstack’s engine navigates pagination, interprets the content, aggregates the results, and returns the final answer. Your agent stays focused on reasoning while we handle the browsing.

Built for Production

Tabstack is designed for teams building AI systems that need to work reliably beyond a prototype. You can start quickly with a simple API and SDKs, then scale without re-architecting your stack or taking on new operational burden. There is no need to manage browser fleets, network layers, or constantly shifting edge cases.

We are seeing developers use Tabstack to build incredible things that go far beyond simple data retrieval:

Autonomous Research: Agents that “read” the news or documentation and synthesize answers rather than hallucinating from outdated training sets.
Action-Oriented Agents: Systems that can book reservations, check flight statuses, or fill out forms on your behalf.
Live Market Analysis: Tools that navigate e-commerce sites to track pricing and availability in real-time.

Privacy You Can Trust (Backed by Mozilla)

We know that giving an external service access to your web traffic requires trust. Tabstack is built by Mozilla.

In an era where data is often harvested without consent, we are building this capability in a way that reflects Mozilla’s values.

No Training on Your Data: Mozilla does not train AI systems on the data that is collected by Tabstack.
Ephemeral by Default: We practice strict data minimization. Internally, we use the returned data solely to execute the requested task. Customer data is treated as ephemeral.
Secure by Design: We utilize end-to-end TLS and scoped API keys to support the secure handling of sensitive information, giving you control and peace of mind.

We are building a tool for developers to access the web responsibly, rather than a mass data harvester for model training.

Get Started

The era of static, isolated AI is over. It is time to let your agents browse.

Tabstack is now in public early access. We want to see what you can build when the entire web is an API.

Website: https://tabstack.ai
Documentation: https://docs.tabstack.ai
Twitter/X: @tabstack

Let’s see what you build.