Build an AI-agent app: the prompt is the program

Let a Landbot AI Agent drive a rich UI — it decides what to ask, generates the data, and emits [[ui:{json}]] widgets your Core-SDK front-end renders. No fixed flow; the instruction prompt is the application logic.

The conversational-app recipe used a deterministic flow — fixed blocks, and a marker that just named the step. This one inverts that: a single AI Agent block is the application logic. Its instruction prompt decides what to ask, invents the data, and emits widgets as [[ui:{json}]] markers carrying a component + props. The front-end is a dumb renderer. The prompt is the program.

This was verified against a live AI Agent: those markers survive the agent → Core stream intact — raw square brackets, not HTML-escaped, not wrapped, not code-fenced, parseable as JSON. (Square brackets specifically; <ui> angle-bracket tags risk HTML-escaping, which is why the protocol uses [[…]].)

How it works

An agent reply arrives as one text message: a line or two of prose, then each widget on its own line.

Here's where things stand this month.
[[ui:{"component":"stats","props":{"items":[{"label":"MRR","value":"$1.32M","dir":"up"}]}}]]
[[ui:{"component":"replies","props":{"options":[{"label":"By region","value":"Show revenue by region"}]}}]]

Your front-end pulls each [[ui:…]] line out, JSON.parses it, and renders the component. It's the same [[ui:…]] convention as the deterministic recipe — there a bare string named a step; here the payload is a full JSON object the agent generated.

Wiring: plan for launcher-first

A floating "Ask AI" launcher that opens a chat on click is the natural shape here — and it's not cosmetic, it dictates two things you must design around from the start:

Floating "Ask AI" input ──send──▶ open modal · connect Core · send the first message
        │
        ▼
  capture Question block (Landbot's required first block)  ──answers it──▶  AI Agent
        │
        ▼
  agent replies: prose + [[ui:{json}]] widgets  ──▶  your renderer

• Landbot web bots open on an input block — your AI Agent block can't be first, so a Question block precedes it and captures the first input. That means you must ignore bot messages until the user has spoken, or that capture-Question's prompt shows up as a stray bubble. Let the launcher's first message both open the modal and answer that block.
• The agent's cold open is empty — on load it tends to greet without widgets; widgets fire in response to a message. Driving it launcher-first (the first user message kicks it off) sidesteps the empty greeting entirely.

The prompt (the program)

The heart of this recipe is the AI Agent's instructions: a role, a length budget, the output protocol, and a widget catalogue the agent fills in. Paste it into a Landbot AI Agent block.

You are Nova, a sales assistant for Northstar (a SaaS). Qualify the visitor and guide
them to a plan + a booked demo. Warm, concise. Invent plausible specifics.

# Length (critical)
- Keep each reply under ~1800 characters — the platform truncates near 2000 and corrupts
  a widget. One short sentence of prose; let widgets carry the detail. Max 2 widgets/reply.

# Output protocol
Emit each widget as a [[ui:…]] marker wrapping ONE JSON object, on its own line, after your
prose — raw, no code fences, no escaping the brackets:
  [[ui:{"component":"<name>","props":{ ... }}]]
Content between [[ui: and ]] MUST be valid JSON. One marker per line.

# Widgets
- replies (inline chips): {"options":[{"label","value","store"?:{"<field>":"<value>"}}]}
- plans   (panel): {"title","field"?,"plans":[{"name","price","features":[…],"recommended"?,"cta"}]}
- calc    (panel): {"title","team","hours","rate","fields"?:{"team","hours","savings"}}
- form    (panel): {"title","fields":[{"label","field"}],"submit"}

# Rules
- Language: reply (and all widget text) in the visitor's language. Keep field KEYS canonical.
- Capture: any answer worth keeping carries a store/field so it's saved to a Landbot Field.

# Steps
1. Greet and ask what brings them in (replies).  2. Qualify (replies, with store).
3. Recommend a plan (plans).  4. Show value (calc).  5. Book + capture details (form).

That catalogue + protocol is the contract. The agent fills it however the conversation goes — you never wire a flow.

Note Keep the widget set small and the JSON compact. The agent reliably emits 1–2 widgets per turn as valid JSON; pushing more (or long payloads) is what trips the ~2000-character truncation. Put the most important widget first so a truncation only ever drops the least critical one.

Widget schema reference

Landbot doesn't define a widget schema — you do. [[ui:…]] carries whatever JSON you choose, so the exact shape of each widget is a contract you write once and share between the prompt and the renderer. Copy these canonical payloads verbatim into both sides so they can't drift:

Every payload is { "component", "props" } — the renderer switches on component, and props is that component's own shape. (store / field / fields are the capture hints — see Capturing data.) To add a widget, add it in three places in the same commit: a row here, an entry in the prompt's catalogue, and a branch in the renderer. Skip one and you get the silent mismatch below.

The front-end: a dispatch renderer

Same Core wiring as Build a custom chat UI — subscribe before init(), discriminate on author_type, dedupe on key — plus a parser that splits prose from markers and a renderer per component. Skip any marker that won't parse (an LLM will occasionally malform one — never let it crash the UI).

import { Core } from "@landbot/core";

// Pull prose + [[ui:{json}]] payloads out of one agent message.
function parseAgent(body) {
  const prose = [], widgets = [];
  for (const line of ("" + body).split("\n")) {
    const m = line.match(/\[\[ui:(.+)\]\]/);
    if (m) { try { const o = JSON.parse(m[1]); if (o?.component) widgets.push(o); } catch { /* skip malformed */ } }
    else prose.push(line);
  }
  return { prose: prose.join("\n").trim(), widgets };
}

// Route by component: glanceable widgets inline, rich/interactive ones in a side panel.
const PANEL = new Set(["plans", "calc", "form", "calendar", "bar", "line"]);
const wantsPanel = (w) => w.surface ? w.surface === "panel" : PANEL.has(w.component);

function onMessage(msg) {
  if (seen.has(msg.key)) return; seen.add(msg.key);
  if (msg.author_type === "user") return renderUser(msg.message);
  if (msg.author_type === "sys" || msg.type === "hidden") return;
  const { prose, widgets } = parseAgent(msg.message ?? msg.title ?? "");
  if (prose) renderBot(prose);
  for (const w of widgets) (wantsPanel(w) ? renderInPanel : renderInline)(w);
}

const config = await fetch(CONFIG_URL).then(r => r.json());
const core = new Core(config);
core.pipelines.$readableSequence.subscribe(onMessage); // before init()
await core.init();

renderInline / renderInPanel build the actual DOM per component (stats, replies, plans, calc, …) — ordinary UI code you own. A clean layout for this: a centered chat modal that widens to reveal a panel when a panel widget arrives, and collapses back when it's dismissed — glanceable widgets (stats, chips) stay in the transcript.

Warning The widget JSON shape is a contract — the prompt and the renderer must agree, exactly. [[ui:…]] carries any JSON; nothing checks that the agent's payload matches what your renderer reads. So the keys are one schema you define once and share on both sides. This recipe uses { "component": "…", "props": { … } }, so the prompt's widget catalogue and the renderer's switch both speak component / props. If they drift — say the prompt emits { "type": "…", "items": [ … ] } while the renderer expects { component, props } — the marker still parses as valid JSON and then renders nothing, with no error. It's a silent failure. Whenever you change the shape on one side (or add a widget), change the other in the same edit, and keep each component's exact prop names in lockstep. Make drift loud: have your renderer console.warn("unknown widget", w) when a component has no matching renderer, so a mismatch shows up immediately instead of silently rendering nothing.

When a marker won't parse

JSON.parse alone isn't always enough — markers fail for concrete reasons:

• Truncation. The reply is hard-cut near ~2000 characters, which can slice a widget mid-JSON. (Mitigate in the prompt: short prose, ≤2 widgets, most important first.)
• LLM slips. An occasional dropped closing brace, trailing comma, or smart-quote.

So parse defensively — try JSON.parse, attempt a light repair, then give up gracefully (never throw):

function tryParse(raw) {
  try { return JSON.parse(raw); } catch {}
  // best-effort repair: drop trailing commas, then balance braces/brackets
  let s = raw.replace(/,\s*([}\]])/g, "$1");
  const count = (ch) => (s.split(ch).length - 1);
  s += "}".repeat(Math.max(0, count("{") - count("}"))) + "]".repeat(Math.max(0, count("[") - count("]")));
  try { return JSON.parse(s); } catch { return null; } // give up → skip this widget
}

A repaired truncated widget may be missing trailing data, but that beats dropping the whole reply. Use it in parseAgent in place of the bare JSON.parse. Combined with the console.warn on unknown components, you can now see both failure classes: a null from tryParse is a parse failure (truncation/syntax); a warned "unknown widget" is a shape mismatch.

Capturing data into Landbot Fields

The agent doesn't store Fields by talking — but the SDK can, directly from the client. Attaching custom_data to a sendMessage writes those keys to the customer's Landbot Fields (verified against a live bot):

core.sendMessage({ message: "Here are my details", custom_data: { name, email, company } });

So each interactive widget persists its answer by sending the value plus a custom_data map. The agent declares what to store via the store / field hints in the widget JSON; the front-end maps those into custom_data on send. The full loop, for a reply chip:

// agent emitted:
//   [[ui:{"component":"replies","props":{"options":[
//     {"label":"11–50","value":"We're 11–50","store":{"team_size":"11-50"}}]}}]]
chip.onclick = () => core.sendMessage({
  message: option.value,                   // what the user "said" — the agent reads this
  custom_data: option.store || undefined,  // → written to the Landbot Field `team_size`
});

Same handoff for the rest, all driven by what the agent put in the JSON:

• plans → custom_data: { [props.field]: chosenPlanName }
• calc → custom_data: { [props.fields.team]: team, [props.fields.savings]: computed, … }
• form → one key per input's field: custom_data: { email: "…", company: "…" }

The agent names the Field; the front-end writes it. Captured Fields are then available to the Platform API, integrations, and your CRM — no Question blocks required.

Warning Never eval message content. The agent's [[ui:…]] payloads are parsed as JSON only; if your bot also uses Code blocks, those arrive as type:"hidden", action:"script" — ignore them. Executing message-borne strings is a code-injection vector against your page.

Test locally first

Don't debug parsing and rendering against the live bot — it's slow and conflates agent problems with renderer problems. Feed your renderer a mock stream of canonical payloads first; a schema drift shows up in seconds, before you ever connect Core.

// during development, drive onMessage() from a mock instead of the live pipeline
const mockAgent = [
  `Here's the overview.\n[[ui:{"component":"stats","props":{"items":[{"label":"MRR","value":"$1.3M","dir":"up"}]}}]]`,
  `Pick a plan.\n[[ui:{"component":"plans","props":{"title":"Plans","plans":[{"name":"Pro","price":"$49","recommended":true,"features":["Automations"],"cta":"Choose"}]}}]]`,
];
mockAgent.forEach((message) => onMessage({ author_type: "bot", type: "text", key: message, message }));

If every widget renders from the mock, the live agent renders too — provided its output matches the schema. (Which is exactly why the reference table belongs in both your prompt and your renderer.)

Styling & layout

The renderers are yours, but a few conventions keep it coherent:

• Inline vs. panel by weight. Glanceable, low-interaction widgets (stats, replies chips, note) render inline in the transcript; large or interactive ones (plans, calc, form, charts) open the panel. That's what the surface / component map decides.
• Theme with CSS variables (accent, text, surface, line) so one palette drives every widget — the agent never needs to know about styling.
• Accessibility: render choices as real <button> elements (focusable, Enter-activatable), label every input, keep the panel open/close keyboard-reachable, and consider an aria-live region so new bot messages are announced.

Troubleshooting: when a widget doesn't render

Three things can go wrong, in order — diagnose by logging the raw message body first:

core.pipelines.$readableSequence.subscribe((m) => console.log(m.author_type, JSON.stringify(m.message)));

1. Did the agent emit it? No [[ui: in the raw body → a prompt problem. Strengthen the instructions; check it didn't hit the truncation limit.
2. Did it arrive intact? If you see <ui>, back-ticks, or a fenced block, the agent escaped/wrapped it — tell it "raw, no code fences, no escaping." (Square brackets normally survive; this is mostly an <ui>-tag problem.)
3. Did your code handle it? Marker present and clean but nothing shows → a schema mismatch: the payload's keys don't match your renderer. This is the silent one — your console.warn("unknown widget") and the schema table are the guardrails.

Gotchas worth knowing

• You can't read Fields back through Core. Pipeline messages carry no customer object, so verify stored Fields in the dashboard or via the Platform API — not from the stream.
• Truncation is real. Keep replies brief and widgets few (see the length note above).
• The cold-open and input-block-first behaviours are covered under Wiring — they're architectural, not afterthoughts.

Next steps

Swap in your own widget set and agent persona — a data-analyst that renders dashboards, a concierge that books things, a configurator. The dispatch loop never changes; only the prompt's widget catalogue and the renderers do. For the deterministic counterpart (you own the flow, the marker names the step), see Build a conversational app; for the SDK basics, Build a custom chat UI.

Reference: Build a conversational app · Build a custom chat UI · Pipelines · Messages