Point-and-edit for an AI builder

• Click to pick
• live coding mode
• in-chat cards
• one hover primitive
• fixes from real sessions

The problem

An AI builder is a chat next to a live preview of your own site. Most people using it are not developers. They can see what is wrong ("this button is too close to the edge") but they cannot name the CSS selector, and describing a spot on a page in words is slow and vague.

Developers solve this with DevTools: click the element, change a value, watch it update. I wanted the same feel for people who have never opened DevTools, with the AI doing the part that writes real code.

Pick an element, then tweak it

I shipped the bridge between the chat app and the preview iframe first, on its own and unused, so the larger changes after it stayed easy to review. Then came the panel. Click any element in the preview and an anchored panel opens with padding, colours, borders and size. Changes apply to the preview straight away. Apply sends the changes with the next turn, and the coding agent writes them into the theme source.

Live coding mode

With live coding mode on, the agent sends small structured edit operations instead of editing files. The relay pulls them out of the stream and the preview applies them at once: no file write, no rebuild, no reload. When the user is happy, Make it real turns the whole session into a normal build turn that writes the code.

user picks / asks ──▶ agent ──edit ops──▶ preview (in memory)
                                              │
                                        "Make it real"
                                              ▼
                                   normal build turn ──▶ theme files

Live edits live in memory, so every navigation could lose work. Reload, navigation, tab close and in-app route changes are all guarded, but only while edits are pending. Discarding them got its own deliberate button.

In-chat cards

Three flows start in the preview and end in the chat: pointing at elements, sending style changes, and making a live session real. Two left no trace in the transcript. The third led with its least useful part.

I gave them cards stored on the user's own message, in an existing metadata field nothing was using, so no migration was needed. The layout came from looking at real cards. People point at things and describe what they want, and changing a CSS value is the rarer path. So the note leads the card and the style changes are the optional extra. "Made it real" now records what was promoted, so the transcript says what actually happened.

One hover primitive

The builder explained itself through many native title tooltips. For a dense toolbar that is close to the worst choice: a slow delay you cannot tune, OS styling that ignores the design system, one plain line, nothing on touch, and you cannot move the pointer into it to read a longer note.

I replaced them with one hover primitive and one provider at the page root, so hints are grouped: once one is open, its neighbours open instantly as you scan a row. Disabled buttons are wrapped so their hints still show, and a locked control can say why it is locked.

Fixes driven by real sessions

The best UX fixes came from watching real sessions, not from guessing.

• Long turns looked like hangs. Real sessions showed that turns of several minutes were normal, not a rare tail, and the progress strip only updated between turns. It now shows an elapsed clock and a step count. (Keeping long turns alive is its own story: durable agent turns.)
• Failed tool calls looked like disasters. They showed as red "Error" pills, but an agent probing and missing is routine. They are grey now, and the label carries the difference: "Executed" versus "Completed".
• The chat fought the reader. It scrolled to the bottom on every token, even while you were reading above. It now follows the stream only when you are already at the bottom, with a "Jump to latest" pill.
• Point-and-edit was silently dead on some pages. Pages that render a full document and exit early never printed the picker script. It now loads at the very end of every request.

Each preview belongs to one person, which keeps this predictable when several people share a project: see one workspace, many editors.

What I'd do differently

• Build a first-run tour with the feature. Point-and-edit is the feature people most often fail to discover, because it does not announce itself. A short tour should arrive with it, not after.
• Finish migrations in one go. The last few native tooltips outlived the sweep. A half-done migration leaves two systems to maintain.

Takeaway

Let people point instead of describe, show the change instantly, and let the AI write the real code afterwards. Then watch real sessions, because what users struggle with is rarely what you expect.