Install WordPress with the title Van der Berg Painting and build a custom
theme with a home page, services, portfolio and contact.
Since mid September we have built a complete website with the Invoker Assistant every week and written down what happens. Not as a demo with an edited recording, but with a log per instruction: the literal text, the time in seconds, the steps it took, the screenshots it made itself and what went wrong. This is what comes out of that after four weeks.
All the numbers below come from those logs, up to and including 11 October. Nothing is estimated.
Three sites, 1 hour and 33 minutes
| Site | Model | Instructions | Time |
|---|---|---|---|
| Empty domain to a hair salon with online booking | Codex, GPT-6-Astra | 4 | 47 min |
| A site from 2012 rebuilt | Claude Opus | 4 | 26 min |
| Export from a website builder into WordPress | Claude Sonnet | 4 | 19 min |
Together: 12 instructions and 1 hour and 33 minutes of machine time for three working websites. An average of 31 minutes per site. That is faster than the name of the series suggests, and still the average is the least interesting number in this article.
The spread per instruction is enormous
The fastest instruction took 24 seconds. The slowest 25 minutes. That is a factor of 60 within the same kind of work.
The slowest was a WordPress installation. It hit the edges of the sandbox, after which the assistant finished the installation by hand, file by file. The end result was fine; the route there was the worst possible one.
The fastest was reading an existing site. It went that fast because the notes from an earlier attempt were still in place. In the attempt before that, the same instruction took 82 seconds. That is not noise, that is the whole point of per website notes: have your site explored once and you save that every time after.
What the models did
You would expect a clear difference per model. Our numbers do not show one, and that is because the instruction matters more than the model.
| Model | Average per instruction |
|---|---|
| Codex, GPT-6-Astra | 11 min 47 s |
| Claude Opus | 6 min 38 s |
| Claude Sonnet | 4 min 45 s |
The top row was the only one starting from nothing: empty domain, install WordPress, build a theme, build a booking plugin. The bottom two rebuilt an existing site, with the content already on the server. So do not compare the models, compare the tasks. What we do take from it: on long tasks the slowest one spent minutes waiting on its own intermediate steps, and you see that back in every instruction of that run.
What went wrong
This is the part a demo always leaves out, and the most useful part for us.
Eight runs for four sites. Three sites are finished. The fourth ran aground on the very first instruction four times in a row up to 11 October. Those are not small hiccups, those are whole evenings.
One run killed by work that vanished into the background. The assistant handed the theme build off to a background task and reported that it would let us know as soon as it was done. That moment never comes, because the instruction ends with the answer. Instructions 3 and 4 then landed on an empty WordPress. Result: 2 of 4 instructions in 56 minutes. We have since pinned down in the assistant that it starts nothing in the background and finishes before it answers. The second attempt ran through flawlessly in 19 minutes.
Zero errors in the logs. Not one of the successful instructions produced an error or a page error. That sounds good and it is exactly the problem: an instruction that gets stuck does not report that as an error. It reports that it is done. So we now check per run whether the site really does what the instruction asked, instead of whether the log is clean.
Zero approvals, and why that is honest
In every run the number of approvals is zero. That is because these runs use automatic mode: changes go through directly and then appear as a card with the differences.
So it is not proof that the assistant did nothing risky. It is the mode you pick when you want to measure how fast it works. If you work with approval per change, you click roughly as often as files change, and on a theme instruction that is dozens of times. See Approvals in the assistant for the modes.
Two numbers that surprised us
42 screenshots, taken by itself. Across the three successful runs the assistant took 42 screenshots to look at its own work, desktop and mobile. Not because we asked per instruction, but because the instruction said it had to check. That is the cheapest quality control there is.
30 follow up suggestions. Every instruction ends with what it would do next. Across the three runs that was 30 suggestions, and almost all of the same kind: supply real photos, set the right email address, send a test message. That is exactly the list you are left with at a real client too.
Common mistakes
Reading an average as a promise. 31 minutes per site is the average of three runs with four instructions. One installation getting stuck doubles it.
Leaving the check out of the instruction. If you do not ask for a screenshot or a test, you do not get them, and then a visitor is the one who tells you half of it does not work.
Trusting an empty error list. Zero errors means nothing fell over, not that the instruction succeeded. Check the result, not the log.
Comparing models on different tasks. Our own table above invites exactly that, and still you must not read it that way. To compare models, have them do the same thing; that is precisely what a later episode does.
The episodes themselves, with the instructions and the screenshots per site, are on Built in an hour.