⚡ Generation Pipeline & Timings

Prompt Wizard 3.1.0 replaced the synthetic 4-second stepper with a real phase pipeline and added end-to-end timing instrumentation. Since 3.2.0 the same SSE channel also carries streaming model thoughts — see thought-stream.

Phase pipeline

Every /api/ai/text call emits user-scoped SSE events over the existing /api/events channel.

Phase	Meaning
queued	Request accepted; waiting for a concurrency slot (max 2)
generating	omp process spawned; payload includes requested_model, attempt, started_at
parsing	Attempt stdout received; parsing JSON
repairing	Parse failed; attempting local repair (strip fences, extract balanced {…}, drop trailing commas)
rerun	Local repair failed; re-running the full generation (attempt 2)
done	Success; payload includes log_id and total_ms
error	Generation failed after all attempts

• events use SSE named event phase (event: phase\ndata: {...})
• scoped by effective identity, not a private person/tab boundary: anonymous visitors all share public, so other public tabs can receive phase metadata; the display filters by request ID
• the client stepper maps phases to whimsical labels and shows elapsed seconds; its numeric position is not a measured completion percentage
• Architect, Specialist and Formatter are combined system-prompt roles, not three separate model calls

JSON repair

Before re-running a failed JSON parse the server tries a local repair:

• strip markdown code fences and surrounding prose
• extract the outermost balanced {…} block
• remove trailing commas before } or ]

The repair strategy is recorded as none, local, or rerun in llm_timings.

Duplicate JSON instruction skip

The server-appended “Respond with ONLY a single valid JSON object…” suffix is skipped when the composed system prompt already contains a case-insensitive match for valid JSON.

Model configuration

• default since D-479g: host default, with no built-in model pin and no default --model flag
• PWIZ_OMP_TEXT_MODEL remains an optional explicit operator override, unset in production unit/env files
• requested_model records host default or that explicit route; resolved_model records final OMP assistant provider/model metadata, or unknown
• existing timing rows preserve genuinely requested historical route values and have unknown resolution; this does not infer an old provider fallback

Timings table

llm_timings in SQLite: JSON text calls with timing data insert successful rows, and timed OMP failures insert failed rows without a successful llm_logs row. Successful non-JSON/diagram text calls have null timing, and image calls do not insert timing rows. This is not instrumentation of every HTTP request or every possible failure.

Column	Type	Note
id	TEXT PK
log_id	TEXT	FK to llm_logs.id, nullable on error
request_id	TEXT	Client-supplied or server-generated UUID
kind	TEXT	e.g. generation, smoke, mermaid
template_key	TEXT
requested_model	TEXT	host default or explicit requested override; old recorded requests preserved
resolved_model	TEXT	Final OMP-reported provider/model; unknown when unreported or historical
queue_ms	INTEGER	Time waiting for concurrency slot
attempt1_ms	INTEGER	Wall time of first omp spawn
attempt2_ms	INTEGER	Nullable; wall time of retry spawn
repair	TEXT	none / local / rerun
total_ms	INTEGER	End-to-end including queue
ok	INTEGER	1 success, 0 failure
created_at	TEXT	ISO 8601

Admin endpoints

• GET /api/admin/llm-timings?limit=N — returns { rows, summaries } where summaries contain per-kind p50/p95 for total and queue latency (last 500 calls)
• requires admins or pwiz-admins group
• D-169 optional SSO permits signed-in admins/pwiz-admins; anonymous and signed-in non-admin requests still return 403 here and for llm_logs

Admin panel

The Debug Drawer has a new Timings tab (admin-only) showing:

• per-kind latency summary table (p50/p95)
• per-call waterfall bars: queue (amber) · attempt1 (purple) · attempt2 (red)
• repair badge and ok/failed status per row

Diagram decision

When includeMermaid is enabled, diagram generation remains blocking. The result is persisted to prompt_history and saved_prompts as a single document; late-arriving diagram content would desync the stored row from the displayed project. A separate diagram phase label (“Sketching the diagram…”) is shown in the stepper.