Halyard
A voice-first field agent for onshore wind technicians — guided procedure execution and diagnostic reasoning with no hands, no eyes, and no connectivity required.
OutcomeSpecified a fixed-phrase confirmation pattern for safety-critical steps that makes an ambiguous "yeah" structurally unable to advance a lockout sequence.
The Challenge
A wind turbine technician spends the working day ninety metres up, inside a nacelle the size of a shipping container, wearing a fall-arrest harness, hearing protection, and gloves rated for electrical work. The machinery is loud. Their hands are on a torque wrench. And yet everything they need — the torque spec for this turbine variant, the isolation checklist, the fault history that would tell them another technician found the same problem four months ago — is on a tablet they cannot safely reach.
The result is a workflow built on interruption: stop, de-glove, retrieve the device, find the information, re-glove, resume. Fourteen times per turbine visit, on average, costing roughly two and a half minutes each — and often skipped entirely, with the technician working from memory instead. My client was building a voice-first agent to remove that interruption. I led research, conversation design, and product design from the acoustic field study through build-ready specification.
- Contextual Inquiry
- Acoustic Field Study
- Conversation Design
- Dialog State Modeling
- Multimodal Wireframes
What I Designed
Three functions, in priority order — documentation is deliberately third, a byproduct of the first two rather than a task the technician performs. Guided procedure execution reads torque specs, sequences, and safety steps aloud at the technician's pace, confirming critical steps explicitly and never advancing on a timer. Diagnostic reasoning lets the technician describe symptoms and hear likely causes ranked by probability, informed by fleet history — including what a specific previous technician found on this specific machine, with attribution spoken aloud.
Underneath both sits a register model that shifts from conversational to terse the moment the interaction crosses a safety boundary — sentences shorten, structure becomes numbered, and the confirmation requirement changes from any casual acknowledgement to one fixed phrase. The shift is itself a signal to the technician that the stakes have changed, backed by a two-tone earcon at the boundary and a 0.95 confidence floor with no margin for interpretation.
Screens
Six moments from a single turbine visit — orientation, retrieval, a safety gate, diagnosis, repair, and close-out. Each shows the ruggedized handheld's ambient screen — the passive glanceable state a technician would actually see, hands busy, mid-task. The full dialog transcript, UX annotations, and modality-split rationale for each screen are one click away.
Tower base, 06:38. Ewan has arrived at Turbine 14 and woken Halyard. The system binds to the work order, verifies cache currency, and reads the context aloud. This is the only moment in the visit where he is likely to look at the screen before climbing.
Nacelle, 07:51. Ewan is at step 7 of the gearbox procedure with a torque wrench in hand. He asks a three-word question that contains no component, no variant, and no specification type — and gets a fully qualified answer, because the context stack supplies all three.
08:14. Ewan needs to open the cooler panel, which requires electrical isolation. This is the screen and the dialog pattern that justify the product's existence. Register has shifted to terse. Only the fixed phrase advances the sequence.
08:32. Isolation complete, panel open, cooler fan confirmed faulty. Ewan wants to know whether this has happened before. Halyard surfaces L3 field knowledge — a previous technician's finding, with attribution — and the register relaxes back to conversational.
T2 · constrained choice → failed
T3 · modality shift → active
09:06. The turbine has been released to idle for a running check and ambient has risen to 81 dB. Recognition is failing. Halyard escalates through three repair tiers — never repeating a strategy, never apologizing — and lands on the screen when voice cannot carry it.
10:24, tower base. Ewan has descended. The report has been assembling itself for three hours and forty-six minutes. He reviews a draft, not a blank form — and signs on screen, because a signature is the one interaction that must not be producible by a misrecognized utterance.
Outcome
- Delivered a build-ready conversation and product design system — dialog states, confidence thresholds, repair strategies, and six multimodal screens — handed to speech and platform engineering.
- Specified the fixed-phrase confirmation pattern as an architectural safety control, not a UX preference — the design artifact Thomas needed to approve the pilot.
- Designed the three-layer organizational learning model (lexicon, protocol, field knowledge) with governance built in at the point of contribution, not bolted on after.
- Set the product's own bar: fewer than 3 device interruptions per visit (from a baseline of 14), and over 96% of safety steps explicitly confirmed — the two numbers that matter most to field adoption.