Flying with a dead screen

Written from the day’s logs.

The MacBook’s panel stopped working. The machine is fine; you just cannot see it. For a laptop that exists to run a voice pilot in a field, that is nearly the end of the project, so the question was whether Hawk could fly with the laptop as a box and the phone as its window.

The microphone rule

The plan was ordinary: Tailscale, SSH from the phone, a tmux session so a dropped connection does not kill the pilot mid-flight. The first test transcribed my voice fine. The second, inside tmux, heard nothing. Not quiet, nothing: a noise floor of exactly zero.

macOS grants microphone access to the process that launched the tmux server, not to the shell inside it. A server started from a terminal window has the microphone. A server started by anything else, in my case an SSH login, records silence forever, with no error. The fix is to start the tmux server from a real terminal once at home and only attach to it from the phone. I proved it both ways with a two-second recording: zeros under one server, real audio under the other.

The field then took one command from the phone. Took off, three moves of two metres that arrived within a metre, and the switch landed it. First flight of the aircraft from a phone.

The latency was not the voice

Everything about the session felt slow, so I measured it afterwards instead of guessing. Transcription on the laptop: a quarter of a second. From the end of my sentence to the command leaving on the radio: a tenth to half a second. The voice pipeline was already fast.

What was slow was speech. After every order Hawk read it back, then made a model call for a witty line, then synthesised that line, all over the phone’s cellular link, and while Hawk talks the microphone is deliberately closed. So I was giving the next order into a closed mic for five to ten seconds and calling it latency. The field mode now says one short cached sentence per order and nothing else. Forty-five read-back phrases are pre-synthesised into a cache that survives reboots. The quips come back for recordings only.

The wake word got its own fix. The speech model hears “Hawk” as hog, hook, hock and hark often enough that orders were being dropped. All of those are accepted now, and the word can land anywhere in the sentence.

Three things I keep

The session recorder: microphone, every spoken line with its timestamp, and the camera view with detection boxes, all on one clock, so the edit needs no clapper. A one-command launcher so the field is one line. And the lesson that “slow” is a measurement, not a feeling.