Fielding an LLM-controlled PX4 drone

Everything here happened to my aircraft between 11 and 26 October 2026 and is in the flight logs.

Connecting a language model to a flight controller takes an afternoon. Flying it takes weeks, and almost none of the time goes where you expect. This is the list I wish I had read first. The setup is a Holybro X500 V2 with a Pixhawk 6C running PX4, a SiK 915 MHz telemetry pair, a Mac running Whisper, a grammar, a validator and Claude, and a voice called Hawk.

1. The safety layer you wrote is not the safety layer you tested

On the 13th the aircraft climbed a metre off the ground on the way down, hard, and I hit kill. The log said: my altitude top-up retried on a timer, the retry fired after the pilot had selected Land, and the call it used switched the aircraft back to Hold before repositioning. The validator checked every command’s arguments and never checked who was in control.

Fix: a latch. The executor watches the flight mode continuously. Any mode it did not command means the pilot has it: every routine aborts, nothing further is sent. Then reproduce the incident in the simulator before trusting it.

2. A new voice order must not clear that latch

My first version cleared the latch on any new order, reasoning that a new order is new intent. Thirteen days later the log showed: pilot selects Land, 0.4 s later a voice “move left” executes. In a field with people talking, orders arrive whether you meant them or not. The latch clears when the aircraft disarms. Nothing else.

3. Lost acknowledgements are normal on a noisy radio

A 915 MHz SiK link in a city sits under the cellular band. Commands arrive and their acknowledgements do not. MAVSDK reports a timeout and your code thinks the command failed while the aircraft is already doing it. Judge by telemetry: after an arm or takeoff command, watch the armed flag and the landed state for eight seconds. If nothing happened, disarm. Also turn on error correction in the radios and keep the phone two metres from the dongle.

4. The estimator’s height reference is not the estimator’s height

I moved the height reference to the barometer after the climb. The next session the aircraft sank onto the grass believing it was two metres up. GPS altitude was still fused, and at PX4’s default noise values a GPS altitude at half a metre of claimed error outweighs a barometer at three and a half. The reference parameter chooses the origin; the fusion bitmask chooses the data. Drop GPS altitude from the fusion and the barometer is finally in charge.

5. Multipath is not a parameter problem

Next to buildings, thirty satellites and a claimed thirty-centimetre accuracy still give you a position that wanders two metres in twenty seconds and a preflight check that refuses to arm. No parameter fixes reflected signals. Fly on open ground, or add a downward optical-flow sensor with a rangefinder, which is how consumer drones hover still.

6. Your latency is probably in your own queue

I measured every stage after a slow session. Transcription 0.1 s. Spoken order to command on the radio: 0.15 s median. The moves that felt slow waited twelve seconds behind the takeoff routine, because PX4 levels off thirty centimetres short and my top-up reposition never closed that gap and burned its full timeout. Make routines pre-emptable by the next order and only correct real shortfalls.

7. The microphone is closed while the aircraft is talking

If the pilot speaks while a model-generated quip is being synthesised and played, you are talking into a muted microphone for ten seconds. In the field, one short cached read-back per order, no model round trip. Pre-synthesise every phrase the grammar can produce. The conversational layer is for the video.

8. Telemetry heading lags; never compute from it mid-manoeuvre

PX4 faces the waypoint on a reposition unless you give it a yaw, so sideways moves swing the nose and the camera. Holding “the current heading” from telemetry made it worse: mid-turn, the current heading is stale, and the move re-commanded the old one. Hold the heading you last commanded.

9. The wake word will be misheard, so accept the mishearings

Whisper renders “Hawk” as hog, hook, hock, hark and hulk. Accept aliases and one-edit variants, let the word land anywhere in the sentence, and keep a short attention window after an order so follow-ups can skip it. Hold and stop never require it. And never let the pilot read a Python exception aloud.

10. Phones are bad access points, and one phone is worse

The camera streams to the laptop over a phone hotspot. A phone cannot host a hotspot and talk to the camera’s own Wi-Fi at the same time, so the camera app on the hotspot phone cannot start the stream. A USB cable from the phone to the laptop adds a second interface on the same subnet and stalls the TCP stream after seconds. Two phones, no cable, hotspot phone on a box, aircraft on a stand during setup because a Wi-Fi antenna at grass level is the worst link there is.

11. macOS gives the microphone to whoever started the tmux server

Run the pilot in tmux so a dropped SSH session does not kill it mid-flight. Start that tmux server from a terminal window, not from an SSH login, or the microphone returns exact zeros forever with no error. And on a Mac, opening a second input stream on the same device stalls the first; one stream, with taps.

12. A forgotten session records the room

The recorder and the transcriber do not know the flight ended. Twice I found hours of room audio and transcribed conversations in the logs. Idle the pilot after ten minutes on the ground, keep recordings out of version control, and read the transcript log before you push anything anywhere.

The part that does not change

Kill is for a person about to be hit. Land on the switch is for everything else, and in every override in my logs it has been perfect. Three kills from height have cost a battery tray and a landing leg. The aircraft was fine every time I used the mode switch instead.