Designing a firmware update that can recover from interruption

Define what recovery means

A hypothetical connected device receives firmware updates in the field. Power can fail during download, installation or the first boot of a new image. The product needs a clear recovery contract: what remains bootable, how an incomplete update is detected and how the device reports its state. A successful update under ideal conditions answers only a small part of that question.

This study assumes an authorized update mechanism and sufficient nonvolatile storage for the selected design. It does not prescribe one universal bootloader layout. Single-slot, dual-slot and external-storage approaches have different constraints. The actual memory map, erase geometry, security policy and hardware behaviour determine which design is feasible.

Make the update state machine explicit

Separate download, validation, installation, trial boot and confirmation. For each state, identify the durable record that allows the next boot to interpret what happened. Record which writes are atomic at the software level and which rely on hardware guarantees. Never assume that a partially programmed flash location contains either the complete old value or the complete new value.

The state model should describe repeated interruptions, not only one failure at a convenient point. Consider a device that loses power on every attempted recovery. It should avoid silently marking an untested image as accepted or exhausting a limited retry mechanism without an observable outcome. The product team must decide how recovery interacts with user data and configuration.

Validate the image and its compatibility

Integrity checking detects accidental corruption, while authenticity checking establishes whether an image comes from an authorized source. Compatibility is a separate question: a valid image may still target the wrong board or require a different data format. Define the hardware identifier, image version and dependency checks that belong in the acceptance path.

Key management and rollback policy require their own threat model. Allowing a fallback image for operational recovery does not automatically justify accepting any older signed image. State how the security policy and availability requirement interact. Where a standard bootloader is used, review its documented update modes and limitations against the product’s exact configuration.

Test interruption at meaningful boundaries

Create a fault-injection plan around erase operations, writes, metadata updates and first-boot confirmation. Record the interruption point and the state observed on the next boot. Where hardware permits, include real power interruption tests; a simulated return code does not reproduce every electrical or storage failure.

Check normal operation after recovery, including persistent settings, peripheral initialization and communication. A device that reaches a boot banner but cannot perform its intended function is not fully recovered. Tests should also cover corrupted downloads, incompatible images and unavailable update servers without conflating those cases with loss of power.

Make field diagnosis possible

A recovery event should leave a useful reason code or diagnostic record where the product can retain one safely. Distinguish verification failure, installation interruption, watchdog reset and application health-check failure. Avoid logging secrets or full user data merely to make debugging easier.

Define how support engineers can determine the active image, previous attempt and compatibility state. The mechanism may be a local service interface, a controlled telemetry record or an on-device status indicator. Choose it according to the product’s connectivity and privacy requirements. Observability is part of the recovery design, not an optional detail added after deployment.

A failure-injection worksheet

Create a table whose rows are update transitions and whose columns describe durable state, intended next boot, diagnostic evidence and the test that exercises interruption. Include download completion, image validation, installation metadata and trial-image confirmation. The table should make it possible to predict recovery from each interruption point before running the experiment.

For one row, interrupt the update twice: first during the original attempt, then during recovery. Check whether the design still has an unambiguous way to decide what to boot. Repeat with an image that is authentic but incompatible with the board. These are different failures and should produce distinguishable outcomes. A single generic update failed flag may not provide enough information for diagnosis.

Review persistent application data separately from executable images. A new firmware version may change a configuration format, making an otherwise valid rollback unable to interpret the data. Decide whether migration is reversible, versioned or deferred until the new image is accepted. Keep this decision in the release policy so that recovery testing covers the complete product state. The worksheet turns broad claims about safe updates into specific conditions that can be implemented and tested.

What would be delivered

The illustrative package contains a state diagram, storage assumptions, update and recovery code, target test procedures and a matrix of observed outcomes. Release notes identify compatible hardware and data migrations. The acceptance review should distinguish tests executed on a host from tests executed on the actual device.

No field reliability or security certification is claimed here. A real project would need evidence from its chosen hardware and boot chain, plus review of the security and operational requirements. The practical value of this example is the decision sequence: specify recovery, implement durable state, challenge the transitions and make the resulting behaviour diagnosable.

What to bring. What to review.

Questions before you begin.