Volume 16 Beginner 5 sub-modules ~30 min read

Writing Reliable Embedded C

Firmware runs for years with nobody watching it, and gets no second chance. This volume is about the difference between code that works and code that keeps working: knowing which parts of C have no defined meaning, checking what you cannot trust, and failing in a way somebody can diagnose afterwards.

You will learn
  • What undefined behaviour is, and what the optimiser does with it
  • How to make it report itself, with a sanitiser and with real diagnostics
  • The MISRA C ideas worth following even when the rules do not apply to you
  • How to write assertions that suit a chip, and what not to put inside one
  • Why a watchdog kicked from a timer interrupt protects nothing
  • Three shapes of error handling, and the one that survives a growing project
You need

16.1 Undefined behaviour, catalogued

Undefined behaviour is not "something unpredictable happens". It is the standard giving no meaning to a construct, which lets the compiler assume you never write it.

That second half is the part that surprises people. If a program can only reach a line by doing something undefined, the compiler may decide the line is unreachable and delete it.

The overflow check that is not one


bool grows_signed(int x)
{
    return x + 1 > x;        /* did adding one make it bigger? */
}

bool grows_unsigned(unsigned x)
{
    return x + 1u > x;       /* the same question, unsigned */
}

The first can only be false if x + 1 overflows. Signed overflow is undefined, so it cannot be false, so it is true. Here is what gcc -O2 produced:


endbr64
movl    $1, %eax
ret

endbr64
cmpl    $-1, %edi
setne   %al
ret

The signed version never looks at x. It returns 1. The unsigned version does real work, because unsigned arithmetic wraps by definition and the answer genuinely can be false.

It gets more pointed with the check people actually write:


bool safe_to_add_signed(int a, int b)
{
    return a + b >= a;
}

endbr64
movl    %esi, %eax
notl    %eax
shrl    $31, %eax
ret

Read that assembly: it takes b, inverts it, and shifts the sign bit down. It computes b >= 0. The compiler removed the addition entirely, because the only way the sum could be smaller is by overflowing, which cannot happen. The check no longer checks anything.


bool safe_to_add_checked(int a, int b)
{
    if (b >= 0) {
        return a <= INT_MAX - b;
    }
    return a >= INT_MIN - b;
}
Remember

Never test for overflow by overflowing. Ask whether the operation would overflow, using subtraction from the limit, which is always defined.

The catalogue

These are the ones that reach real firmware.

What Example Usual symptom
Signed overflow INT_MAX + 1 a check optimised away
Shifting too far 1u << 33 works on one chip, not another
Out-of-bounds access table[4] in int table[4] corruption somewhere unrelated
Dividing by zero n / d with d zero a fault, or a wrong answer
Reading uninitialised memory a local never assigned works at -O0, fails at -Os
Using a pointer after free Volume 14's demo works until it does not
Breaking alignment a uint32_t * into odd bytes slow, or a fault
Reading a union member not last written Volume 08 usually works, is not guaranteed
volatile missing on shared data Volume 10 writes deleted, loops never end

Make it report itself

A compiler can insert checks. Build the test firmware with a sanitiser and undefined behaviour stops being silent.


vol16_ub.c:24:13: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
vol16_ub.c:32:45: runtime error: shift exponent 33 is too large for 32-bit type 'unsigned int'
vol16_ub.c:39:48: runtime error: index 4 out of bounds for type 'int [4]'
vol16_ub.c:39:9: runtime error: load of address 0xADDRESS with insufficient space for an object of type 'int'
vol16_ub.c:46:44: runtime error: division by zero

Each one names the file, the line and the column. Without the sanitiser the program printed numbers and carried on, which is the whole danger: undefined behaviour usually looks like it worked.

Where this fits in an embedded project

The sanitiser needs a runtime, so it will not fit on a small chip. What it is for is the part of your firmware that can be built and run on a desktop: protocol parsing, maths, state machines, buffer handling. Keep that code free of hardware dependencies and you can run it under the sanitiser, with tests, on every build.

Quick check

Why did a + b >= a compile to a test of b >= 0?

Show the answer

Answer: B. The compiler is entitled to assume no undefined behaviour occurs. Given that, a + b >= a is exactly b >= 0, and that is what it generated.

16.2 MISRA C ideas in plain words

MISRA C is a list of rules for C in safety-related software. Most of them are one idea: stay away from the parts of C whose behaviour is surprising or undefined.

You may never work on a project that requires it. The reasoning is still worth having, because the rules were written by people who had already had the bugs.

The ideas worth taking

  1. Use the fixed-width types. uint16_t rather than int, so the size is the same everywhere
  2. One exit from a loop, and no fall-through in a switch. Both make the control flow readable at a glance
  3. Always brace a body. if (x) do_thing(); is one edit away from a bug
  4. Do not mix signed and unsigned in a comparison. The signed value is converted, and a negative number becomes enormous
  5. Give every switch on an enum a default. Volume 12 used it to reach a safe state
  6. No dynamic memory after start-up. Volume 14 gave the reasons in detail
  7. No recursion. The stack depth stops being something you can calculate
  8. Check every return value, or cast it to (void) deliberately. Then ignoring one is a decision rather than an oversight

The signed and unsigned trap, since it catches everyone


for (unsigned i = 10u; i >= 0u; i--) {      /* i is never negative */
    use(i);
}

An unsigned is never less than zero, so the condition is always true. When i is 0 and the loop decrements it, it wraps to 4294967295 and carries on.


int  a = -1;
unsigned b = 1u;

if (a < b) { /* not taken */ }              /* -1 converts to 4294967295 */
Common mistake

Turning on -Wsign-compare and then suppressing the warnings with casts. A cast silences the message and keeps the bug. Change the types so the comparison makes sense instead.

The compiler flags that encode most of this

-Wall -Wextra -Wconversion -Wsign-conversion -Wshadow -Wundef -Werror. Every C program in this course is built with them. That is why several of the bugs in these volumes had to be written carefully, to survive the compiler long enough to be demonstrated.

Quick check

Which of these is not a reason MISRA discourages recursion?

Show the answer

Answer: D. Recursion needs no heap; it uses the stack. The objection is entirely about not being able to prove how deep the stack will get, which on a chip with a few kilobytes matters.

16.3 Defensive coding and assertions

Defensive coding checks what other people might get wrong. An assertion checks what you believe cannot be wrong. They are different tools and they belong in different places.


status_t set_speed(int rpm)
{
    if (rpm < 0 || rpm > MAX_RPM) {
        return ERR_BAD_ARG;
    }
    /* ... */
}

static void write_pin(uint8_t pin)
{
    ASSERT(pin < 16u);
    /* a private function, and
       every caller is in this
       file */
}

The rule of thumb: if a value comes from outside your code - a caller you do not control, a sensor, a message, a user - handle it. If it comes from your own code and being wrong means you have a bug, assert it.

An assertion that suits a chip

The standard assert prints to stderr and calls abort. A microcontroller has neither. So embedded projects write their own, which records where it happened and then goes somewhere safe.


static void assert_failed(const char *file, unsigned line)
{
    last_failure.file  = file;      /* in RAM the start-up code does not clear */
    last_failure.line  = line;
    last_failure.valid = true;
    /* on a real chip: stop the motor, open the relay, then reset */
}

#define ASSERT(c) \
    do { if (!(c)) { assert_failed(__FILE__, __LINE__); } } while (0)

1. an assertion that holds, and one that does not
   speed 40 <= 100: nothing happened, safe state entered 0 times
   speed 140 <= 100: failed at vol16_assert.c line 59
   safe state entered 1 time

Storing it somewhere that survives a reset is the whole point. A board that resets and can then tell you which line failed is diagnosable. One that just resets is not.

The trap: work inside the assertion

In a release build the macro usually expands to nothing. Everything inside the brackets goes with it.


2. the trap: work inside the assertion
   debug build:   init_hardware called 1 time
   release build: init_hardware called 0 times
   the hardware is never initialised in the build you ship

bool ok = init_hardware();      /* happens in every build */
ASSERT(ok);                     /* only the check disappears */
(void)ok;                       /* so it is not unused when ASSERT is empty */
Common mistake

Asserting on anything the outside world decides. ASSERT(temperature < 100) turns a hot day into a crash. A sensor reading out of range is a condition to handle, and possibly to report, but it is not evidence of a bug in your code.

Should assertions stay in the release build

There are two schools, and the answer depends on what failing costs.

Leaving them in means the device stops and records the fault rather than carrying on with a state your code believes is impossible. For anything that can hurt somebody, that is clearly right: a motor controller in an unknown state should stop.

Taking them out saves flash and cycles, and avoids a device that resets in the field over something that might have been survivable. For a consumer device with no safety implication, that is defensible.

The middle way is common: keep them, but make the failure handler proportionate. Record the file and line, put the hardware in a safe state, and then reset, rather than sitting in a loop forever.

Quick check

ASSERT(queue_push(&q, item)); - what is wrong with it?

Show the answer

Answer: A. The macro expands to nothing when assertions are disabled, taking the call with it. Do the work on its own line and assert about the result it returned.

16.4 Watchdogs

A watchdog resets the chip unless the program keeps telling it not to. It only helps if the telling proves the work is happening.

The hardware is simple: a counter that counts down, and resets the chip if it reaches zero. Your code writes a value to it periodically - kicking it - to start it again.

The question is where the kick goes, and there is one answer that looks convenient and is worthless.

A watchdog kicked from a timer interrupt compared with one kicked only when every task has reported timer interrupt every task reports control task hangs still kicked, so no reset ever comes RESET one watchdog period later the kick has to prove the work happened
Figure 16.1 - Each upward tick is a kick. The control task stops at the dashed line. Above, the timer interrupt keeps kicking regardless, so the watchdog never fires and the dead board stays powered. Below, the kicks stop with the work, and the watchdog resets the board one period later.

The version that protects nothing

Put the kick in a timer interrupt and it is neat, regular, and impossible to forget. It also keeps running perfectly while the rest of the firmware is dead.


1. kicked from a timer interrupt
   control task ran 200 times and stopped at 2000 ms
   watchdog resets in 5000 ms: 0
   the interrupt is still running, so the watchdog is happy
   the board is doing nothing useful and will never recover

Three seconds with the control loop dead, and not one reset. The watchdog was doing exactly what it was told, and what it was told had nothing to do with whether the firmware worked.

The version that works

Each task reports that it has run. The kick happens only when all of them have reported since the last one.


if (sensor_task())  { sensor_ok  = true; }
if (control_task()) { control_ok = true; }
if (display_task()) { display_ok = true; }

if (sensor_ok && control_ok && display_ok) {
    watchdog_kick();
    sensor_ok = control_ok = display_ok = false;
}

2. kicked only when every task has reported
   control task ran 200 times and stopped at 2000 ms
   watchdog fired at 2990 ms, 990 ms after the hang
   the board resets and comes back working

Within one watchdog period of the hang, the board resets and recovers.

Remember

The kick must be downstream of the work. If anything can kick the watchdog without the work having happened, the watchdog is decoration.

Choosing the timeout

Long enough that the slowest legitimate pass of the loop never trips it. Short enough that a hang is noticed before it matters.

  1. Measure the worst-case loop time, including the longest interrupt storm
  2. Multiply by a comfortable factor, two or three
  3. Check the result against how long the device may safely misbehave
  4. If those two numbers do not overlap, the design needs changing, not the timeout
What a watchdog is not

It is not an excuse for a bug. A device that recovers by resetting every few hours is still broken, and a reset loses state, loses data, and takes time to come back.

It is also not protection against a fault that happens before it is started, or against one that resets the board into the same failure immediately. Record the reset reason - most chips have a register saying whether the last reset came from power-on, the watchdog, or a brown-out - and count them somewhere that survives.

Quick check

A watchdog is kicked inside the main loop, which also calls a blocking flash_erase() that takes up to 2 seconds. The timeout is 1 second. What happens?

Show the answer

Answer: C. Nothing kicks it for two seconds, so it fires. The answer is either a longer timeout, or a kick inside the erase loop - and the second one weakens the protection, which is the trade being made.

16.5 Error-handling patterns

C has no exceptions, so a function that can fail must say so in its return value, and every caller must look. The shape you pick decides whether that stays true as the project grows.

Shape one: a special value


int read_temperature(void)
{
    if (failed) {
        return -1000;       /* "error" */
    }
    return 235;             /* 23.5 degrees */
}

1. a sentinel value in the return
   working:  read_temperature_sentinel() = 235
   failing:  read_temperature_sentinel() = -1000
   -1000 means an error, and also means -100.0 degrees
   a caller that forgets to check gets a number, not a warning

Two problems. The sentinel is a legal reading, so a caller cannot always tell. And there is only one of it, so "timed out" and "checksum wrong" look identical.

Shape two: a status code, and the value through a pointer


status_t read_temperature(int16_t *out)
{
    if (out == NULL) {
        return ERR_BAD_ARG;
    }
    if (failed) {
        return ERR_TIMEOUT;     /* *out is left alone */
    }
    *out = 235;
    return OK;
}

2. a status code, with the value through a pointer
   working:  status OK, value 235
   failing:  status ERR_TIMEOUT, value untouched
   every reading is a valid temperature, and the status is separate
   a null pointer is caught too: ERR_BAD_ARG

The whole range of the value is available, the reason for failing is specific, and adding a new failure reason does not change any existing code.

One enum for the whole project

Define the error codes once, in one header, and use them everywhere. A project where each module invents its own conventions - zero for success here, non-zero there, a boolean somewhere else - spends its life converting between them, and getting it wrong.

Shape three: several things that can fail, and things to undo

This is where C's lack of exceptions is felt, and where the one legitimate use of goto lives.


static status_t send_message(const uint8_t *msg, unsigned len)
{
    status_t st;

    if (msg == NULL || len == 0u) {
        return ERR_BAD_ARG;         /* nothing acquired yet */
    }

    st = step_power_up();
    if (st != OK) {
        return st;
    }
    power_on = true;                /* record only after it succeeded */

    st = step_open_port();
    if (st != OK) {
        goto release_power;
    }
    port_open = true;

    st = step_send();
    if (st != OK) {
        goto release_port;
    }

    st = step_read_reply();

release_port:
    if (port_open) { close_port(); port_open = false; }
release_power:
    if (power_on)  { power_down(); power_on  = false; }
    return st;
}

The labels are in reverse order of acquisition, so each one undoes exactly what was acquired above it. Jumping to the right label releases everything that was taken and nothing that was not.


the port will not open
   power up        ok
   open port       failed
   power down      done
   result: ERR_NOT_READY, power off, port closed

the reply never comes
   power up        ok
   open port       ok
   send bytes      ok
   read reply      failed
   close port      done
   power down      done
   result: ERR_CHECKSUM, power off, port closed

Notice the failed open. It does not close a port that never opened, because the flag is set only after the step succeeds. That ordering is the part people get wrong.

Common mistake

Using goto for anything else. This one pattern - jumping forward to cleanup labels in one function - is readable and widely accepted, including in the Linux kernel. Jumping backwards, or between unrelated parts of a function, is not.

Making the compiler insist

GCC and Clang can mark a function so that ignoring its result is a warning:


__attribute__((warn_unused_result))
status_t send_message(const uint8_t *msg, unsigned len);

Now send_message(buf, n); on its own line produces a warning, and with -Werror it fails the build. A caller that genuinely does not care writes (void)send_message(buf, n);, which is a decision recorded in the source rather than an oversight.

Wrap it in a macro so the code still builds on a compiler that does not support it, and apply it to anything where ignoring a failure would be serious.

Quick check

Why does the cleanup code set port_open = true after the open succeeds rather than before?

Show the answer

Answer: B. The flag means "this was acquired and must be released". Setting it before the attempt makes it mean "this was attempted", and the cleanup then releases something that does not exist.

What you learned

Practice

Practice 1

Find the undefined behaviour in each, and say what a compiler might do with it.

for (int i = 0; i < n; i++) sum += a[i]; where n may exceed the array length / uint8_t x = 200; uint8_t y = 100; uint8_t z = x + y; / int shift_by(int v, int n) { return v << n; }

Show the solution

The loop. If n exceeds the array length, the reads past the end are undefined. In practice they read whatever is next in memory, which is usually another variable. So the sum is wrong in a way that changes whenever anything else in the file changes. The fix is to pass the array length and check it, or to use a type that carries its own length.

The addition. This one is not undefined, and it is still a bug. Both operands are promoted to int, so the sum really is 300, and assigning it to a uint8_t truncates to 44. Defined behaviour, surprising result. -Wconversion warns about it.

The shift. If n is negative, or greater than or equal to the width of int, the behaviour is undefined. If v is negative it is undefined too, in C before C23. A compiler is free to emit whatever the hardware's shift instruction does. On many machines, shifting a 32-bit value by 33 shifts by 1, because only the low five bits of the count are used. So it appears to work on one chip and does something else on another.

Practice 2

A colleague adds ASSERT(spi_write(cmd) == OK); to a driver. The debug build works and the release build never talks to the device. Explain, and rewrite it.

Show the solution

In the release build ASSERT expands to nothing, and the whole expression goes with it, including the call to spi_write. The command is never sent.


status_t st = spi_write(cmd);
ASSERT(st == OK);
(void)st;

The (void)st stops it being an unused variable when ASSERT is empty, which would otherwise be a warning in exactly the build nobody compiles while developing.

There is a second question worth asking: should this be an assertion at all? An SPI write failing is a hardware condition, not a bug in the code. It probably deserves real handling, which means returning an error up to a caller who can retry or report it. Assertions are for things that cannot happen if the code is right.

Practice 3

Design the watchdog strategy for a device with three demands on it. A 10 ms control loop. A display refreshed every 500 ms. And a firmware update mode where a flash erase can block for 3 seconds.

Show the solution

Normal running. Both tasks report in, and the kick happens only when both have reported since the last kick. The display is the slow one at 500 ms, so the watchdog period has to exceed that with margin. Two seconds is reasonable, and a hang is then caught within two seconds.

If catching a control-loop hang within 2 seconds is too slow, do not shorten the period - split the supervision. Kick on the control task alone with a short period, and have the control task itself check that the display task has run recently, reporting a fault if not.

Update mode. The 3-second erase exceeds any sensible period, and this is the case where a special arrangement is justified:

  1. Erase in blocks rather than as one operation, kicking between blocks
  2. Or lengthen the watchdog period for the duration of the update, and put it back afterwards
  3. Or stop the watchdog during the update, accepting that the device is unprotected while it runs

The first is best where the hardware allows it, because protection is never given up. The third is the most common and the most dangerous, since a hang during a firmware update leaves a device that cannot boot.

Whichever is chosen, record the reset reason and count watchdog resets somewhere that survives, so a device that is resetting regularly can be identified rather than quietly limping.

Practice 4

Rewrite this so that a failure cannot be ignored and nothing leaks.


void configure(void)
{
    open_bus();
    write_config(0x40);
    set_mode(FAST);
    close_bus();
}
Show the solution

Three calls that can fail, none of them checked, and a close_bus that is skipped if anything returns early once checks are added.


status_t configure(void)
{
    status_t st = open_bus();
    if (st != OK) {
        return st;                  /* nothing to release */
    }

    st = write_config(0x40u);
    if (st != OK) {
        goto release_bus;
    }

    st = set_mode(FAST);

release_bus:
    {
        status_t close_st = close_bus();
        if (st == OK) {
            st = close_st;          /* do not hide an earlier failure */
        }
    }
    return st;
}

Two details worth arguing for. The function now returns a status, so the caller can react. And the result of close_bus is only used when nothing has failed yet, because the first failure is the interesting one and overwriting it loses the diagnosis.

Marking the declaration warn_unused_result makes the compiler insist that callers look.

Practice 5

Your firmware resets every few hours in the field and never on the bench. You have a watchdog and a reset-reason register. What would you add to find it?

Show the solution

Make the reset tell you something. On start-up, read the reset reason and store it in a RAM section the start-up code does not clear, along with a counter per reason. Report those over the existing interface, or keep them for whoever reads the device next.

Record where the firmware was. A few bytes in that same uncleared section, written as the main loop moves between phases. Which task last started, the tick count, the last state of the main state machine. After a watchdog reset, those bytes say what it was doing when it stopped.

Capture the fault handlers. A hard fault handler that records the stacked program counter tells you the instruction that faulted. That is often the whole answer.

Add the high-water marks. Stack and buffer high-water marks, as Volumes 09 and 11 measured them, report themselves alongside the reset counts. A stack that reaches its limit after several hours is a common cause of exactly this symptom.

One thread runs through all of it. A reset destroys the evidence, so the work is to write a little of it down beforehand, somewhere the reset does not erase.

Interview corner

Interview question 1

Undefined behaviour

"What is undefined behaviour, and why does it matter more with optimisation on?"

Show the solution

"It is a construct the standard deliberately gives no meaning to - signed overflow, shifting past the width of a type, reading out of bounds. The part that matters is not that the result is unpredictable; it is that the compiler is entitled to assume it never happens, and to optimise on that assumption.

The example I would give is an overflow check. a + b >= a looks like a test for overflow. At -O2 gcc compiled it to a test of b >= 0 and dropped the addition, because the sum can only be smaller if it overflowed, and that cannot happen. The check no longer checks anything, and the code looks correct.

At -O0 the compiler does less of this, which is why code with undefined behaviour often works until somebody turns optimisation on. That is also why I treat 'it only works at -O0' as a bug in my code, not in the compiler."

Interview question 2

assert versus error handling

"When would you use an assertion rather than returning an error?"

Show the solution

"An assertion is for something that cannot be false unless my own code is wrong. A pin index in range inside a private function. A state machine in a state it knows about. An invariant at the top of a function whose callers are all in the same file.

An error return is for anything the outside world decides: a sensor reading, a message from another device, an argument from a caller I do not control. Those are conditions to handle, not evidence of a bug.

Two practical points. Never put work inside the assertion, because the release build removes the whole expression, not just the check - I have seen an entire hardware initialisation disappear that way. And on a chip, make the failure handler record the file and line somewhere a reset does not clear, then go to a safe state. An assertion that resets with no trace is barely better than a crash."

Interview question 3

Watchdogs

"How should a watchdog be used, and what is the common mistake?"

Show the solution

"The common mistake is kicking it from a timer interrupt. It is convenient and regular, and it proves nothing about whether the firmware is working. I ran exactly that: the control task stopped, and the watchdog did not fire once in the next three seconds, because the interrupt was still running.

The kick has to be downstream of the work. Each task sets a flag when it has completed a pass, and the watchdog is kicked only when every flag is set, after which they are cleared. Then a hung task stops the kicks within one period.

For the timeout I would measure the worst-case pass of the loop, including the heaviest interrupt load, and allow a factor of two or three. Then I would check that against how long the device can safely misbehave. And I would record the reset reason and count watchdog resets in memory that survives, because a device quietly resetting once an hour is still broken."

Interview question 4

Error handling in C

"How do you handle errors in C without exceptions?"

Show the solution

"A status code as the return value, and any actual result through a pointer parameter. That keeps the whole range of the result available, so there is no sentinel that is also a legal value. It also lets me distinguish a timeout from a checksum failure, rather than having one catch-all error.

I define the codes once for the whole project, because modules that each invent their own convention spend their lives converting between them.

For functions that acquire several things, I use the cleanup-label pattern. Acquire, record that it succeeded, and on failure jump forward to the label that releases everything taken so far, in reverse order. That is the one use of goto I am comfortable with, and it is what the Linux kernel does.

I would also mark functions warn_unused_result where ignoring a failure would be serious, so that ignoring one has to be written down as (void) rather than happening by accident."

Next, Volume 17 is about the part nobody teaches: what to do when it does not work. Reading a hard fault, using a debugger properly, printf that does not lie, and finding bugs that only happen on the real board.

Key words from this volume

Every word below has a plain-English entry in the glossary.