Volume 11 Beginner 5 sub-modules ~25 min read

Safe and Fault-Tolerant State Machines

Every machine so far has assumed that its state register only ever holds legal codes. Real hardware breaks that promise: a timing slip, a badly released reset or a particle from space can flip a bit. This volume shows what happens then, how to make every machine recover, why the synthesis tool may quietly remove your recovery logic, and how to build machines that correct the error by themselves.

You will learn
  • What illegal states are, and how a machine can end up in one
  • Recovery strategies: back to reset, to a safe state, or correct the bits
  • Why synthesis may delete recovery logic, and the settings that keep it
  • What single-event upsets are, and who must worry about them
  • Detecting and correcting bit flips with parity, Hamming codes and TMR
You need
  • Volume 05: state encoding and Hamming distance
  • Volume 04: case statements and default branches

11.1 Illegal and unreachable states

A state register can hold more codes than the machine has states. The extra codes are illegal states - and in real hardware, a machine can land in one.

More codes than states

A machine with 5 states in binary needs 3 flip-flops, and 3 flip-flops can hold 8 codes. So 3 codes belong to no state. In one-hot, the same 5 states need 5 flip-flops, which can hold 32 codes - so 27 codes belong to no state. These codes are illegal states.

Encoding States Flip-flops Codes Illegal codes
Binary 5 3 8 3
One-hot 5 5 32 27
Binary 4 2 4 0

A related idea from Volume 10 is the unreachable state: a real state that no input sequence reaches from reset. Both kinds are never visited in normal work. The difference is that an unreachable state is part of your design, while an illegal code is not a state at all.

How does a machine get there?

A correct design never steps into an illegal code by itself. But real hardware is not always correct:

What happens then?

That depends on logic nobody designed. In Volume 03, the unused codes were marked "don't care", so their next states are whatever made the gates smallest. There are three possible outcomes:

The last two are called lock-up. The machine stops doing its job until someone resets it - and in a car or a satellite, there may be nobody to do that.

Two real examples

Take the Moore edge detector built from gates in Volume 03, with the unused code 11. Put in 11 and follow its formulas. Q1+ = Q0·b + Q1·b becomes b, and Q0+ = Q1'·Q0'·b becomes 0. So it steps to 00 or 10 - both legal. It recovers. But pulse = Q0, and Q0 is 1 in code 11, so while it is there it sends out a false pulse. Recovering is not the same as behaving.

The ring counter from Volume 07 is worse. If a glitch sets a second bit, both 1s go round and round for ever:

A ring counter that gains a second hot bit and locks up 0 1 2 3 4 5 6 7 8 9 clk ring 0001 0010 0100 1000 0001 0101 1010 0101 1010 0101 1010
Figure 11.1 - A 4-bit ring counter. In the middle of cycle 4, a fault sets a second bit. From then on, two 1s rotate for ever, alternating between 0101 and 1010. The counter never returns to its four legal codes.
Common mistake

"The machine can never get into that code, so I do not care what happens there." For a hobby project, perhaps. For anything that must keep working - a car, a medical pump, a network switch running for years - you must know what every code does.

Quick check

A one-hot machine has 6 states. How many illegal codes can its state register hold?

Show the answer

Answer: B. 6 flip-flops can hold 26 = 64 codes. Only 6 of them have exactly one bit set, so the other 64 - 6 = 58 are illegal - including 000000 and every pattern with two or more 1s.

11.2 Recovery strategies

A safe state machine has a defined, tested way back from every illegal code - usually to its reset state, sometimes to a special safe state, sometimes by correcting the bits.

Three strategies

Strategy What it does Good for
Back to reset any illegal code leads to the reset state most control machines
To a safe state leads to a state whose outputs cannot cause harm, and raises an error machines that control dangerous things
Correct the bits extra bits let the machine work out the right state machines that must not even pause - sub-module 11.5

A safe state depends on the job. For a traffic light it is all lights red. For a motor controller it is motor off. For a serial receiver it is "drop this byte, go back to waiting".

Back to reset: the default branch

In Verilog, the recovery is one line: a default branch in the case statement, which catches every code not listed above it.


module safe_ctrl (
  input  wire clk, rst, go, fin,
  output wire busy, err
);
  localparam [1:0] IDLE = 2'b00, RUN = 2'b01, DONE = 2'b10;   // 2'b11 is illegal
  reg [1:0] state, next_state;

  always @(posedge clk)
    if (rst) state <= IDLE;
    else     state <= next_state;

  always @(*) begin
    next_state = state;
    case (state)
      IDLE:    if (go)  next_state = RUN;
      RUN:     if (fin) next_state = DONE;
      DONE:             next_state = IDLE;
      default:          next_state = IDLE;          // any illegal code: go home
    endcase
  end

  assign busy = (state == RUN);
  assign err  = (state != IDLE) && (state != RUN) && (state != DONE);   // an illegal code now
endmodule
A safe controller recovering from a fault that puts the illegal code 11 into its state register 0 1 2 3 4 5 6 clk go state IDLE RUN RUN 11 IDLE RUN err
Figure 11.2 - In the middle of cycle 2, a fault writes the illegal code 11 into the state register. The err output rises at once. At the next clock edge, the default branch sends the machine back to IDLE. A new go starts it again as normal.

One-hot machines need their own check

A one-hot machine written bit by bit, as in Volume 05, has no case statement, so there is no default branch to catch anything. Instead, check the pattern itself: it is legal only if exactly one bit is 1.


// s holds one bit per state; exactly one must be 1
wire illegal = (s == 5'b00000) || ((s & (s - 5'd1)) != 5'b00000);

always @(posedge clk)
  if (rst || illegal) s <= 5'b00001;        // back to IDLE on any illegal pattern
  else                s <= s_next;          // the normal one-hot next-state logic

The trick s & (s - 1) removes the lowest 1 from s. Take s = 00110: then s - 1 = 00101, and s & (s - 1) = 00100, which is not zero, so there were two or more 1s. For a legal code such as 00100, s - 1 = 00011, and the AND gives 00000.

Test it: inject a fault

A recovery path that has never run is a recovery path you hope works. In a testbench, write an illegal code straight into the design, and check what happens next. This is called fault injection:


@(negedge clk) dut.state = 2'b11;                     // a "particle hit" on the state
@(negedge clk)
  if (dut.state === 2'b00) $display("PASS: back in IDLE");
  else                     $display("FAIL: no recovery, state = %b", dut.state);

dut.state reaches inside the design by name - a testbench may do that, although a design should not. Try every illegal code, in every state, with every input value.

Common mistake

Recovering to the reset state is not always safe. If the machine was half way through a job - say, halfway through sending a byte - going back to IDLE may leave another block waiting for ever. Think about what the rest of the system sees, and raise an error flag so that it can react.

Quick check

In safe_ctrl.v, what does the machine do in the cycle after its state register is hit and holds 11?

Show the answer

Answer: C. 11 matches none of IDLE, RUN or DONE, so the case statement takes the default branch, and next_state is IDLE. At the next clock edge the machine is back in IDLE, and err, which was 1 while it sat in 11, drops to 0.

11.3 Safe-FSM synthesis settings

A default branch in your code is not enough on its own. The synthesis tool may prove that illegal codes never happen, and delete the recovery logic - unless you tell it to keep it.

The logic you wrote may not be the logic you get

Remember FSM extraction from Volume 05: the synthesis tool recognises your state machine, works out its states and arrows, and often re-encodes it. While doing that, it can also prove something about your code: starting from reset, the machine never reaches an illegal code. From the tool's point of view, the default branch can never run. So the logic for it looks useless, and it may be removed to make the design smaller and faster.

The tool is right about your code. It is wrong about your hardware, which can be hit by the faults in sub-module 11.1. The tool has no idea that particles exist.

Telling the tool to keep it

Every major synthesis tool has a setting for this.

Setting Where What it does
default: branch your code describes the recovery - but may be removed by FSM extraction
fsm_safe_state attribute Vivado builds recovery logic even for codes it proves are unreachable
"safe state machine" option many other tools the same idea under another name - check the tool's guide
fsm_encoding = "none" Vivado turns off FSM extraction for that machine, so your code is kept as written

In Vivado, the attribute sits on the state register, like the fsm_encoding attribute you met in Volume 05:


(* fsm_safe_state = "reset_state" *) reg [1:0] state;   // recover to the reset state

Its settings include going back to the reset state, and going to the state named in your default branch. One setting even adds extra bits, so that a single flipped bit is corrected automatically. The exact names are in the Vivado synthesis guide.

Beware of promises in the code

Some directives promise the tool that no other value can appear, and so give it permission to drop recovery logic. The old full_case directive does this, and so can SystemVerilog's unique case (Volume 04). Avoid them in machines that must be safe. The safe FSM section of the Verilog course shows the danger in detail.

Check the result, not the code

RTL simulation always runs the code you wrote - including the default branch - even if synthesis threw it away. So a passing RTL fault-injection test proves nothing about the chip. To be sure:

  1. Read the synthesis report. It lists the state machines found, their encoding, and whether safe logic was added.
  2. Run the fault-injection test on the netlist - the gate-level design that synthesis produced - not only on your RTL.
Common mistake

"My simulation shows the machine recovering, so it is safe." Simulation of your RTL always shows the default branch working. Only the report and a gate-level test show whether that branch survived synthesis.

Quick check

Your RTL simulation shows the machine recovering from an illegal code. Why might the real chip still lock up?

Show the answer

Answer: A. RTL simulation runs the source code, where the default branch exists. FSM extraction may have proved the illegal codes unreachable and deleted that branch from the hardware. Only the synthesis report or a gate-level simulation shows what really got built.

11.4 Radiation and single-event upsets

A particle from space, or from the chip's own materials, can flip a bit in a flip-flop. Machines that must be safe have to expect random bit flips.

Bits that flip by themselves

Particles pass through everything, all the time. High-energy particles from space strike the atmosphere and produce showers of neutrons, some of which reach the ground. Tiny traces of radioactive material in a chip's own packaging give off alpha particles. When such a particle passes through a transistor, it leaves behind a trail of electric charge. If that charge lands on the node that stores a flip-flop's value, it can flip it: a 0 becomes a 1.

This is a single-event upset, or SEU. Nothing is damaged - the next write puts a correct value back - so it is also called a soft error. But until then, the bit is wrong. In a state register, a wrong bit means a wrong state - perhaps an illegal one.

Think of it like this

Think of a light switch in a busy corridor. Most of the time nobody touches it. But once in a long while, someone brushes past and knocks it. Nothing is broken - you can switch it back - but for a while the light is in the wrong position, and anything that depends on it is wrong too.

How often?

For a single flip-flop at sea level, an upset is very rare. But a chip holds millions of flip-flops and memory bits, and a product may run for many years, so over a whole fleet upsets do happen. The rate also rises sharply with height. Where aircraft fly, the flow of neutrons is hundreds of times stronger than at sea level. In space, there is no atmosphere to shield the chip at all.

Who must worry?

Field Why
Cars millions of vehicles, each running for years; safety standards such as ISO 26262
Aircraft and space far more particles at height, and no one to press reset
Medical devices a wrong state could harm a patient
Networks and data centres huge numbers of chips running without a break

All of these fall under functional safety: engineering that keeps a system safe even when parts of it fail.

Going deeper: FPGAs and their configuration memory

Most FPGAs hold their circuit in configuration memory built from the same kind of cells as ordinary memory. An upset there does not just flip a stored bit - it can change the circuit itself, for example cutting a wire. FPGAs used in harsh places therefore check and rewrite their configuration memory continuously while they run, a process called scrubbing.

Common mistake

"Upsets only matter in space." They matter anywhere that enough chips run for long enough, or where a single failure is unacceptable - including cars and hospitals at sea level.

Quick check

What is a soft error?

Show the answer

Answer: D. A soft error, such as an upset from a particle strike, changes a stored value without damaging the hardware. The flip-flop works perfectly afterwards; it simply holds the wrong value until it is written again.

11.5 Protecting the state register (Hamming, TMR)

Extra bits can let a state register notice a flipped bit, or even correct it. Or three copies of the register can simply outvote a flip in any one of them.

Detect: a parity bit

Recall the Hamming distance from Volume 05: the number of bits in which two codes differ. If every legal code differs from every other in at least two bits, then flipping any single bit always lands on an illegal code. The flip cannot turn one legal state into another - so it can always be caught.

The simplest way to get there is a parity bit: an extra bit chosen so that every legal code has an even number of 1s. Here is the drink machine from Volume 03 with one:

State Code Code with parity bit Number of 1s
C0 00 000 0
C1 01 011 2
C2 11 110 2
VEND 10 101 2

Any single flip makes the number of 1s odd, which the machine checks with one XOR of all three bits, ^state in Verilog. On an odd count, it goes to a safe state. One-hot codes have the same property for free: any two legal one-hot codes differ in exactly two bits.

Correct: a distance-3 code

Push the distance up to three, and a single flip lands on a code that is still nearest to the original. It is one bit away from the original, and at least two bits away from every other legal code. The machine can then work out which state it should be in, and carry on as if nothing happened. This is the idea behind a Hamming code. For four states:

State Code
S0 00000
S1 01011
S2 10101
S3 11110

Every pair of these codes differs in at least three bits. The next-state logic treats every code within one bit of S1 as S1, and so on. This costs five flip-flops instead of two, and more logic - the price of never stopping.

Outvote: triple modular redundancy

The bluntest method keeps three copies of the state register and takes a vote. This is triple modular redundancy, or TMR. A majority voter outputs, for each bit, the value that at least two of the three copies agree on. A flip in any one copy is simply outvoted.

A state machine protected by triple modular redundancy: one next-state logic block feeding three copies of the state register and a majority voter Next-state logic Majority voter copy A copy B copy C in voted state the voted state feeds back, so every copy is rewritten each cycle
Figure 11.3 - TMR. The next-state logic feeds three copies of the state register. A majority voter picks, bit by bit, the value that at least two copies agree on. The voted state is used everywhere, including the next-state logic, so every copy is rewritten with the correct state at each edge.

(* dont_touch = "true" *) reg [1:0] s_a, s_b, s_c;          // three copies - keep all three
wire [1:0] state = (s_a & s_b) | (s_b & s_c) | (s_a & s_c);  // bit-by-bit majority vote

always @(posedge clk)
  if (rst) begin s_a <= IDLE; s_b <= IDLE; s_c <= IDLE; end
  else     begin s_a <= next_state; s_b <= next_state; s_c <= next_state; end

// the next-state and output logic use the voted "state", as usual

Two details make it work:

TMR at work: one copy of the state register is flipped, the vote ignores it, and the next clock edge repairs it 0 1 2 3 4 5 6 7 clk copy_a 00 01 10 00 copy_b 00 01 01 11 01 10 00 copy_c 00 01 10 00 voted 00 01 10 00
Figure 11.4 - In the middle of cycle 2, a particle flips a bit in copy B: 01 becomes 11. Copies A and C still hold 01, so the voted state never changes. At the next edge, all three copies are rewritten, and copy B is correct again.

Choosing the protection

Method Extra cost A single flipped bit is Machine keeps running?
Default branch almost none caught only if it lands on an illegal code after a trip to reset
Parity bit one flip-flop, one XOR always detected after a trip to a safe state
Distance-3 code several flip-flops, more logic corrected yes
TMR three times the flip-flops, plus voters outvoted and repaired yes

Space hardware often goes further, triplicating the logic and the voters as well, because the voter itself could be hit.

Common mistake

Protecting the state register but not the outputs. If an output is decoded by gates from the voted state, a particle hitting those gates can still cause a glitch. For the most critical outputs, register them - three times, if need be - after the voter.

Quick check

In a TMR state register, a particle flips a bit in copy C. What happens?

Show the answer

Answer: B. The voter outputs the value at least two copies agree on, so a single bad copy changes nothing. The voted state drives the next-state logic, which writes the same correct next state into all three copies at the next clock edge, repairing copy C.

What you learned

Key words from this volume

Every word below has a plain-English entry in the glossary.

Practice

Practice 1

Count and protect

A controller has 6 states in binary. How many illegal codes does it have? If you add a parity bit, how many flip-flops does it use, and how many codes are illegal then?

Show the solution

6 states need 3 flip-flops, so 8 codes: 2 illegal. With a parity bit it uses 4 flip-flops, so 16 codes. Only codes with an even number of 1s can be legal, and 6 of those are used. So 16 - 6 = 10 codes are illegal, and every single flip from a legal code lands on one of them.

Practice 2

Find the lock-up

A 3-bit Johnson counter steps 000, 001, 011, 111, 110, 100 and back to 000. It shifts in the opposite of its top bit. What happens if a fault puts it into 010, one of its two unused codes?

Show the solution

Follow the rule, shifting in the opposite of the top bit: 010 has top bit 0, so shift in 1, giving 101. 101 has top bit 1, so shift in 0, giving 010. The counter now circles between 010 and 101 for ever - lock-up, with the two unused codes forming their own little loop. A safe version detects 010 and 101 and forces the counter back to 000.

Practice 3

Vote it

The three copies of a TMR state register hold 10, 11 and 10. What is the voted state? Which copy was hit?

Show the solution

Vote bit by bit. The left bit is 1 in all three copies, so it is 1. The right bit is 0, 1, 0 - two copies say 0, so it is 0. The voted state is 10, and the second copy was hit.

Interview corner

Interview question 1

Make it safe

"How do you make a state machine safe against illegal states?"

Show the solution

"First, make every code of the state register go somewhere defined: a default branch in the case statement for binary machines, and a legality check for one-hot. Choose where it goes by what is safe for the system - usually reset, sometimes a safe state with an error flag. Second, make sure synthesis keeps it: use the tool's safe-state setting, avoid full_case and unique case in that machine, and check the synthesis report. Third, prove it with fault injection on the gate-level netlist, not only on RTL. For high-reliability designs, protect the state itself - parity to detect upsets, a Hamming code or TMR to correct them."

Interview question 2

Why not just use TMR everywhere?

"TMR corrects upsets. Why not use it for every register in the chip?"

Show the solution

"Cost. TMR more than triples the area and power of what it protects, and adds a voter delay to every path. So designers protect what matters most: the state registers of controllers, where one flipped bit can wreck the whole system, and key configuration registers. Data paths are usually protected differently - with error-correcting codes on memories, and checks such as CRCs on data that moves around."

Next, Volume 12 turns from designing machines to proving them right: tests that visit every state and every arrow, coverage, and assertions.