Five questions behind a meaningful coverage review

What requirement does this coverage answer?

Coverage is useful when it helps determine whether a verification plan has exercised the behaviour the product requires. A code-coverage metric and a functional-coverage model answer different questions. Neither automatically proves that a design is correct or that the specification itself is complete.

Start the review from requirements and planned scenarios. For each important behaviour, identify the stimulus, checker and coverage observation. A covered event without a checker can show that something happened without establishing that it happened correctly. Conversely, a strong assertion may provide evidence that is not summarized by the headline coverage percentage.

Which configuration produced the result?

Record the design revision, parameters, testbench version, tool setup and included test runs. If results from several configurations are merged, explain what the combined view means. A feature covered in one configuration may remain untested in another configuration that changes its behaviour.

Keep the merging process reproducible and exclude stale or invalid runs deliberately. A test that terminated before completing its intended checks should not be treated as equivalent to a completed test simply because it wrote coverage data. The report should distinguish execution status from the observations collected during execution.

Why are bins missing?

A missing coverage item can indicate inadequate stimulus, an unreachable condition, a faulty coverage model or a specification inconsistency. Investigate representative examples before deciding to add more random tests. A targeted sequence is often a better next experiment than increasing the regression count without a hypothesis.

For crosses, review whether all combinations are meaningful and legal. A large cross-product can create noise that distracts from the behaviours that matter. Simplify or constrain the model based on the specification, and document the reasoning. Do not remove a difficult item merely because it is inconvenient to close.

Are exclusions justified and reviewable?

Exclusions should identify the exact item, the reason, supporting evidence and the review decision. Reasons may include an unsupported configuration, a structurally unreachable state or functionality outside the current scope. Those are different explanations and should not collapse into one generic waiver category.

Revisit exclusions when the design, parameter set or specification changes. An exclusion that was valid for one revision can hide a new gap later. Keep the exclusion list under the same revision discipline as the test plan so that a future reviewer can reconstruct the meaning of the final result.

What remains beyond coverage?

A coverage review should be read alongside failures, assertions, formal results where used, structural analysis and integration evidence. A single percentage does not capture protocol correctness, analog behaviour, physical timing or the quality of the reference model. State those boundaries clearly.

The practical output is a closure discussion: which requirements have evidence, which limitations are accepted and which questions remain open. The decision belongs to the project’s acceptance process. This brief does not prescribe a universal coverage threshold or claim that any current Avecas design has reached one.

An example closure discussion

Suppose an error-response scenario has not been covered. First ask whether the environment can generate the triggering condition and whether the design is expected to support it. Next inspect the checker and coverage sampling point. The event may occur but be sampled at the wrong time, or the stimulus may prevent it through an unintended constraint. Each possibility calls for a different correction.

If the scenario is truly unreachable, record the architectural reason and the evidence supporting that conclusion. If it is reachable but unsupported, decide whether the specification or implementation must change. If it is required and simply untested, create a focused experiment with a predicted outcome. This sequence keeps coverage closure tied to engineering questions rather than to manipulating a number.

At the next review, compare the new evidence with the original question. Did the experiment establish the intended behaviour, or did it merely hit the coverage point? Retain any newly discovered requirement ambiguity and update the plan. A disciplined closure discussion can improve both the test environment and the product specification, even when the numerical change is small. The record should explain that improvement in concrete terms that another reviewer can follow.

Use a repeatable review agenda

Begin with the scope and configuration, then inspect missing or excluded items that affect important requirements. Review the status of failing tests before celebrating a metric. For each open question, agree the next experiment and the evidence that would resolve it. End by recording the decision and the conditions under which it must be revisited.

This creates a more useful conversation than comparing two percentages in isolation. It also helps a new engineer understand why a result was accepted, which assumptions were involved and what additional work is required when the design is reused in a different system.

Keep the review record close to the verification plan. A later engineer should be able to understand why an exclusion exists without searching private messages. When a new feature is added, use that record to decide which coverage assumptions need to be reopened and which remain valid.

What to bring. What to review.

Questions before you begin.