Automating C test cases for embedded system verification

Dave Kelf, Breker Verification Systems
embedded.com (April 28, 2020)

As system-on-chip (SoC) designs proceed on their march to greater complexity, test suites containing thousands of lines of code for system-level verification continue to be written by hand, a quaintly old school and ineffective practice defying the adage “automate whenever possible.” This is especially true for C tests that run on an SoC’s embedded processors to verify the entire device prior to fabrication.

Automating verification test composition where possible has been shown to increase productivity for many phases of SoC development. Constrained Random techniques, for example, in a Universal Verification Methodology (UVM) testbench, make use of randomized test vectors directed at specific scenarios to increase coverage. While these have increased verification efficiency at the hardware block level, the design is still perceived as a black box with stimulus, checks and coverage code written separately, still an onerous and error-prone task for large blocks.

It is hard to extend this methodology to the system level, given the need to combine processor test code with I/O transactions, often executed on an emulator or prototyping system. To properly verify an SoC, the processors themselves must be exercised. UVM and other constrained-random approaches do not account for code running on the processors. In fact, to use UVM on an SoC, the processors are often removed and replaced by virtual inputs and outputs onto the SoC bus allowing the sub-system minus the processor to be verified.

SoC verification engineers recognize the limitations of constrained-random testbenches, driving them to handwrite C tests to run on the processors for both simulation and hardware emulation, even though they are limited in fully exercising the SoC design. The performance of these verification platforms is not good enough to run a full operating system (OS), so these tests execute “bare-metal,” which adds a significant overhead to composition effort. It is unusual for handwritten tests, especially without the aid of OS services, to run in a coordinated way across multi-core processors leveraging multiple threads. The result is that aspects of SoC behavior, such as concurrent operations and coherency, are minimally verified.

Click here to read more ...