Creating SoC Integration Tests with Portable Stimulus and UVM Register Models

By Matthew Ballance - Mentor, A Siemens Business

OVERVIEW

Writing and reading registers is the primary way that the behavior of most IPs is controlled and queried. As a consequence of how fundamental registers are to the correct operation of designs, register tests are a seemingly-simple but important aspect of design verification and bring-up. At IP level, the correct implementation of registers must be verified – that they are accessible from the interfaces on the IP block and that they have the correct reset levels. At subsystem level, verifying access to registers helps to confirm that the interconnect network and address decode have been implemented as per the spec. At SoC level, verifying access to registers confirms that the processor byte order matches the interconnect implementation, and that the boot code properly configures the memory management unit (MMU) such that IP registers are visible to the processor.

In this article, we will explore how portable stimulus, via Accellera’s Portable Stimulus Standard (PSS), can leverage information captured in a register model to automate creation of block, subsystem, and SoC register-access tests.

UVM BUILT-IN REGISTER TESTS

UVM provides a register model for modeling the register space – the available registers, fields, and their attributes such as address, accessibility, and reset value. The UVM library also provides built-in directed sequences that iterate through the UVM register model and check the reset value of registers and confirm registers are accessible by modifying and checking all register fields that are read/write. These built-in tests are incredibly useful at the IP level to confirm that registers inside an individual IP block have been implemented correctly.

Attempting to reuse these test sequences at the subsystem level becomes difficult, however, because of the number of registers present in a subsystem and the fact that the built-in UVM sequences are directed sequences that expect to test every register in the design. Attempting to reuse these built-in test sequences at SoC level is also difficult because the built-in sequences are self-checking sequences implemented in SystemVerilog. In order to run on the embedded processor of an SoC, we need to have bare-metal C or assembly code.

By modeling our test intent with portable stimulus, we will gain the flexibility to partition our test space into multiple smaller tests. We will also gain the flexibility to target SoC level tests by creating embedded software tests in C or assembly.

TEST INTENT AND TEST REALIZATION

Portable stimulus separates two elements of a test that are most commonly merged in a directed test. Test intent is the high-level design of what to test. Test realization is the mechanism by which that test is carried out.

In the case of testing a register model, our test intent looks something like the following:

• Select a register from the register block
• Select a read/write field from the register
• Select a bit within that field to test

The test intent described above is independent of whether we are verifying a block, subsystem, or SoC level design. It is also independent of whether we are targeting a SystemVerilog environment or an embedded-software environment.

Neglecting proper constraints for a moment, our test intent for testing a single register is captured by the PSS code above:

• We have a reg_id field to capture the register we are testing
• We have a flip_bit field to capture which bit within the register we wish to test
• We have a reg_addr field to capture the address of the field in the memory maps

Now, without test realization to connect this test intent to a specific test environment, our test intent is pretty worthless. Ideally, we can design our test realization with portable stimulus in mind. Doing so allows us to design a common API that will be available in all environments. For example, we might specify that a method named testbit will be available in all environments, and that this method will test the ability to modify the specified bit. The function prototype for such a function is shown below.

In a SystemVerilog IP- or subsystem-level environment, we could implement this method as a task within a UVM sequence. The SystemVerilog task shown below uses a memory-access API to read and write memory, and implements the register bit-test operation by:

• Reading the current value of the register
• Negating the target bit of the register, and writing the new register value
• Reading back the register address, and checking whether the bit retains its value

In the example shown above, we will continue to run if an error is encountered. This is pretty typical of UVM tests. However, an embedded software environment has different constraints.

The example above shows a possible implementation of the testbit function for an embedded processor. As you can see, the approach is very similar to the SystemVerilog version, though the specifics are different. One of the biggest differences is that, in this case, we assume if a register bit test fails, we will end the entire test.

Separating test intent and test realization is a core element of portable stimulus, and is key to enabling tests intent to easily be retargeted across environments.

CREATING A SUBSYSTEM REGISTER MODEL

Now that we’ve looked at an approach to capturing test intent and test realization for register testing with PSS, let’s take a brief look at how our UVM register model comes into existence. For the purposes of this example, let’s look at the very simple subsystem in figure 1. This subsystem has a processor, two Ethernet controllers, and two DMA engines.

A UVM register model was created for the Ethernet and DMA controllers when they were verified at the block level. These UVM register models could have been created by hand, but more likely they were created by using a register-creation tool as shown in figure 2.

Register-creation tools accept register specifications in a variety of standard (IP-XACT, SystemRDL) and non-standard (e.g., CSV) formats and generate various views of that register specification. Generating an RTL implementation of the register model saves time on design implementation. Generating a UVM register model shortens the time to bring-up a testbench. In all cases, ensuring everything is aligned with the high-level register specification saves significant time and wasted effort!

When we reach subsystem level, a register-creation tool may help us to assemble a register model for the entire subsystem. Or, we may simply take the individual block-level register models and assemble the subsystem-level register model. The code below assembles a subsystem-level register model from the Ethernet and DMA controller register models. As you can see, not much code is required.

• The available registers in the design, and their addresses
• The fields within those registers and any access restrictions

What might be surprising is just how quickly the number of registers accumulate. By the time we’ve created a register model for just two Ethernet controllers and two DMA controllers, we have 989 registers with testable fields. Just think how many registers a full SoC contains!

CREATING A REGISTER-TEST PSS MODEL

Now that we’ve captured a subsystem- or SoC-level register model, how do we proceed to create register-access tests? First, we need to create a PSS model of the register-access test intent. Then, we need to connect that test intent to specific verification environments with test realization.

Creating Core Test Intent

Previously, we looked at the core test intent for register-access testing: select a register, and bit within that register to test. This high-level test intent must, of course, be constrained based on the registers in the design that is being tested. The good news is that we have already captured all the information that is needed to generate these constraints in our register model.

In some cases, our register-creation tool may be able to create PSS register tests directly as one of its outputs. If it does not, then one approach to automating creation of our test intent is to run some SystemVerilog code that iterates through the UVM register model and writes out our PSS register-test intent. The code below shows a UVM test that calls a class named regmodel2pss to create the PSS test intent.

The result is a portable stimulus description that captures register-test intent for our subsystem register map. The code shown below is the first portion of the PSS component and action created to test our register model.

Our action (whose name is derived from the register-block name) declares three rand fields. The reg_id field contains an ID for the register being targeted, and will range between 0 and the number of registers minus one. The flip_bit field specifies the register bit to be tested. This field, as well as the register address, will be constrained based on register id.

The auto-created constraints above will ensure that our action produces a valid register address and flip_bit based on the register being tested.

In addition to automatically creating constraints, a PSS coverage model can automatically be created to ensure that we have covered all registers and all bits within the registers.

Creating Top-Level Test Scenario

Once we have the core register-test intent, we need to integrate that into a top-level PSS scenario. While our core register-test intent was automatically derived from the register model, our top-level register test scenario will be created by hand.

The figure above shows our test scenario, which is built on top of the core action that encodes our test intent. Our test scenario is encapsulated in a top-level component, as required by PSS. We create an instance of the register-test component (subsys_reg_block_c) inside the top-level component, since we will be using the action from this component. In our top-level action (my_subsys_regtest_a), we create an instance of the subsys_reg_block_regs_a action named testbit. Inside this action is an instance of the register-test fields and covergroup. In the top-level action’s activity, we run testbit 100 times, which means that we will test 100 register bits each time our test runs.

Mapping to Test Realization

Our top-level test scenario doesn’t actually do anything yet because we’re missing test realization. Fortunately, PSS allows us to easily layer in test realization without changing the core description – in this case, the subsys_reg_block_regs_a action.

The figure above shows a test realization description for SystemVerilog that leverages the PSS procedural interface. The signature of an external function is declared, and that function is invoked from the exec block of an action. This style of test realization works for any environment that supports callable procedures – C, C++, SystemVerilog, etc.

In most cases, our bare-metal embedded software tests for our SoC will be written in C. But, what if we needed to create an assembly-language test? Fortunately, PSS provides a way to do this as well!

In the test realization snipped above, we are using a PSS target-template exec block to specify a snippet of assembly code (RISC-V in this case) that must be generated to test a register bit. The curly braces (e.g., {{reg_addr}}) are used to reference the current value of a field in the PSS model and substitute that value into the generated code. Doing test realization in assembly language certainly has its limitations, but PSS makes it possible when that’s the technique that is needed!

APPLYING PSS TESTS

Now that we have test intent and test realization to test access to our registers, we can begin running tests. In a UVM environment, PSS gives us flexibility to either pre-generate directed tests or run a PSS solver engine along with the running simulation. Both approaches have benefits and drawbacks. On the one hand, a directed test is easy to understand.

The UVM sequence above, for example, is very easy to understand, and always does exactly the same thing. But, we actually want our tests to do something slightly different as we run different seeds. And, we want to be able to easily partition the tens of thousands of register tests across different simulations running in regression. This is where using a PSS solver engine that runs along with simulation really helps. Running the same sequence with different seeds results in different behavior, and PSS tools like Questa® inFact provide dedicated features for dynamically partitioning tests across simulations running in a regression.

If we are able to use C in our bare-metal software environment, we can use the testbit function implementation and the PSS procedural interface to generate C test code, as shown above.

However, if we need to use assembly, we can use our target-template exec block test realization to generate a fully standalone test. A snippet from that test is shown below, left, with tests for two register/bit combinations.

SUMMARY

Register tests are a very useful smoke test in all environments from IP to subsystem to SoC level. While the built-in register-test sequences in the UVM library are primarily useful at IP level, capturing register-test test intent in a PSS model makes register-test functionality portable from IP to SoC level, and provides more flexibility in controlling which registers are checked in a given test run.

Both the input files to register-generation tools and the resulting UVM register model contain sufficient information to automatically create portable stimulus test intent. This makes it very easy for register-generation tools to add support for portable stimulus tests. It also makes it very easy to derive portable stimulus tests from existing UVM register models, regardless of how they were created.

So, portable stimulus isn’t reserved for only the most difficult of tests. Sometimes it can be applied just as easily and productively to seemingly-simple test tasks, like register-access tests, where it brings portability and saves duplicated effort.

If you wish to download a copy of this white paper, click here