Automating VHDL Testbench Creation

I do quite a bit of VHDL development in my job. Most VHDL blocks (entities) that I write are quite short. In my opinion, this is a good thing. It means that you can hold all of the variables in your head at the same time and actually feel confident that you understand how it all works. Smaller blocks are usually easier to reuse. They are also much simpler to test. Having many smaller blocks versus one large block does however mean writing more test benches.  Compounding this additional effort is the fact that VHDL is a very verbose language. My usual approach is to modify a template file which I keep to hand, however this can still take almost as much time and effort as writing the design under test (DUT)!

My goal is to find a better way of writing test benches, so that I can spend more time doing the fun bit, designing the functional design unit! I want to reduce the barrier to setting up a test bench. Ultimately I want to be confident that my design will work before spending time synthesising and testing the design on hardware.

I have found a variety of approaches to automating test bench generation. They seem to fall into the following broad categories:

Test vector based verification

This is the sledgehammer approach. You describe the state of all the input signals for each clock cycle/step of you simulation and then step through each line applying the signals. If you add in assertion statements you can catch errors during simulation. The input test vectors could be a spreadsheet, or generated by hand or by another piece of software. I don't like this approach because it is time consuming, error prone and very hard to read and modify. I can image certain situations where this really may be the best technique, perhaps if you wanted to verify an arithmetic function or a complex checksum calculator.

Transaction based verification

A transaction based simulation relies on using many of the non-synthesisable, sequential features of VHDL to call specific test procedures when certain signal events occur. By using VHDL procedures or components you can write reusable test constructs such as reacting to a read request signal, or the start of a bus transaction. Typically you create a master process which sequences calling each of the processes in turn. You can read more about this in [1] (see sources at end of post). I want to find out more about this style of VHDL and experiment with it to see if it really can expedite the process of writing test benches.

My main concern with this approach is that it requires a completely different style of HDL compared to that which you write when designing the DUT. This makes it quite difficult to switch between writing synthesisable code and writing the testbench. I'm also sceptical because it makes it too easy to forget about how the actual hardware will behave! If your testbench is purely sequential you may end up excluding the consequncies of concurrent events. You may also end up writing asynchronous code which doesn't accurately model the hardware the design under test will be interacting with.

Method 3

This is approach that I take at the moment. I'm not really sure if it has a name. It is simplistic, synchronous and (almost) looks like real, synthesisable VHDL. I typically define a clock and reset as follows:


clk0: process
begin
    Clk <= '0';
    wait for 6 NS;
    Clk <= '1';
    wait for 6 NS;
end process clk0;

rst0: process
begin
    Reset <= '1';
    wait for 12 NS;
    Reset <= '0';
    wait;
end process rst0;

I then set about writing a VHDL module which mimics the behaviour of the hardware which the DUT will interact with. I usually combine this with a counter which I use to synchronise stimulus to the DUT and to get things kick started. The synchronous counter ensures all events happen on active clock edges, as they will in the final design. It also means its easy to add new events to build up the simulation.

Typically I start with a test bench counter:

    -- Generate a test bench counter which we can use to
    -- synchronise events
    process(Clk, Reset)
    begin
        if(Reset = '1') then
            TbCounter   <= to_unsigned(0,8);
        elsif(Clk'event and Clk='1') then
            TbCounter <= TbCounter + to_unsigned(1,8);
        end if;
    end process;


I then generate DUT input signals from the count value:

    -- Example write strobe
    Write <= '1' when TbCounter = to_unsigned(5,8) else '0';

    -- Example address counter
    Address <= std_logic_vector(TbCounter(7 downto 4));

Sometimes it is helpful to capture the outputs of the DUT as they would be by subsequent hardware blocks in the final design. For example capturing a register write:

    -- Implement a storage register to capture the output from the DUT
    process(Clk, Reset)
    begin
        if(Reset = '1') then
            TbRegData   <= X"00000000";
        elsif(Clk'event and Clk='1') then
            if(Write = '1') then
                TbRegData   <= Data;
            end if;
        end if;
    end process;

With these simple building blocks you can set up simulations which exercise the DUT in realistic ways and which make analysis of simulation waveforms straight forward.

Room for improvement

As I mentioned before, writing test benches is time consuming. It is also fairly repetitive. A lot of the information in the DUT has to be replicated in the test bench, such as defining all the i/o signals, wiring up the port map and generating a clock, reset and test bench counter. I have therefore set about automating these initial stages of test bench generation. My end goal is to be able to embed test bench directives in the comments of the DUT port definitions and then run a script to automatically produce a test bench file which is immediately ready for simulation. This can then be used as a starting point for testing the DUT. In many cases, it would probably be all that is required. This is what I have in mind:

entity DUT is
port(
    Clk    : IN std_logic; -- @clk,6250,6250  
    ClkEn  : IN std_logic; -- @clkpulse,1,4,Clk  
    Reset  : IN std_logic; -- @pulse,10000,1,0
    DUTA   : INOUT std_logic_vector(7 downto 0);   -- @static,255
    DUTB   : INOUT std_logic_vector(7 downto 0);   -- @static,0
    DUTOUT : OUT std_logic;                        -- output only
);
end entity DUT;

This would create:
Clk - a clock with the specified high and low times in ps.
ClkEn - a synchronous pulse lasting 1 'Clk' period in every 4
Reset - an asynchronous pulse lasting 10000ps, starting at '1' then transitioning to '0'
DUTA - a statically assigned value of 255 b"11111111"
DUTB - a statically assigned value of 0 b"0000000"

I currently have a script, written in Perl, which performs the following stages:

1. Creates the entity and architecture statments
2. Defines all the signals necessary to interface with the DUT
3. Instantiates the DUT entity and sensibly wires up the port map

I'm working on the next stage: interpreting the '@' directives in the comments and generating the VHDL code required to achieve them. I'm looking forward to publishing it when its more...err...complete?! But for now you can download and run the script on any VHDL file and generate your own test benches. I have tested it on Windows and Linux.

The code is here: http://pastebin.com/0VJJPh95


Please let me know what you think. If you have your own ways of creating test benches I'd look forward to reading them!

Sources:
[1] Manual and Automatic VHDL/Verilog Test Bench Coding Techniques