Add first-party tests

[Timmmm]

I think we should have a place for first-party riscv-tests style tests so we can write tests for PRs and bug fixes. These wouldn't be exhaustive tests or verification, but specific to the Sail model (though we should try and write them so they can run on any RISC-V CPU).

Some examples of tests that I've wanted recently:

• Test accesses with pmpaddr to 0xFFFFFFFF (Incorrect behaviour with INT_MAX pmpaddr #739)
• Test updating the A/D bits of a PTE in the TLB where the PTE cannot be accessed due to PMP (Possible reachable assertion when updating PTEs #711)
• Verify mstatus[FS] is read-only without S or F implemented (mstatus[FS] must be read-only zero in some cases #661)

I have a prototype using Zig that I will make a draft PR. Zig is an unusual choice, but I think it has a number of very significant advantages:

• The compiler is trivial to install and can cross-compile to RISC-V.
• Fast compilation.
• Sane modern language and build system.
• Writing unsafe code is not difficult.
• Integrates seamlessly with C and assembly (so you can still write tests in C if you really want).
• Focus on optimal code generation - shorter assembly output makes debugging easier.

The big downside is that it's far from a 1.0 release and relatively unstable, so we'd probably need to pin the compiler version, and do some work when 1.0 is released. Also the documentation is sparse, but ChatGPT helps a lot.

Here's an example test for #739

const std = @import("std");
const runtime = @import("runtime");

export fn main() u8 {
    if (test_main()) |_| {
        return 0;
    } else |_| {
        return 1;
    }
}

const PMPCFG_L = 0b1 << 7;
const PMPCFG_NA4 = 0b10 << 3;
const PMPCFG_NAPOT = 0b11 << 3;
const PMPCFG_X = 0b1 << 2;
const PMPCFG_W = 0b1 << 1;
const PMPCFG_R = 0b1 << 0;

const MSTATUS_MPP_MASK = 0b11 << 11;
const MSTATUS_MPRV_MASK = 0b1 << 17;

fn test_main() !void {
    const stdout = runtime.htif.getHtifWriter();

    try stdout.print("Testing 0xFF..FF PMP addresses\n", .{});

    const ones: usize = std.math.maxInt(usize);

    asm volatile ("csrw pmpaddr0, %[ones]"
        :
        : [ones] "r" (ones),
    );

    // Set mstatus.MPP and mstatus.MPRV so that loads/stores are
    // done in user mode. If they don't match the pmp then
    // they'll fail.
    const mpp: usize = MSTATUS_MPP_MASK;
    const mprv: usize = MSTATUS_MPRV_MASK;
    asm volatile (
        \\ csrc mstatus, %[mpp]
        \\ csrs mstatus, %[mprv]
        :
        : [mpp] "r" (mpp),
          [mprv] "r" (mprv),
    );

    const pmpcfg_napot: usize = PMPCFG_NAPOT | PMPCFG_X | PMPCFG_W | PMPCFG_R;

    asm volatile ("csrw pmpcfg0, %[cfg]"
        :
        : [cfg] "r" (pmpcfg_napot),
    );

    // Access memory to test loads/stores.
    try stdout.print("Passed\n", .{});
}

[Alasdair]

Zig is an unusual choice

Unusual, but I think actually quite compelling for this use case with the out of the box cross-compilation. I suppose C would be a more standard choice.

The only thing I'm worried about is adding to our reputation for using 'esoteric' languages, I've had comments on Sail's use of OCaml as an implementation language, even though it's A) very well suited to the task and B) really not that unusual within the compiler/programming language space.

[arichardson]

Clang allows cross-compilation out-of-the-box as well and using C (or my preference C++) would be a bit less surprising to newcomers?

[jrtc27]

Yeah can we not use weirdo languages like Zig? I know it has its following but it's far from mainstream. Yes, that's true for Sail too, but that's because ISA modelling is inherently niche; writing boilerplate tests is not.

[Timmmm]

Clang is way harder to install though.

One option is just to use Zig as a cross-compiler and build system, and write the tests in C. I'll push a PR with examples of both.

[arichardson]

Clang is way harder to install though.

It won't be needed for people to use the sail model, only people writing new tests (or running the existing ones).
I think we should be able to either use the system package manager or download https://github.com/llvm/llvm-project/releases/download/llvmorg-19.1.0/LLVM-19.1.0-Linux-X64.tar.xz

One option is just to use Zig as a cross-compiler and build system, and write the tests in C. I'll push a PR with examples of both.

[allenjbaum]

Note that there are ACT tests that tests a lot of the above kinds of examples. I'm not sure we specifically tests for accesses to the top by of memory, but we do have PMP and MMU crossproducts.

…

On Fri, Feb 14, 2025 at 1:28 PM Alexander Richardson < ***@***.***> wrote: Clang is way harder to install though. It won't be needed for people to use the sail model, only people writing new tests (or running the existing ones). I think we should be able to either use the system package manager or download https://github.com/llvm/llvm-project/releases/download/llvmorg-19.1.0/LLVM-19.1.0-Linux-X64.tar.xz One option is just to use Zig as a cross-compiler and build system, and write the tests in C. I'll push a PR with examples of both. — Reply to this email directly, view it on GitHub <#740 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJTKLTPUI47SNSMFS4D2PZNXZAVCNFSM6AAAAABXE4HOUGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRQGMYDENRWGQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> [image: arichardson]*arichardson* left a comment (riscv/sail-riscv#740) <#740 (comment)> Clang is way harder to install though. It won't be needed for people to use the sail model, only people writing new tests (or running the existing ones). I think we should be able to either use the system package manager or download https://github.com/llvm/llvm-project/releases/download/llvmorg-19.1.0/LLVM-19.1.0-Linux-X64.tar.xz One option is just to use Zig as a cross-compiler and build system, and write the tests in C. I'll push a PR with examples of both. — Reply to this email directly, view it on GitHub <#740 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJTKLTPUI47SNSMFS4D2PZNXZAVCNFSM6AAAAABXE4HOUGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRQGMYDENRWGQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>