Handle missing riscv64-unknown-linux-gnu

[cgwalters]

$ head -5  /etc/os-release                                                                                                                                                                                                                                                           03/01/24 16:06:05 PM
NAME="CentOS Stream"
VERSION="9"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="9"
$ rustc --version
rustc 1.75.0 (82e1608df 2023-12-21) (Red Hat 1.75.0-1.el9)
$ cargo-vendor-filterer
Gathering metadata
Skipping rpmostree-rust
Skipping rpmostree-client
Skipping libdnf-sys
Gathering metadata for platforms
error: Executing cargo metadata: Error during execution of `cargo metadata`: error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `rustc - --crate-name ___ --print=file-names --target riscv64gc-unknown-linux-gnu --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=split-debuginfo --print=crate-name --print=cfg` (exit status: 1)
  --- stderr
  error: could not create LLVM TargetMachine for triple: riscv64-unknown-linux-gn: No available targets are compatible with triple "riscv64-unknown-linux-gnu"
$

This however works with the Fedora rustc.

It seems to me RHEL rust toolchain apparently omits support for riscv64-unknown-linux-gnu...and hence using our default of tier = 2 breaks there.

Compare:

• Fedora rust package
• c9s rust package

This is messy. If we do something like detect what targets the toolchain has, then the filtering is suddenly dependent on the host.

Maybe we can just allow opting-in to that via e.g.

[workspace.metadata.vendor-filter]
platforms = ["*-unknown-linux-gnu"]
tier = "2"
filter-host = true

And filter-host acknowledges some potential unreproducibility.

[cgwalters]

If we do want to filter, what we need to figure out is if there's a way to get the valid list of targets that the compiler has enabled. (I don't understand actually why rustc --print=target-list isn't this)

[cgwalters]

Or hmm, we could add a tier = 2-rhel-9.2 or something that is defined to "intersection of tier-2 and rhel-9.2 targets". In practice, the Fedora rustc enables windows...but we don't care about that, so that intersection may just end up omitting riscv. (And also in practice, I doubt there are riscv specific crates anyways)

[cuviper]

rustc --print=target-list reports everything that has an internal target spec, including many that we will never actually support in Fedora or RHEL builds.

The error you're getting is because RHEL's LLVM does not have the RISC-V target arch enabled, so it can't initialize the backend. The rest of the triple doesn't matter in this case, just the arch, so for example you could examine targets like x86_64-fuchsia without trouble.

This is messy. If we do something like detect what targets the toolchain has, then the filtering is suddenly dependent on the host.

Yes, but I think this is unavoidable. Cargo [target.'cfg(..)'.dependencies] can use any of rustc --print=cfg, including stuff like target_feature that will depend on the target CPU, so we have to ask LLVM about that. In theory, a crate built on Fedora can resolve a different dependency tree than on RHEL 9 just because the latter targets x86-64-v2! In practice, I don't know if anyone really does that though...

If we do want to filter, what we need to figure out is if there's a way to get the valid list of targets that the compiler has enabled.

I don't know any way that this is exposed by rustc, apart from just trying it, but you can use llvm-config:

(c9s)$ llvm-config --targets-built
X86 AMDGPU PowerPC NVPTX SystemZ AArch64 ARM Mips BPF WebAssembly

(f39)$ llvm-config --targets-built
AArch64 AMDGPU ARM AVR BPF Hexagon Lanai LoongArch Mips MSP430 NVPTX PowerPC RISCV Sparc SystemZ VE WebAssembly X86 XCore

we could add a tier = 2-rhel-9.2 or something that is defined to "intersection of tier-2 and rhel-9.2 targets".

That's not so bad either, because that list doesn't change very often. Maybe you don't even need to distinguish y-stream, because if we add a new arch, new vendorers are probably only working on the latest. I don't know that you need an intersection with tier-2 either -- you could consider tier = rhel-9 to be separate from upstream's support.

[cgwalters]

This issue started because someone wanted to build rpm-ostree on c9s (going through the vendor-filter path). The more I think about this the more I think there are two separate paths:

• As an upstream project, one may want to support a maximal set of targets (specifically, given Fedora is testing out riscv, I don't want to exclude it from our vendor tarballs)
• As a local developer whose goal is really to build e.g. an RPM, I only care about filtering to the set that the build host (or I guess in theory the target OS) could support

So I think the fix is probably like:

• rpm-ostree (and other projects) should use a container with Rust upstream toolchain or equivalent in order to generate the official vendor tarballs
• We should add here tier = rhel-9.2
• rpm-ostree (and other projects) can learn how to (when e.g. make rpm is invoked) pass --tier=rhel-9.2 into the cargo-vendor-filterer invocation in order to override the upstream default in Cargo.toml

We actually may want a --filter-to-host option for the last case, i.e. don't even bother with other architectures/os etc.

[cgwalters]

llvm-config --targets-built
X86 AMDGPU PowerPC NVPTX SystemZ AArch64 ARM Mips BPF WebAssembly

That set of strings looks totally different from the target triples as known to Rust though. Do you know offhand if there's a defined way to map between them? It looks like lowercasing would get a lot of them, but not all (SystemZ -> systemz != s390x) e.g.

[cuviper]

Do you know offhand if there's a defined way to map between them?

Rust doesn't know the mapping at all, but LLVM parses the arch for its Triple class here:

https://github.com/llvm/llvm-project/blob/d36c9f828d795d77127ea0fada6be4b7b5e19dbb/llvm/lib/TargetParser/Triple.cpp#L529

... and each backend calls RegisterTarget for the Triple arches that it handles.