-
Notifications
You must be signed in to change notification settings - Fork 13k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
emscripten: Use the latest emsdk 3.1.68 #131533
emscripten: Use the latest emsdk 3.1.68 #131533
Conversation
This should fix our precompiled std being unsound in `std::fs` code.
I have to say that I'm often clueless when reviewing these kinds of PRs :) Do we have some documentation about what does it mean to update this? Do we even document somewhere which version of emscripten we use for given targets, or is it just an "internal implementation detail". |
We do not, but we should, probably. I can add the relevant details for our platform support pages? I think since we build the SDK, it shouldn't have a significant consequence? (aside from the bugfix, of course) I am not entirely sure what the consequences of this are, tbh. I'm also not entirely sure this won't break Pyodide for @juntyr, either, given 3.1.68 is not 3.1.46, but I'm not sure what to make of emsdk's versioning. It could be SemVer, in which case this should be fine, but then why did 3.1.42 break ABI? Perhaps @alexcrichton or @Manishearth might know? |
Updating the Emscripten version to something newer is definitely a good idea. But it should go hand-in-hand with some documentation (I'll submit a PR later) on this coupling between Emscripten and Rust that won't matter in most cases until you get a weird bug, and how to work around it with I think I was experiencing this issue for a long while but simply never noticed. Pyodide can use ~2GB RAM before anything breaks, and up until a few days ago even the senseless overallocation during file reading never hit the boundary. So it was by pure chance and additional up-front memory pressure that allocating an extra 1.8GB was no longer fine and I noticed the ABI mismatch. |
And make the changes in commit 63b0d67 unconditional. Supersedes: rust-lang#3569. Cherry-picks a couple of changes from: rust-lang#3307. Depends on: rust-lang/rust#131533.
Thanks for helping to support the Emscripten target @workingjubilee! I really appreciate it. |
Alas I'm ignorant in the ways of Emscripten here so I also don't understand the full consequences of what an update here might have. In the absence of more information it seems reasonable to go ahead with the update and use it as a learning process for what to and what not to do in the future. |
atm I don't think anyone understands how Rust interacts with Emscripten, tbh. @sbc100 if you could weigh in? |
I also don't know much about how emscripten and rust work together, but updating certainly seems like a good idea. 2.0.5 is ancient. I assume there is some amount of CI testing that occurs here to make sure at least some things work? |
...we make sure things build! :D |
Only a tier 1 target is actually expected to run tests, and they run all the tests, so tier 2 targets like emscripten Basically Don't. There's a few which run anyways, but we do not extensively test tier 2 non-host targets. In fact there were quite a few ignores in the test suite just to make sure that no one even tries to run them for emscripten. Some of that was unavoidable because e.g. emscripten doesn't have enough IO support and some tests are of IO-using programs. But uh, I am the very person who ripped out asmjs-unknown-emscripten support because it had been silently broken for years, as the only vestiges of CI testing that we have for tier 2 had been completely disabled for it, and no one noticed or complained. |
It seems like merging this and seeing if it breaks anyone externally might be the only reasonable way forward (assuming that it passes CI). You can r=me in that regard. |
Yeah, that was my feeling as well. It would be nice to land more tests for emscripten but even for the tests that we do run, it already has a custom test harness in compiletest. More tests may thus require wrangling more Fun, and such tests will not necessarily be checked in CI. Indeed, a large portion of the UI test suite seems to fall prey to linking failures when I try to build it, for reasons that don't have any clear relationship with the EMSDK version, and probably have been true for quite some time. As this is a soundness concern, it doesn't seem worth blocking on a task which will take an indefinite amount of time. Thus, let us give scream-based deployment testing a go. @bors r=Kobzol |
…ersion, r=Kobzol emscripten: Use the latest emsdk 3.1.68 This should fix our precompiled std being unsound in `std::fs` code. Should resolve rust-lang#131467 I hope.
…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#128967 (std::fs::get_path freebsd update.) - rust-lang#129794 (uefi: Implement getcwd and chdir) - rust-lang#130629 (core/net: add Ipv[46]Addr::from_octets, Ipv6Addr::from_segments.) - rust-lang#131274 (library: Const-stabilize `MaybeUninit::assume_init_mut`) - rust-lang#131473 (compiler: `{TyAnd,}Layout` comes home) - rust-lang#131533 (emscripten: Use the latest emsdk 3.1.68) - rust-lang#131593 (miri: avoid cloning AllocExtra) - rust-lang#131616 (merge const_ipv4 / const_ipv6 feature gate into 'ip' feature gate) r? `@ghost` `@rustbot` modify labels: rollup
…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#128967 (std::fs::get_path freebsd update.) - rust-lang#130629 (core/net: add Ipv[46]Addr::from_octets, Ipv6Addr::from_segments.) - rust-lang#131274 (library: Const-stabilize `MaybeUninit::assume_init_mut`) - rust-lang#131473 (compiler: `{TyAnd,}Layout` comes home) - rust-lang#131533 (emscripten: Use the latest emsdk 3.1.68) - rust-lang#131593 (miri: avoid cloning AllocExtra) - rust-lang#131616 (merge const_ipv4 / const_ipv6 feature gate into 'ip' feature gate) - rust-lang#131660 (Also use outermost const-anon for impl items in `non_local_defs` lint) r? `@ghost` `@rustbot` modify labels: rollup
…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#128967 (std::fs::get_path freebsd update.) - rust-lang#130629 (core/net: add Ipv[46]Addr::from_octets, Ipv6Addr::from_segments.) - rust-lang#131274 (library: Const-stabilize `MaybeUninit::assume_init_mut`) - rust-lang#131473 (compiler: `{TyAnd,}Layout` comes home) - rust-lang#131533 (emscripten: Use the latest emsdk 3.1.68) - rust-lang#131593 (miri: avoid cloning AllocExtra) - rust-lang#131616 (merge const_ipv4 / const_ipv6 feature gate into 'ip' feature gate) - rust-lang#131660 (Also use outermost const-anon for impl items in `non_local_defs` lint) r? `@ghost` `@rustbot` modify labels: rollup
…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#128967 (std::fs::get_path freebsd update.) - rust-lang#130629 (core/net: add Ipv[46]Addr::from_octets, Ipv6Addr::from_segments.) - rust-lang#131274 (library: Const-stabilize `MaybeUninit::assume_init_mut`) - rust-lang#131473 (compiler: `{TyAnd,}Layout` comes home) - rust-lang#131533 (emscripten: Use the latest emsdk 3.1.68) - rust-lang#131593 (miri: avoid cloning AllocExtra) - rust-lang#131616 (merge const_ipv4 / const_ipv6 feature gate into 'ip' feature gate) - rust-lang#131660 (Also use outermost const-anon for impl items in `non_local_defs` lint) r? `@ghost` `@rustbot` modify labels: rollup
Rollup merge of rust-lang#131533 - workingjubilee:update-emscripten-version, r=Kobzol emscripten: Use the latest emsdk 3.1.68 This should fix our precompiled std being unsound in `std::fs` code. Should resolve rust-lang#131467 I hope.
Rust 1.84 uses the latest emsdk 3.1.68 [1], and it fixed an issue with Emscripten dynamic linking and libc [2]. After that no native-static-libs in the output if running: ``` rustc --target=wasm32-unknown-emscripten --crate-type staticlib --print native-static-libs - < /dev/null ``` [1] rust-lang/rust#131533 [2] rust-lang/libc#4002
…mulacrum Remove emsdk version update from 1.84.0 relnotes See [this comment](rust-lang#131467 (comment)). The reproducer in that comment does indeed show that rustup's `rust-std` component is still compiled with the old emscripten ABI because libc's config flag `emscripten_new_stat_abi` is not set. rust-lang#131533 presumably had no effect because the wrong CI file was modified. So nothing has changed since 1.83.0. The PR author (workingjubilee) is currently on vacation. Also the issue rust-lang#131467 should be reopened.
…mulacrum Remove emsdk version update from 1.84.0 relnotes See [this comment](rust-lang#131467 (comment)). The reproducer in that comment does indeed show that rustup's `rust-std` component is still compiled with the old emscripten ABI because libc's config flag `emscripten_new_stat_abi` is not set. rust-lang#131533 presumably had no effect because the wrong CI file was modified. So nothing has changed since 1.83.0. The PR author (workingjubilee) is currently on vacation. Also the issue rust-lang#131467 should be reopened.
Rollup merge of rust-lang#135266 - kadiwa4:no_emsdk_update, r=Mark-Simulacrum Remove emsdk version update from 1.84.0 relnotes See [this comment](rust-lang#131467 (comment)). The reproducer in that comment does indeed show that rustup's `rust-std` component is still compiled with the old emscripten ABI because libc's config flag `emscripten_new_stat_abi` is not set. rust-lang#131533 presumably had no effect because the wrong CI file was modified. So nothing has changed since 1.83.0. The PR author (workingjubilee) is currently on vacation. Also the issue rust-lang#131467 should be reopened.
Pkgsrc changes: * Adapt patches, one of the patched files were restructured upstream. * Checksum changes. Upstream changes: Version 1.84.1 (2025-01-30) ========================== - [Fix ICE 132920 in duplicate-crate diagnostics.] (rust-lang/rust#133304) - [Fix errors for overlapping impls in incremental rebuilds.] (rust-lang/rust#133828) - [Fix slow compilation related to the next-generation trait solver.] (rust-lang/rust#135618) - [Fix debuginfo when LLVM's location discriminator value limit is exceeded.] (rust-lang/rust#135643) - Fixes for building Rust from source: - [Only try to distribute `llvm-objcopy` if llvm tools are enabled.] (rust-lang/rust#134240) - [Add Profile Override for Non-Git Sources.] (rust-lang/rust#135433) - [Resolve symlinks of LLVM tool binaries before copying them.] (rust-lang/rust#135585) - [Make it possible to use ci-rustc on tarball sources.] (rust-lang/rust#135722) Version 1.84.0 (2025-01-09) ========================== Language -------- - [Allow `#[deny]` inside `#[forbid]` as a no-op] (rust-lang/rust#121560) - [Show a warning when `-Ctarget-feature` is used to toggle features that can lead to unsoundness due to ABI mismatches] (rust-lang/rust#129884) - [Use the next-generation trait solver in coherence] (rust-lang/rust#130654) - [Allow coercions to drop the principal of trait objects] (rust-lang/rust#131857) - [Support `/` as the path separator for `include!()` in all cases on Windows] (rust-lang/rust#125205) - [Taking a raw ref (`raw (const|mut)`) of a deref of a pointer (`*ptr`) is now safe] (rust-lang/rust#129248) - [Stabilize s390x inline assembly] (rust-lang/rust#131258) - [Stabilize Arm64EC inline assembly] (rust-lang/rust#131781) - [Lint against creating pointers to immediately dropped temporaries] (rust-lang/rust#128985) - [Execute drop glue when unwinding in an `extern "C"` function] (rust-lang/rust#129582) Compiler -------- - [Add `--print host-tuple` flag to print the host target tuple and affirm the "target tuple" terminology over "target triple"] (rust-lang/rust#125579) - [Declaring functions with a calling convention not supported on the current target now triggers a hard error] (rust-lang/rust#129935) - [Set up indirect access to external data for `loongarch64-unknown-linux-{musl,ohos}`] (rust-lang/rust#131583) - [Enable XRay instrumentation for LoongArch Linux targets] (rust-lang/rust#131818) - [Extend the `unexpected_cfgs` lint to also warn in external macros] (rust-lang/rust#132577) - [Stabilize WebAssembly `multivalue`, `reference-types`, and `tail-call` target features] (rust-lang/rust#131080) - [Added Tier 2 support for the `wasm32v1-none` target] (rust-lang/rust#131487) Libraries --------- - [Implement `From<&mut {slice}>` for `Box/Rc/Arc<{slice}>`] (rust-lang/rust#129329) - [Move `<float>::copysign`, `<float>::abs`, `<float>::signum` to `core`] (rust-lang/rust#131304) - [Add `LowerExp` and `UpperExp` implementations to `NonZero`] (rust-lang/rust#131377) - [Implement `FromStr` for `CString` and `TryFrom<CString>` for `String`] (rust-lang/rust#130608) - [`std::os::darwin` has been made public] (rust-lang/rust#130635) Stabilized APIs --------------- - [`Ipv6Addr::is_unique_local`] (https://doc.rust-lang.org/stable/core/net/struct.Ipv6Addr.html#method.is_unique_local) - [`Ipv6Addr::is_unicast_link_local`] (https://doc.rust-lang.org/stable/core/net/struct.Ipv6Addr.html#method.is_unicast_link_local) - [`core::ptr::with_exposed_provenance`] (https://doc.rust-lang.org/stable/core/ptr/fn.with_exposed_provenance.html) - [`core::ptr::with_exposed_provenance_mut`] (https://doc.rust-lang.org/stable/core/ptr/fn.with_exposed_provenance_mut.html) - [`<ptr>::addr`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.addr) - [`<ptr>::expose_provenance`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.expose_provenance) - [`<ptr>::with_addr`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.with_addr) - [`<ptr>::map_addr`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.map_addr) - [`<int>::isqrt`] (https://doc.rust-lang.org/stable/core/primitive.i32.html#method.isqrt) - [`<int>::checked_isqrt`] (https://doc.rust-lang.org/stable/core/primitive.i32.html#method.checked_isqrt) - [`<uint>::isqrt`] (https://doc.rust-lang.org/stable/core/primitive.u32.html#method.isqrt) - [`NonZero::isqrt`] (https://doc.rust-lang.org/stable/core/num/struct.NonZero.html#impl-NonZero%3Cu128%3E/method.isqrt) - [`core::ptr::without_provenance`] (https://doc.rust-lang.org/stable/core/ptr/fn.without_provenance.html) - [`core::ptr::without_provenance_mut`] (https://doc.rust-lang.org/stable/core/ptr/fn.without_provenance_mut.html) - [`core::ptr::dangling`] (https://doc.rust-lang.org/stable/core/ptr/fn.dangling.html) - [`core::ptr::dangling_mut`] (https://doc.rust-lang.org/stable/core/ptr/fn.dangling_mut.html) These APIs are now stable in const contexts - [`AtomicBool::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicBool.html#method.from_ptr) - [`AtomicPtr::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicPtr.html#method.from_ptr) - [`AtomicU8::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicU8.html#method.from_ptr) - [`AtomicU16::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicU16.html#method.from_ptr) - [`AtomicU32::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicU32.html#method.from_ptr) - [`AtomicU64::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicU64.html#method.from_ptr) - [`AtomicUsize::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicUsize.html#method.from_ptr) - [`AtomicI8::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicI8.html#method.from_ptr) - [`AtomicI16::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicI16.html#method.from_ptr) - [`AtomicI32::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicI32.html#method.from_ptr) - [`AtomicI64::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicI64.html#method.from_ptr) - [`AtomicIsize::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicIsize.html#method.from_ptr) - [`<ptr>::is_null`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.is_null-1) - [`<ptr>::as_ref`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.as_ref-1) - [`<ptr>::as_mut`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.as_mut) - [`Pin::new`] (https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.new) - [`Pin::new_unchecked`] (https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.new_unchecked) - [`Pin::get_ref`] (https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.get_ref) - [`Pin::into_ref`] (https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.into_ref) - [`Pin::get_mut`] (https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.get_mut) - [`Pin::get_unchecked_mut`] (https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.get_unchecked_mut) - [`Pin::static_ref`] (https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.static_ref) - [`Pin::static_mut`] (https://doc.rust-lang.org/stable/core/pin/struct.Pin.html#method.static_mut) Cargo ----- - [Stabilize MSRV-aware resolver config] (rust-lang/cargo#14639) - [Stabilize resolver v3] (rust-lang/cargo#14754) Rustdoc ------- - [rustdoc-search: improve type-driven search] (rust-lang/rust#127589) Compatibility Notes ------------------- - [Enable by default the `LSX` target feature for LoongArch Linux targets] (rust-lang/rust#132140) - [The unstable `-Zprofile` flag ("gcov-style" coverage instrumentation) has been removed.](rust-lang/rust#131829) This does not affect the stable flags for coverage instrumentation (`-Cinstrument-coverage`) and profile-guided optimization (`-Cprofile-generate`, `-Cprofile-use`), which are unrelated and remain available. - Support for the target named `wasm32-wasi` has been removed as the target is now named `wasm32-wasip1`. This completes the [transition] (rust-lang/compiler-team#607) [plan](rust-lang/compiler-team#695) for this target following [the introduction of `wasm32-wasip1`] (rust-lang/rust#120468) in Rust 1.78. Compiler warnings on [use of `wasm32-wasi`] (rust-lang/rust#126662) introduced in Rust 1.81 are now gone as well as the target is removed. - [The syntax `&pin (mut|const) T` is now parsed as a type which in theory could affect macro expansion results in some edge cases] (rust-lang/rust#130635 (comment)) - [Legacy syntax for calling `std::arch` functions is no longer permitted to declare items or bodies (such as closures, inline consts, or async blocks).] (rust-lang/rust#130443 (comment)) - The `wasm32-unknown-emscripten` target's binary release of the standard library is now [built with the latest emsdk 3.1.68] (rust-lang/rust#131533), which fixes an ABI-incompatibility with Emscripten >= 3.1.42. If you are locally using a version of emsdk with an incompatible ABI (e.g. before 3.1.42 or a future one), you should build your code with `-Zbuild-std` to ensure that `std` uses the correct ABI. - [Declaring functions with a calling convention not supported on the current target now triggers a hard error] (rust-lang/rust#129935) - [The next-generation trait solver is now enabled for coherence, fixing multiple soundness issues] (rust-lang/rust#130654)
This should fix our precompiled std being unsound in
std::fs
code.Should resolve #131467 I hope.