OpenBSD port tracking

[3405691582]

Description

Instead of using the forums, use this Github issue to track OpenBSD porting status and notes that are beneficial for other people trying to build Swift on this platform but not necessarily appropriate for putting into prs yet for official documentation.

Notes

Last edit: 2025-02-04.

Currently the project contains a percentage of C++ dependencies. However, Swift will eventually be written with critical components in Swift, requiring us to build a bootstrap toolchain. Therefore, there are three main different types of build output to think about, namely,

0. the legacy compiler toolchain containing just the legacy Swift libraries and compiler (i.o.w., no Dispatch, no Foundation)
1. the bootstrap toolchain containing the legacy compiler toolchain plus the minimal set of dependencies to build the standard toolchain components written in Swift (swift-driver, etc.)
2. the standard toolchain, including components written in Swift

The legacy compiler toolchain is more or less working, but is not broadly useful. The standard toolchain is not completed yet. We need to create a bootstrap toolchain, since there is no prebuilt toolchain for OpenBSD, and we will eventually need one in the future to build Swift after some point.

• To create the legacy compiler toolchain (complete):

  • Prepare repositories (follow the doc, mostly).
    • Use Build without swift host tools #77815.
    • Use a compiler directive #if hasFeature(Macros) in stdlib/public/core/SwiftifyImport.swift as well.
    • Add #if SWIFT_BUILD_SWIFT_SYNTAX guards around ASTScopeImpl::unqualifiedLookup in ASTScope::unqualifiedLookup in lib/AST/ASTScope.cpp.
    • On arm64 builds, use Fix crash lowering stack guard on OpenBSD/aarch64. llvm/llvm-project#125416
  • Build (follow the doc, mostly).
    • The symlink trick appears no longer necessary; you can skip it.
    • You can turn on SWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY and SWIFT_IMPLICIT_CONCURRENCY_IMPORT; but you must turn these on for the bootstrap toolchain.

• To create the bootstrap toolchain (mostly complete/in progress):

  • Prepare repositories, as per above.
  • Prepare additional repositories.
    • Clone swift-corelibs-libdispatch, swift-corelibs-foundation, swift-foundation, swift-collections, swift-foundation-icu.
    • Use Support OpenBSD. swift-corelibs-libdispatch#559 for Dispatch.
    • Use OpenBSD support. swift-foundation#1126 for swift-foundation.
    • Create an implementation for stdlib/public/Synchronization/Mutex.swift using pthread_mutex_t?, or use the 5.10 version of Foundation.
      • Add the implementation and dependency in stdlib/public/Synchronization/CMakeLists.txt.
      • While this is not ideal to upstream, since pthread_mutex_t could be a value type with internal pointers, this should be safe enough for now since OpenBSD uses a pointer for pthread_mutex_t.
      • Note the 5.10 version of swift-corelibs-foundation is the one prior to Mutex being adopted.
  • Build.
    • Use install flags for build-script: --install-swift --install-foundation --install-xctest --install-libdispatch.
      • Note this doesn't install the just-built clang/LLVM. I don't know if this is better or not yet.
    • You will also likely need to turn on SWIFT_ENABLE_EXPERIMENTAL_STRING_PROCESSING, SWIFT_BUILD_REGEX_PARSER_IN_COMPILER and SWIFT_ENABLE_SYNCHRONIZATION if they are not on already; Foundation needs these turned on.
    • Disable SWIFT_ENABLE_EXPERIMENTAL_CXX_INTEROP to turn off C++ interop.
      • I think Swift's C++ interop doesn't work with libc++ platforms and needs to be provided for C++ interop to work. (I think this is why there are errors about a missing Clang module for CxxStdlib). Moreover, OpenBSD's C++ includes are missing a modulemap.
      • This is probably not going to be good enough long term.
    • Use a non-Release build type, or add target_compile_options(FoundationEssentials PRIVATE "SHELL:$<$<COMPILE_LANGUAGE:Swift>:-Xfrontend -disable-large-loadable-types-reg2mem>") to Sources/FoundationEssentials/CMakeLists.txt.
      • Non-Release compiles will crash building FoundationEssentials (and not just on OpenBSD);
        • details in Assert failure/crash in LoadableByAddress pass in getAddressForValue when building FoundationEssentials #78954.
        • The other workaround might not be working quite right on some platform versions (probably due to cmake weirdness?)
    • Foundation build for FoundationEssentials has error if any output files are specified, they all must be
      • this is due to cmake adding the static library dependencies for _FoundationCShims and _FoundationCollections as files and not library flags.

• To create the standard toolchain (mostly complete/in progress):

  • The two main tools built at this stage are swift-driver and swiftpm. You could potentially skip building swiftpm, but other platforms do include it.
  • Prepare repositories, as per above.
  • Prepare additional repositories: dependencies for swift-driver.
    • Clone swift-llbuild (as llbuild), swift-tools-support-core, swift-argument-parser and swift-driver.
      • Use OpenBSD gets backtrace from libexecinfo. swift-llbuild#972
      • llbuild and TSC built with cmake includes -lc for some reason linking libllbuildSwift, which gives a duplicate symbol error with atexit at link time. Workaround by clearing CMAKE_C_IMPLICIT_LINK_LIBRARIES and/or CMAKE_CXX_IMPLICIT_LINK_LIBRARIES .
      • Add target_link_options(Foundation PRIVATE "LINKER:-z" "LINKER:origin") in swift-corelibs-foundation and target_link_options(${target} PRIVATE "LINKER:-z" "LINKER:origin") in swift-syntax's AddSwiftHostLibrary.cmake
      • Add the PACBTI flags with add_compile_options to Dispatch and llbuild's CMakeLists.txt, since by default they use the just-built Clang.
  • Prepare additional repositories: dependencies for swiftpm.
    • Clone swift-system, swift-crypto, swift-asn1, swift-certificates, and swift-package-manager (as swiftpm)
    • Use OpenBSD support. apple/swift-system#215.
    • swift-crypto needs platform conditionals in Sources/CCryptoBoringSSL/CMakeLists.txt otherwise there is an unhelpful cmake error.
    • swift-crypto also has dependencies on ManagedBuffer.capacity. Some of this is easy to disentangle, but some requires more aggressive refactoring. This is the current blocker on building swiftpm presently.
  • Build.
    • Use env PATH=<path to the installed toolchain above> when invoking build-script.
    • If you've decided for whatever reason not to prepare for swiftpm, swift-driver will build fine, but the installer bootstrap will fail since it assumes that swiftpm exists already. This can be worked around, though.

• Needs investigation:

  • swiftc -num-threads can be memory-hungry and cause lockups; check and make sure threading is WAI
  • As mentioned, it's probably not going to be enough to disable C++ interop long term. We should possibly have a custom modulemap (much like the one for libstdc++), or figure out if the existing module-related errors can be worked around.
  • Check this all works back on amd64 at some point.

[finagolfin]

I've assigned you on this issue.

there are a number of issues with modules that prevent a successful build. Current suspicion is bad libc++ interaction with modules and/or existing upstream bugs with modules.

Have you tried turning off inline bridging mode, as currently done for Windows and as I used to do for Android?

I suggest you apply to be a committer if you plan to work on this port more. I can't say if you will get it, as I don't decide that, but based on your extensive track record, I suspect you will.

[finagolfin]

Note that yams has been removed in trunk, #78826.

[3405691582]

Have you tried turning off inline bridging mode, as currently done for Windows and as I used to do for Android?

I've experienced trouble with both PURE and INLINE bridging modes. Thankfully by turning interop off for now this sidesteps the issue and haven't been forced to deal with this problem further yet, though I likely will eventually.

[finagolfin]

I've experienced trouble with both PURE and INLINE bridging modes.

OK, pure bridging also stopped working for me natively on Android last March in trunk, but worked for building 5.10. You may want to try it with 5.10 on OpenBSD and see if that works. At the very least, that could give you a working "standard toolchain" that you could use to bootstrap everything else.

Thankfully by turning interop off for now this sidesteps the issue and haven't been forced to deal with this problem further yet, though I likely will eventually.

Yes, you will need it, as more and more of the compiler is written in Swift and directly calls LLVM/Swift C++ APIs inside the compiler.

I think Swift's C++ interop doesn't work with libc++ platforms and needs to be provided for C++ interop to work.

It does work, as most Swift platforms currently use libc++, such as all Darwin platforms and Android. Egor even got it working on linux, #75589.

That said, it can be a bit flaky and required some small changes to work on Android.

(I think this is why there are errors about a missing Clang module for CxxStdlib).

That module is added in this repo, you may just need to enable OpenBSD there, as Alex did for Android.

Moreover, OpenBSD's C++ includes are missing a modulemap.

That's surprising, as libc++ has shipped with a module map for years now. If OpenBSD is not including it for some reason, try copying it over and using it.

[3405691582]

I have what I think is a janky standard toolchain already atop HEAD/6.2 with C++ interop turned off, but granted, the next step is testing it to make sure it actually properly incorporates the Swift compiler sources. These are good breadcrumbs for when I get back to trying to reenable interop, thanks!

[jakepetroules]

@3405691582, thanks for your work on the OpenBSD port! We've recently open sourced Xcode's build system engine (Swift Build) which will eventually become part of the toolchain. I'm currently working on adding FreeBSD support there, and I expect OpenBSD support would be a similar change. Thought I'd mention it here so you can make sure you're aware of potential changes needed in this space. I filed an issue to track: swiftlang/swift-build#114