ORC: Broken on LLVM 17 due to expected EH functions

[maleadt]

Our ORC wrappers are broken on LLVM 17:

❯ jl +nightly --project examples/sum_orc.jl
ERROR: LoadError: LLVM error: Symbols not found: [ llvm_orc_registerEHFrameSectionWrapper ]
Stacktrace:
 [1] macro expansion
   @ ~/Julia/pkg/LLVM/src/executionengine/utils.jl:32 [inlined]
 [2] LLJIT(builder::LLJITBuilder)
   @ LLVM ~/Julia/pkg/LLVM/src/executionengine/lljit.jl:38
 [3] #LLJIT#66
   @ ~/Julia/pkg/LLVM/src/executionengine/lljit.jl:59 [inlined]
 [4] top-level scope
   @ ~/Julia/pkg/LLVM/examples/sum_orc.jl:45
 [5] include
   @ ./Base.jl:559 [inlined]
 [6] exec_options(opts::Base.JLOptions)
   @ Base ./client.jl:325
 [7] _start()
   @ Base ./client.jl:533
in expression starting at /home/tim/Julia/pkg/LLVM/examples/sum_orc.jl:45

Looks like ORC now expects certain LLVM symbols to be globally visible: llvm/llvm-project#74671. This basically requires a change in how Julia links against LLVM, re-exporting those symbols. The alternatives aren't viable (see apache/arrow#39695 (comment)). I think I agree with llvm/llvm-project#74671 (comment) that this is a bad design decision on LLVM's side.

Attempted workaround 1: add llvm_orc_*; to Julia's julia.expmap. Doesn't work.

Attempted workaround 2: register globally visible c-callable functions:

if LLVM.version() >= v"17"
    struct CWrapperFunctionResult
        Data::Ptr{Cchar}
        Size::Csize_t
    end
    Base.@ccallable llvm_orc_registerJITLoaderGDBWrapper(Data::Ptr{Cchar}, Size::UInt64)::CWrapperFunctionResult =
        @ccall libllvm.llvm_orc_registerJITLoaderGDBWrapper(Data::Ptr{Cchar}, Size::UInt64)::CWrapperFunctionResult
    Base.@ccallable llvm_orc_registerEHFrameSectionWrapper(Data::Ptr{Cchar}, Size::UInt64)::CWrapperFunctionResult =
        @ccall libllvm.llvm_orc_registerEHFrameSectionWrapper(Data::Ptr{Cchar}, Size::UInt64)::CWrapperFunctionResult
    Base.@ccallable llvm_orc_deregisterEHFrameSectionWrapper(Data::Ptr{Cchar}, Size::UInt64)::CWrapperFunctionResult =
        @ccall libllvm.llvm_orc_deregisterEHFrameSectionWrapper(Data::Ptr{Cchar}, Size::UInt64)::CWrapperFunctionResult
end

Doesn't work.

cc @vchuravy @gbaraldi @lhames

[vchuravy]

For the ccallable one likely needs https://github.com/maleadt/LLVM.jl/blob/dc7f828da0d219cca50916917bf861cfac140637/test/orc_tests.jl#L45

Alternatively we could use absolute symbol resolvers https://github.com/maleadt/LLVM.jl/blob/dc7f828da0d219cca50916917bf861cfac140637/test/orc_tests.jl#L169

[maleadt]

This happens during JIT construction though:

julia> using LLVM

julia> tm = JITTargetMachine()
TargetMachine(Ptr{LLVM.API.LLVMOpaqueTargetMachine} @0x0000000002589d00)

julia> jit = LLJIT(; tm=JITTargetMachine())
ERROR: LLVM error: Symbols not found: [ llvm_orc_registerEHFrameSectionWrapper ]

[maleadt]

I was going to revisit this as I just remembered about LLVMAddSymbol (to add the result of the ccallable's above), but it seems like this just works on LLVM 18? Not sure what changed, but 🤷