Failed assertion during IR decompression

[maleadt]

Introduced very recently, likely in #56880

julia: /source/src/julia.h:1319: jl_array_ptr_ref: Assertion `i < ((*(size_t*)jl_svecref(((jl_datatype_t*)(((((jl_taggedvalue_t*)((char*)(a) - sizeof(jl_taggedvalue_t)))->header) & ~(uintptr_t)15)))->parameters, 1)) == 1 ? (((jl_array_t*)(a))->dimsize[0]) : (((jl_array_t*)(a))->ref.mem->length))' failed.

[278] signal 6 (-6): Aborted
in expression starting at /home/pkgeval/.julia/packages/PyCall/1gn3u/test/runtests.jl:36
unknown function (ip: 0x7b80c81afd3c) at /lib/x86_64-linux-gnu/libc.so.6
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7b80c814b394) at /lib/x86_64-linux-gnu/libc.so.6
__assert_fail at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
jl_array_ptr_ref at /source/src/julia.h:1319
jl_array_ptr_ref at /source/src/julia.h:1319 [inlined]
lookup_root at /source/src/method.c:1483
jl_decode_value at /source/src/ircode.c:807
jl_decode_value_expr at /source/src/ircode.c:682 [inlined]
jl_decode_value at /source/src/ircode.c:840
ijl_uncompress_ir at /source/src/ircode.c:1158
_uncompressed_ir at ./runtime_internals.jl:1402
typeinf_ext_toplevel at ./../usr/share/julia/Compiler/src/typeinfer.jl:1215
jfptr_typeinf_ext_toplevel_117523.1 at /opt/julia/lib/julia/sys.so (unknown line)
_jl_invoke at /source/src/gf.c:3444 [inlined]
ijl_apply_generic at /source/src/gf.c:3644
jl_apply at /source/src/julia.h:2246 [inlined]
jl_type_infer at /source/src/gf.c:452
jl_compile_method_internal at /source/src/gf.c:2942
_jl_invoke at /source/src/gf.c:3436 [inlined]
ijl_apply_generic at /source/src/gf.c:3644
jl_apply at /source/src/julia.h:2246 [inlined]
do_call at /source/src/interpreter.c:125
eval_value at /source/src/interpreter.c:243
eval_stmt_value at /source/src/interpreter.c:194 [inlined]
eval_body at /source/src/interpreter.c:687
eval_body at /source/src/interpreter.c:562
eval_body at /source/src/interpreter.c:562

As seen on PkgEval: https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_date/2025-01/05/report.html

rr traces are available:

• https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/rr/PyCall-1736070706.tar.zst
• https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/rr/JuliaInterpreter-1736062570.tar.zst

[vtjnash]

I can't reproduce these, so I assume #56975 fixes it

[maleadt]

Latest PkgEval contains that commit, and there's still assertion failures during IR decompression, albeit slightly differently.

e.g. FileIO: https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_date/2025-01/09/FileIO.primary.log

julia: /source/src/method.c:1474: lookup_root: Assertion `key == 0' failed.

[21] signal 6 (-6): Aborted
in expression starting at /home/pkgeval/.julia/packages/FileIO/PtqMQ/test/query.jl:51
unknown function (ip: 0x7af52ab40ebc) at /lib/x86_64-linux-gnu/libc.so.6
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7af52aadc394) at /lib/x86_64-linux-gnu/libc.so.6
__assert_fail at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
lookup_root at /source/src/method.c:1474
jl_decode_value at /source/src/ircode.c:805
jl_decode_value_memory at /source/src/ircode.c:599
jl_decode_value_array1d at /source/src/ircode.c:642 [inlined]
jl_decode_value at /source/src/ircode.c:831
ijl_uncompress_ir at /source/src/ircode.c:1161
_uncompressed_ir at ./runtime_internals.jl:1404

I cannot reproduce this specific one though, so maybe one of the subsequent commits on master fixed this already.

[vchuravy]

That one is weird. It looks similar to #56947 which was fixed by #54738

[maleadt]

Still happens on 090e291: https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_date/2025-01/22/report.html

[maleadt]

The error has slightly changed, but more packages are affected:

julia: /source/src/method.c:1411: lookup_root: Assertion `key == 0' failed.

[21] signal 6 (-6): Aborted
in expression starting at /home/pkgeval/.julia/packages/FileIO/PtqMQ/test/query.jl:51
unknown function (ip: 0x79154923febc) at /lib/x86_64-linux-gnu/libc.so.6
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7915491db394) at /lib/x86_64-linux-gnu/libc.so.6
__assert_fail at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
lookup_root at /source/src/method.c:1411
jl_decode_value at /source/src/ircode.c:805
jl_decode_value_memory at /source/src/ircode.c:599
jl_decode_value_array1d at /source/src/ircode.c:642 [inlined]
jl_decode_value at /source/src/ircode.c:831
ijl_uncompress_ir at /source/src/ircode.c:1161
_uncompressed_ir at ./runtime_internals.jl:1413
typeinf_ext_toplevel at ./../usr/share/julia/Compiler/src/typeinfer.jl:1241
jfptr_typeinf_ext_toplevel_118592.1 at /opt/julia/lib/julia/sys.so (unknown line)
_jl_invoke at /source/src/gf.c:3474 [inlined]
ijl_apply_generic at /source/src/gf.c:3674

I still cannot reproduce, but all of the crashes have rr traces, e.g., https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/rr/FileIO-1737876863.tar.zst for FileIO.jl. These are replayable on any of the amdci servers.