-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Modify-after-free bug #202
Comments
This comment was marked as outdated.
This comment was marked as outdated.
I haven't tried to reproduce locally, but this is a pretty bad error. Let me see if I can find a minimal example. |
I can reproduce this on macOS. Working on simplifying. |
(hyp) pkg> st
Status `/private/tmp/hyp/Project.toml`
[87dc4568] HiGHS v1.8.1
[b99e6be6] Hypatia v0.7.4
[4076af6c] JuMP v1.19.0
[2f354839] Pajarito v0.8.2
[ab19e554] PajaritoExtras v0.0.0 `https://github.com/chriscoey/PajaritoExtras.jl#main`
[37e2e46d] LinearAlgebra
[9a3f8284] Random
(hyp) pkg> st -m HiGHS_jll
Status `/private/tmp/hyp/Manifest.toml`
[8fd58aa0] HiGHS_jll v1.6.0+1
julia> versioninfo()
Julia Version 1.10.0
Commit 3120989f39b (2023-12-25 18:01 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: macOS (x86_64-apple-darwin22.4.0)
CPU: 8 × Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
Threads: 1 on 8 virtual cores A smaller example is using JuMP
import HiGHS
import Hypatia
import Pajarito
import PajaritoExtras # https://github.com/chriscoey/PajaritoExtras.jl
import Random
function main()
Random.seed!(2)
A = rand(30,7)
m, n = size(A)
opt = optimizer_with_attributes(
Pajarito.Optimizer,
"oa_solver" => optimizer_with_attributes(
HiGHS.Optimizer,
"mip_feasibility_tolerance" => 1e-8,
"mip_rel_gap" => 1e-6,
),
"conic_solver" => optimizer_with_attributes(Hypatia.Optimizer),
)
model = Model(opt)
@variables(model, begin
w[1:m]
x[1:m], Int
Z_mat[1:m, 1:n]
T_mat[1:m, 1:n]
J_mat[1:n, 1:n]
t
end)
@objective(model, Max, t)
J_diag = [J_mat[i, i] for i in 1:n]
@constraints(model, begin
sum(w) == 1
x .≤ 3
x .>= 0
w .>= 0
x .== 10w
sum(A[i, :] * Z_mat[i,:]' for i in 1:m) .== J_mat
[j in 1:n], sum(T_mat[:,j]) <= J_mat[j,j]
T_mat .>= 0
[t; J_diag] in Hypatia.HypoGeoMeanCone{Float64}(n+1)
[i in 1:m, j in 1:n], [T_mat[i,j]/2, w[i], Z_mat[i,j]] in Hypatia.EpiPerSquareCone{Float64}(3)
end)
for i in 1:(n-1), j in (i+1):n
JuMP.fix(J_mat[i,j], 0)
end
optimize!(model)
return
end
main()
|
So I caught the failing MPS file, but solving it in a new session doesn't trigger the crash. I think the problem is I'm missing the MIP start. So let me try and snatch that.
|
@dhendryc perhaps in the mean time, use a different MIP solver, like Gurobi or SCIP. |
I'm really struggling to create a standalone reproducer. A large part of it seems to be the iterative nature of the Pajarito solve. @jajhall is this stack trace helpful at all? julia(53732,0x7ff84b818700) malloc: Incorrect checksum for freed object 0x7fb3c477d200: probably modified after being freed.
Corrupt value: 0x0
julia(53732,0x7ff84b818700) malloc: *** set a breakpoint in malloc_error_break to debug
[53732] signal (6): Abort trap: 6
in expression starting at /private/tmp/hypatia.jl:51
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
pthread_kill at /usr/lib/system/libsystem_pthread.dylib (unknown line)
abort at /usr/lib/system/libsystem_c.dylib (unknown line)
malloc_vreport at /usr/lib/system/libsystem_malloc.dylib (unknown line)
malloc_zone_error at /usr/lib/system/libsystem_malloc.dylib (unknown line)
small_free_list_remove_ptr_no_clear at /usr/lib/system/libsystem_malloc.dylib (unknown line)
free_small at /usr/lib/system/libsystem_malloc.dylib (unknown line)
_ZNK11HSimplexNla16debugCheckInvertENSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEi at /Users/oscar/.julia/artifacts/c0c2656556da74046e66a7912b6d3d299b4c5a2d/lib/libhighs.1.6.0.dylib (unknown line)
_ZNK4HEkk19debugNlaCheckInvertENSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEi at /Users/oscar/.julia/artifacts/c0c2656556da74046e66a7912b6d3d299b4c5a2d/lib/libhighs.1.6.0.dylib (unknown line)
_ZN4HEkk13computeFactorEv at /Users/oscar/.julia/artifacts/c0c2656556da74046e66a7912b6d3d299b4c5a2d/lib/libhighs.1.6.0.dylib (unknown line) |
Here's a different one:
|
I managed to trigger this in a very unhelpful way WARNING: Failed to solve node with all integer columns fixed. Declaring node infeasible.
547 14 234 94.56% -0.1546302467 -0.1536649524 0.63% 135 0 332 259760 200.7s
581 11 251 96.72% -0.1546302467 -0.1536649524 0.63% 137 0 356 271128 205.7s
614 6 270 99.65% -0.1538506824 -0.1536649524 0.12% 150 0 390 280301 211.3s
Solving report
Status Optimal
Primal bound -0.153664952386
Dual bound -0.153665178278
Gap 0.000147% (tolerance: 0.000651%)
Solution status feasible
-0.153664952386 (objective)
0 (bound viol.)
0 (int. viol.)
0 (row viol.)
Timing 211.42 (total)
0.01 (presolve)
0.00 (postsolve)
Nodes 620
LP iterations 280918 (total)
47854 (strong br.)
0 (separation)
56809 (heuristics)
julia> import MathOptInterface as MOI; import LinearAlgebra
julia> x = MOI.VariableIndex(1)
julia(55690,0x7ff84b818700) malloc: Heap corruption detected, free list is damaged at 0x600000947200
*** Incorrect guard value: 8589934593992
julia(55690,0x7ff84b818700) malloc: *** set a breakpoint in malloc_error_break to debug
[55690] signal (6): Abort trap: 6
in expression starting at none:0
julia(55690,0x700005667000) malloc: Heap corruption detected, free list is damaged at 0x600000946000
*** Incorrect guard value: 4631491120
julia(55690,0x700005667000) malloc: *** set a breakpoint in malloc_error_break to debug
zsh: abort julia --project=. |
The warning you get from the HiGHS MIP solver is not one I've ever seen. It could be that it yields incorrect behaviour. I'll have a look. Rather than use gurobi or SCIP, users can try setting a different random seed before calling HiGHS::run() |
It works with SCIP. And it does work for HiGHS in some instances. Would be weird if that was due to a random seed, though. |
I don't think the |
OK Fair enough |
With a different random seed the solver will take a different path so maybe not hit the same error |
Just confirming that this is still a problem. But my efforts to create a pure HiGHS reproducer have not been successful. Coefficient ranges:
Matrix [2e-09, 3e+01]
Cost [1e+00, 1e+00]
Bound [0e+00, 0e+00]
RHS [1e+00, 3e+00]
Assessing feasibility of MIP using primal feasibility and integrality tolerance of 1e-08
Solution has num max sum
Col infeasibilities 0 0 0
Integer infeasibilities 0 0 0
Row infeasibilities 0 0 0
Row residuals 0 0 0
Presolving model
1332 rows, 465 cols, 4985 nonzeros 0s
1322 rows, 465 cols, 4928 nonzeros 0s
MIP start solution is feasible, objective value is -0.151653076173
Solving MIP model with:
1322 rows
465 cols (0 binary, 30 integer, 0 implied int., 435 continuous)
4928 nonzeros
Src: B => Branching; C => Central rounding; F => Feasibility pump; H => Heuristic; L => Sub-MIP;
P => Empty MIP; R => Randomized rounding; S => Solve LP; T => Evaluate node; U => Unbounded;
z => Trivial zero; l => Trivial lower; u => Trivial upper; p => Trivial point
Nodes | B&B Tree | Objective Bounds | Dynamic Constraints | Work
Src Proc. InQueue | Leaves Expl. | BestBound BestSol Gap | Cuts InLp Confl. | LpIters Time
0 0 0 0.00% -inf -0.1516530762 Large 0 0 0 0 0.0s
R 0 0 0 0.00% -0.1580085932 -0.1543643964 2.36% 0 0 0 2463 0.1s
[75316] signal (11.1): Segmentation fault: 11
in expression starting at /tmp/hypatia/main.jl:53
_ZN7HFactor11buildFinishEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN7HFactor5buildEP15HighsTimerClock at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN11HSimplexNla6invertEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN4HEkk13computeFactorEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN4HEkk33initialiseSimplexLpBasisAndFactorEb at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN4HEkk18initialiseForSolveEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN4HEkk5solveEb at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_Z14solveLpSimplexR19HighsLpSolverObject at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_Z7solveLpR19HighsLpSolverObjectNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN5Highs11callSolveLpER7HighsLpNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZZN5Highs5solveEvENK3$_0clER7HighsLpRKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEERd at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN5Highs5solveEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN17HighsLpRelaxation3runEb at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN17HighsLpRelaxation9resolveLpEP11HighsDomain at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN11HighsSearch12evaluateNodeEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN11HighsSearch4diveEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN14HighsMipSolver3runEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN21HighsPrimalHeuristics11solveSubMipERK7HighsLpRK10HighsBasisdNSt3__16vectorIdNS6_9allocatorIdEEEESA_iii at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN21HighsPrimalHeuristics15rootReducedCostEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN18HighsMipSolverData16evaluateRootNodeEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN14HighsMipSolver3runEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN5Highs12callSolveMipEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
_ZN5Highs5solveEv at /Users/oscar/.julia/artifacts/c167241db7de018a8569de84fb9cb4fe2e5c16b9/lib/libhighs.1.9.dylib (unknown line)
Highs_run at /Users/oscar/.julia/dev/HiGHS/src/gen/libhighs.jl:364 [inlined]
optimize! at /Users/oscar/.julia/dev/HiGHS/src/MOI_wrapper.jl:2158 |
The MPS file: log2.txt (download and rename) using HiGHS
highs = Highs_create()
Highs_readModel(highs, "/tmp/hypatia/log2.mps")
ret = Highs_run(highs)
index = Int32[160, 210, 211]
value = [0.027716058468715117, 0.09140473855744354, -0.14285714401611166]
ret = Highs_addRow(highs, 0.0, Inf, 3, index, value)
index = Int32[168, 210, 212]
value = [0.08645287577455307, -0.07110776325022987, -0.14285714401611166]
ret = Highs_addRow(highs, 0.0, Inf, 3, index, value)
index = Int32[176, 210, 213]
value = [0.18014292280865904, -0.17598648905497463, -0.14285714401611166]
ret = Highs_addRow(highs, 0.0, Inf, 3, index, value)
col_value = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 1.0, 1.0, 0.0, 0.10000000000000003, 0.09999999999999998, 0.10000000000000002, 0.0999999999999997, 0.09999999999999999, 0.10000000000000003, 0.2000000000000001, 0.10000000000000005, 0.10000000000000002, 6.143958570009227e-17, 0.08989012769442448, 0.07220932128680141, 0.0352521601580178, 0.023049307836325993, 0.03596218790993544, 0.0883353771762734, 0.10997880021778395, 0.08587975576854548, 0.04279544275988975, 1.7030821197749901e-12, 0.004234421424524501, 0.002935771224649386, 0.04550112833408798, 0.02302043831686157, 0.025235813220781028, 0.031938549301744945, -0.07353058737711736, -0.05472034203659071, 0.05343311365435198, 3.4280772664985193e-12, -0.04689131669823902, 0.018029419466667973, 0.01956701067655975, 0.0062964331045876065, 0.037005089525556284, -0.006460324867265374, -0.020043369079720735, 0.03319608436924963, -0.01006043021655742, -1.2794210135780304e-12, -0.013800411699883858, 0.023655375566162318, 0.030200785352213905, -0.0009828992876134632, -0.028860357902237788, 0.007061242560819414, 0.01993373656745131, -0.016113299457449673, -0.01887629925630621, -5.219297216640939e-13, 0.011085029717693151, 0.012312972942049333, -0.007674787911775963, 0.06616491552902533, -0.018421196580493633, -0.02113278323984752, -0.020275039522451258, 0.009213007692458558, -0.00670884182608282, 1.2446571551194552e-12, -0.01047846128625865, 0.027615408365906036, -0.01601079047024813, -0.0007079982072612095, 0.008281915481510214, -0.022871049160056056, 0.02955442470181519, -0.017186391929977243, 0.025746299517474966, 8.977887877570367e-13, -0.020636049811552537, -0.04204880656145055, 0.01092714816697224, 0.029645278088322957, 0.011995671961611207, 0.007319149900961663, 0.0624020670144705, -0.010164996744226464, 0.004354891135213923, 3.274769344585593e-12, 0.08080235371546947, 0.052141863954588495, 0.012427151107469675, 0.005312709067068595, 0.012932792742076099, 0.07803139175514197, 0.06047668563537779, 0.07375332765524344, 0.018314502359284238, 6.305953906297738e-9, 0.0001793042707088809, 8.618854954448377e-5, 0.02070352781804701, 0.005299406825345989, 0.006368463711423901, 0.010200710337094614, 0.027033737421798376, 0.029943159346376553, 0.028550977368757616, 2.0522193188881523e-9, 0.021987956271258755, 0.0032506001191893764, 0.003828679524191073, 0.00039645115480422025, 0.013693766963023794, 0.0004173584303198368, 0.002008683676652387, 0.011019800630026843, 0.0010121230178007778, 9.188495105156244e-10, 0.001904513894941692, 0.005595768195369511, 0.009120874622153843, 9.661174350469305e-6, 0.008329202845747051, 0.0004986117292101184, 0.00198676953182457, 0.0025963844580329143, 0.0035631469999948882, 5.344029005644164e-10, 0.0012287793004723392, 0.0015160934887099237, 0.0005890241569749031, 0.043777960927599543, 0.0033934052963916226, 0.004465945736274884, 0.0020553866002056657, 0.0008487955694550754, 0.00045008604854832377, 9.301992323621557e-10, 0.001097981753823956, 0.007626108035892825, 0.0025634543591829396, 5.012859288031187e-6, 0.0006859014850380895, 0.005230849140907799, 0.0043673203416162145, 0.0029537209201066237, 0.006628719632447226, 4.95296736502626e-10, 0.0042584659500023965, 0.017681021763164266, 0.0011940261027683563, 0.00878842556104972, 0.0014389618902403267, 0.0005356999849043904, 0.01947009026926299, 0.0010332720203051681, 0.0001896512002343511, 8.711881395423164e-10, 0.39419278744899733, 0.2825736006925665, 0.23393675981842824, 0.3693796862871027, 0.23954835023091225, 0.26233937933520035, 0.2579261079888629, 2.7755575615628914e-17, 0.12836547872533643, 0.0554170117731227, -0.030113498666962377, 0.0071677469163563545, 0.02091717775616718, 0.012134053229346381, -3.469446951953614e-17, 5.551115123125783e-17, 0.057615421163861466, 0.011792497523566349, 0.06510468420691196, -0.006857554509634614, 0.02848542734578438, 2.7755575615628914e-17, -2.7755575615628914e-17, -1.3877787807814457e-17, 0.0336049342515613, 0.011119217951726046, 0.003986818070404647, 0.008967713393544907, 4.85722573273506e-17, 0.0, 3.469446951953614e-18, 5.551115123125783e-17, 0.05832547851826049, 0.0074414186342552785, -0.024644296226585816, 3.122502256758253e-17, 0.0, 6.938893903907228e-18, 0.0, 1.3877787807814457e-17, 0.031159069269547358, -0.01612106260584778, 3.8163916471489756e-17, 0.0, 2.0816681711721685e-17, -2.0816681711721685e-17, 2.7755575615628914e-17, -6.938893903907228e-17, 0.05458961604663723, 0.07167683009560058, 0.07167683017667047, 0.12218552502002347, 0.04176708897942017, -0.015652591601132956, -0.05429481444959562, -0.014774639181157226, -0.05971125745361597, -0.01951931131394186]
ret = Highs_setSolution(highs, col_value, C_NULL, C_NULL, C_NULL)
ret = Highs_run(highs)
Highs_destroy(highs) Yields
|
Closing in favor of the upstream ERGO-Code/HiGHS#2109 |
Hi all,
I'm trying to solve some MISOCP with Pajarito using Hypatia as conic solver and HiGHS as MIP solver, see the attached files.
Weird_HiGHS_error.txt
spectral_functions_JuMP.txt
It works for some instances but for others I get a weird error, see
error_message.txt .
It seems to me that some HiGHS library files are not found. This also completely crashes my terminal, as in both Ctrl + C as well the ESC don't work.
Additionally, this happens on both my linux as well as my windows computer.
I'm on Julia 1.9.0 and have the most up-to-date HiGHS version.
Thank you for your help!
The text was updated successfully, but these errors were encountered: