Gurobi Output Typing Errors

[Foggalong]

Pylance reports that Gurobi is sometimes returning a float value when a numpy.ndarray is expected.

Background

The solver functions have appropriate numpy.typing return types, specified for example as

gurobi_standard_genetics(...) -> tuple[npt.NDArray[np.float64], float]:

and then returned with

return w.X, model.ObjVal

where w.X and model.ObjVal are the outputs from Gurobi.

Issue

Sometimes, but not always, Pylance flags that Gurobi is returning a float type object for w.X, producing an error like the below.

Expression of type "tuple[float, float]" cannot be assigned to return type "tuple[NDArray[float64], float]"
  "float" is incompatible with "NDArray[float64]"Pylance
(constant) X: float

When w.X is examined though it's clearly a numpy.ndarray, and type(...) returns what we'd expect, so I can only assume either:

1. Pylance is incorrectly interpreting MVar.X,
2. Gurobi is misreporting the type of MVar.X (note this was only even added in v11)
3. I've setup the of MVar's type incorrectly for w

[Foggalong]

I found the source of this issue, turned out to be in Gurobi! In their documentation for the MVar.X attribute they say that its type (regardless of language used) is double.

X

Type: double
Modifiable: No
Variable value in the current solution.

When MVar is used in practice with the shape or size used to create a matrix variable though (in Python, at least) MVar.X does attain the correct NDArray type. This creates the conflict with typing which Pylance detects.

Unfortunately this means the issue can't be fixed on our end, it's just a case of Gurobi's typing not being mature yet. We've used the correct typing which is what matters as far as the interpreter is concerned, though there may be some false flag errors produced like this which might make debugging harder than it would be otherwise.

Example

To see an example of this, take Gurobi's matrix1.py example and modify the solution print statement to the following.

print(type(x.X), x.X)
print(f"Obj: {m.ObjVal:g}")

The output when running will then end with

Optimal solution found (tolerance 1.00e-04)
Best objective 3.000000000000e+00, best bound 3.000000000000e+00, gap 0.0000%
<class 'numpy.ndarray'> [1. 0. 1.]
Obj: 3

showing that x.X attained the correct NDArray type. At the same time, Pylance reports that the type of x.X should be

(constant) X: float

which highlights the conflict, and subsequent error which would occur if typing was enforced.

[jajhall]

Well done on getting to the bottom of this. Gurobi's Python interface is clearly not mature.

[Foggalong]

Do you know how they handle typing through other language's APIs? Curious why their choice of type for MVar.X was double (cross-language) when it's an attribute that conceivably could be int, array, etc of varying size and precision.

[jajhall]

In C, C++ or Fortran, what's passed is (a pointer to) the memory address of an object of a particular type. When compiled, the types on both the call and method have to match. In C and C++ (at least) there are subtleties like "pass by reference" (so changes in what's passed affect the calling method) and "pass by value" where changes aren't passed back. Passing by reference isn't possible in Python, which may say something about the disconnect between the calling and called code. Modified values are passed back in the return value - that may be a tuple

[Foggalong]

I see, makes sense that the same issue isn't in those languages because it's all pointers and references.

This isn't a priority issue now so I won't go down the rabbit hole searching for a fix, but off the back of what you've said I did just have a quick look how this is handled in the Java API. It has typing but (if my limited Java serves me right), doesn't have pointers, so was curious how it was handled there.

In their Java QP example, they define the model variables using

GRBVar[] vars = model.addVars(lb, ub, null, vtype, null);

and then after model.optimize(); access the solution using

for (int j = 0; j < cols; j++)
    vars[j].get(GRB.DoubleAttr.X);

In other words, it seems like when the variable is initialised the API creates an NDArray[MVar] type object. This is contrary to the Python MIP example which has a single MVar with type NDArray[float].

The Python behaviour seems more intuitive, but I wonder whether the Python API expects similar typing behaviour to Java, but (in Pythonic fashion) works regardless which way round the model variables are typed. Given how recently Gurobi added typing too it makes sense that the Python example might be inconsistent with the current API behaviour.