Error report from th.LevenbergMarquardt() in PyPI's theseus

[LiuxhRobotAI]

❓ Questions and Help

Hi. Thanks for the great library. I was running simple_example.py in the examples for curve fitting. I meet an error when I want to replace th.GaussNewton() with th.LevenbergMarquardt() method. Here are the details.

1. For th.GaussNewton(), everything looks fine.
   The example code in simple_example.py is shown as:

layer = th.TheseusLayer(th.GaussNewton(objective, max_iterations=10))

I can get a result using th.GaussNewton() method as follows:

(theseuspy310) <USER_Ubuntu_20.04>:~/<PATH_TO>/demo$ /<PATH_TO>/miniconda3/envs/theseuspy310/bin/python /<PATH_TO_PROJECT>/demo/simple_example.py
Outer loss:  0.0022639785893261433    Value:  0.452418714761734
Outer loss:  0.0022211121395230293    Value:  0.45287132263183594
Outer loss:  0.002180063398554921    Value:  0.4533088505268097
Outer loss:  0.0021396512165665627    Value:  0.4537436366081238
Outer loss:  0.0020994667429476976    Value:  0.4541800618171692
Outer loss:  0.002059338614344597    Value:  0.4546200633049011
Outer loss:  0.0020191799849271774    Value:  0.45506471395492554
Outer loss:  0.001978947315365076    Value:  0.4555146396160126
Outer loss:  0.001938618253916502    Value:  0.45597025752067566
Outer loss:  0.0018981901230290532    Value:  0.4564317762851715

With th.LevenbergMarquardt(), I get the following error:

(theseuspy310) <USER_Ubuntu_20.04>:~/<PATH_TO>/demo$ /<PATH_TO>/miniconda3/envs/theseuspy310/bin/python /<PATH_TO_PROJECT>/demo/simple_example.py
Traceback (most recent call last):
  File "/<PATH_TO_PROJECT>/demo/simple_example.py", line 78, in <module>
    main()
  File "/<PATH_TO_PROJECT>/demo/simple_example.py", line 52, in main
    optimizer = th.LevenbergMarquardt(
  File "/<PATH_TO>/miniconda3/envs/theseuspy310/lib/python3.10/site-packages/theseus/optimizer/nonlinear/levenberg_marquardt.py", line 69, in __init__
    super().__init__(
  File "/<PATH_TO>/miniconda3/envs/theseuspy310/lib/python3.10/site-packages/theseus/optimizer/nonlinear/nonlinear_least_squares.py", line 73, in __init__
    if objective._error_metric_fn is not error_squared_norm_fn:
AttributeError: 'NoneType' object has no attribute '_error_metric_fn'
(theseuspy310) <USER_Ubuntu_20.04>:/<PATH_TO>/demo$

The same error exists when I was running examples/pose_graph/pose_graph_benchmark.py, one example in theseus also using th.LevenbergMarquardt() method.

Here is my code in simple_example.py. I was using th.TheseusLayer(), to replace th.GaussNewton():

    # layer = th.TheseusLayer(th.GaussNewton(objective, max_iterations=10))
    optimizer = th.LevenbergMarquardt(
        objective.to(torch.float64),
        max_iterations=10,
        step_size=1,
        linearization_cls=th.SparseLinearization,
        linear_solver_cls=th.CholmodSparseSolver,
        vectorize=True,
    )
    layer = th.TheseusLayer(optimizer)

Here is my environment. I am working on a Ubuntu 20.04, and Python=3.10.
I installed theseus from PyPI using pip install theseus-ai according to README.

(theseuspy310) <USER_Ubuntu_20.04>:/<PATH_TO>/demo$ pip list |grep theseus
theseus-ai                    0.2.1
(theseuspy310) <USER_Ubuntu_20.04>:/<PATH_TO>/demo$
(theseuspy310) <USER_Ubuntu_20.04>:/<PATH_TO>/demo$ pip list |grep torch
torch                         2.1.2
torchaudio                    2.1.2
torchkin                      0.1.1
torchlie                      0.1.0
torchvision                   0.16.2
(theseuspy310) <USER_Ubuntu_20.04>:/<PATH_TO>/demo$

Is this problem caused by th.TheseusLayer(), as seems it returns a 'NoneType' object? Could someone provide some help?

[luisenp]

Hi @L53317. I'm really sorry for the long response time, somehow I lost track of this issue for too long. Thanks for reporting this!

Are you still having this error? I'm not able to reproduce this error with pypi install and latest main.

(theseus) code/theseus/ - (main) > grep Levenberg examples/simple_example.py
layer = th.TheseusLayer(th.LevenbergMarquardt(objective, max_iterations=10))
(theseus) code/theseus/ - (main) > python examples/simple_example.py
Outer loss:  0.0022639816161245108
Outer loss:  0.0022211093455553055
Outer loss:  0.0021800606045871973
Outer loss:  0.002139648422598839
Outer loss:  0.002099463948979974
Outer loss:  0.0020593360532075167
Outer loss:  0.0020191771909594536
Outer loss:  0.001978941960260272
Outer loss:  0.0019386130152270198
Outer loss:  0.0018981823232024908
Outer loss:  0.001857658731751144
Outer loss:  0.0018170438706874847
Outer loss:  0.0017763579962775111
Outer loss:  0.0017356161260977387
Outer loss:  0.0016948339762166142
Outer loss:  0.0016540373908355832
Outer loss:  0.001613246975466609
Outer loss:  0.0015724883414804935
Outer loss:  0.0015317798824980855
Outer loss:  0.0014911541948094964
(theseus) code/theseus/ - (main) > conda list | grep theseus
# packages in environment at XXXXXXXXXXXXXXXXXXXXXXXXXXX
theseus-ai                0.2.1                    pypi_0    pypi
(theseus) code/theseus/ - (main) > git log --oneline -n 1
e075691 (HEAD -> main, origin/main, origin/HEAD) Fix track best solution in nonlinear optimizer with matrix vars and implicit diff (#637)
(theseus) code/theseus/ - (main) >

[LiuxhRobotAI]

Hi @luisenp, thanks for your kind reply. Your solution works directly for examples/simple_example.py, which is:

layer = th.TheseusLayer(th.LevenbergMarquardt(objective, max_iterations=10))

Furthermore, I can run simple_example.py by replacing objective.to(torch.float64) with objective in this file, which is given as follows:

    # layer = th.TheseusLayer(th.GaussNewton(objective, max_iterations=10))
    optimizer = th.LevenbergMarquardt(
        objective,
        max_iterations=10,
        step_size=1,
        linearization_cls=th.SparseLinearization,
        linear_solver_cls=th.CholmodSparseSolver,
        vectorize=True,
    )
    layer = th.TheseusLayer(optimizer)

I am not sure if replacing objective.to(dtype) with objective would work for examples/pose_graph/pose_graph_benchmark.py or not, as my CUDA can not be recognized by thesues. But I may consider the previous problem was solved, and the new problem is not caused by th.LevenbergMarquardt() anymore as when I replace it in pose_graph_benchmark.py, thesues only reports an error about CUDA.

Anyway, thanks for your reply, it does solve my problem. I will test pose_graph_benchmark.py after I solve the problem caused by CUDA and then see if everything works well.

Feel free to close this issue.

[luisenp]

Ah. I think the problem is because to() function stopped returning None only in recent versions of Theseus that we haven't yet pushed to pypi. I'll try to push an update in the next few days. Thanks for reporting this.

[LiuxhRobotAI]

Hi @luisenp, thanks for your kind reply. Your solution works directly for examples/simple_example.py, which is:

layer = th.TheseusLayer(th.LevenbergMarquardt(objective, max_iterations=10))

Furthermore, I can run simple_example.py by replacing objective.to(torch.float64) with objective in this file, which is given as follows:

    # layer = th.TheseusLayer(th.GaussNewton(objective, max_iterations=10))
    optimizer = th.LevenbergMarquardt(
        objective,
        max_iterations=10,
        step_size=1,
        linearization_cls=th.SparseLinearization,
        linear_solver_cls=th.CholmodSparseSolver,
        vectorize=True,
    )
    layer = th.TheseusLayer(optimizer)

I am not sure if replacing objective.to(dtype) with objective would work for examples/pose_graph/pose_graph_benchmark.py or not, as my CUDA can not be recognized by thesues. But I may consider the previous problem was solved, and the new problem is not caused by th.LevenbergMarquardt() anymore as when I replace it in pose_graph_benchmark.py, thesues only reports an error about CUDA.

Anyway, thanks for your reply, it does solve my problem. I will test pose_graph_benchmark.py after I solve the problem caused by CUDA and then see if everything works well.

Feel free to close this issue.

I have checked objective for examples/pose_graph/pose_graph_benchmark.py again, and it works well right now. Looking forward to your new version of Theseus.