Implement Cartpole

[dtch1997]

Hi there @dawsonc , I've got a mostly functional version of Cartpole which I believe would be a nice addition to this repo, and I would like to request some help with merging it in.

I have implemented Cartpole following these equations of motion, which are also used by the OpenAI gym implementation.

After some algebraic manipulation, I have derived the control-affine form of the equations implemented in neural_clbf/systems/cartpole.py. Running the file checks that the closed-loop dynamics derived from these equations is identical to the full dynamics for several randomly-sampled states and controls.

I have implemented an associated training script, however I invariably get issues with infeasible QPs as follows:

$ python neural_clbf/training/train_cartpole.py --n-epochs 1
...
    raise SolverError("Solver scs returned status %s" % status)
diffcp.cone_program.SolverError: Solver scs returned status infeasible

I'm struggling to figure out the cause and a bit of advice would be much appreciated. Thanks very much!

[dtch1997]

Reducing the state limits seems to have fixed it somewhat. Experiment ongoing, viewable at https://wandb.ai/dtch1997/NeuralCLBF/runs/th3qiqh6?workspace=user-dtch1997

[dawsonc]

Thanks for your interest in contributing to the project!

I'm happy to support merging this system in, but I'd first want to see some evidence that you can get training working with the new model (for at least one of the types of neural certificate, e.g. neural cbf or clbf, and bonus points for extra demonstrations). For posterity, would you mind attaching the plots and simulation results showing successful training to this PR (when available).

PS I get a 404 at that link you provided.

[dtch1997]

Thanks for getting back to me!

Sorry about the messy state of the PR. I went somewhat overboard trying various changes to improve performance which didn't really pan out.

I'll revert to the working version and fix the WandB link soon.