Memory leaks

[Coderitter-GmbH]

There are multiple memory leaks, which needs to be removed

[alrevuelta]

Yes, agree that we have some memory leaks. How did you find them? Unexpected crashes or were you using some tool like valgrind? Could you list the ones that you are aware of?

[Coderitter-GmbH]

The hardware the library is running on has very limited ram (2 megabytes). It was running out of memory very quickly. For the detection of leaks i used the gcc address sanitizer. I have already some fixes but i wanted to discuss one more aspect. The memory used for an inference can be freed while the inference is running, to reduce the memory foot print. Only the results of prior node is needed to calculate the actual node.

[alrevuelta]

Could you provide the commands you used for the gcc address sanitizer? In the future some kind of memory leak detection should be included in the GitHub Action CI.

Regarding freeing memory from past nodes, I agree that there are some optimisations that can be made, but note that its not that straight forward. A model like mnist is a set of nodes connected in series to each other, so freeing the memory from the previous node output will work just fine, but there are more complex models that contain "branches" and "Only the results of prior node is needed to calculate the actual node." won't be true.

So we would need some kind of algorithm that scans the model an decides which outputs can be freed, and which ones will be needed later on in a different node. Having a flag with this choice (free unused outputs vs keep them) would be a really nice future, but imho it needs a bit of thinking. Feel free to open a new issue with this feature request :)

[Coderitter-GmbH]

You can compile the software with the additional gcc flag -fsanitize=address. This will enable the address sanitizer, which detect out-of-bounds and use-after-free bugs and memory leaks.