Memory allocation failure in benchmark example IAR + KW24D

[mazimkhan]

Error:

RSA-4096                 :  FAILED: RSA - The private key operation failed : BIGNUM - Memory allocation failed

IAR 7.8
KW24D
CI log https://jenkins-internal.mbed.com/job/mbedtls-mbed-os-examples-pr/75/execution/node/592/log/
mbed-os 5.7

@MarceloSalazar please check.

[MarceloSalazar]

@maclobdell would you like to review this for the KW24D platform?

[maclobdell]

@bentcooke - did you recently make a heap size change to get KW24D working in another example? Perhaps that could be relevant to this issue. Please share any details you have.

[bentcooke]

@maclobdell I had to reduce the heap size (20k->15k) and move the nanostack heap into the system heap ("mbed-mesh-api.use-malloc-for-heap": true,) in order to get the mesh minimal example working on the KW24D.

[ciarmcom]

ARM Internal Ref: IOTSSL-2427

[k-stachowiak]

The impact of this issue has increased by one more test failing to allocate memory on the heap. Currently there are two allocation failures in the BIGNUM module:

...
RSA - The private key operation failed : BIGNUM - Memory allocation failed
...
DHM - Making of the public value failed : BIGNUM - Memory allocation failed
...

[simonbutcher]

I understand the underlying issue is there's not enough heap available on this target for the test, which has been raised as issue #8306.

We've got no way of externally determining if the target can support the test or not, so I'd like to propose we change the benchmark to only test RSA-2048, and skip RSA-4096. RSA-4096 is demanding in memory and in my view it's better to disable the test until #8306 is fixed, which is open ended and has no immediate resolution.

Equally, I suggest we disable the Diffie-Hellman tests. They're not part of the default configuration of Mbed TLS in Mbed OS, so why run them? The lines enabling them in the config.h file, can just be removed.

[andresag01]

@sbutcher-arm

I understand the underlying issue is there's not enough heap available on this target for the test

I think that (strictly speaking) this is not the issue. As I explained here and here, the problem is that the IAR linker script for that target sets the heap size to just 16KB, which is too small for this application. However, the devices does have enough memory (64K) to run the application. Unfortunately, the allocator used by the IAR libc does not allow the heap to grow and shrink dynamically at runtime and instead is fixed at compile-time, so we get those allocation failures while is succeeds in other targets.

As the Mbed OS team suggested (see internal ticket IOTSSL-2427 and GitHub) is that we just change the linker script for the IAR target in Mbed OS as a temporary fix while they roll out automatic generation of linker scripts that will enable us to set different heap sizes for each application and solve the problem.

I'd like to propose we change the benchmark to only test RSA-2048, and skip RSA-4096... I suggest we disable the Diffie-Hellman tests

Sure, that would also make the test pass by not running the problematic code. I personally like testing/running the other primitives using this application, but I don't mind if we just remove it....

[simonbutcher]

I think that (strictly speaking) this is not the issue

I was trying to summarise the issue in one line with less than 20 words, but thanks for the clarification anyway. ;)

[simonbutcher]

@maclobdell / @MarceloSalazar / @ashok-rao - We think we've fixed this issue with PR #213, but we don't have the KW24D board in our CI, nor one available to test.

Could one of you please confirm this is fixed? It may well have been tested in the last OOB run for 5.11, so might not even need a new test.

[maclobdell]

Yes, I can test it.

With respect to: "we don't have the KW24D board in our CI"
@OPpuolitaival @studavekar - Can you provide visibility to the Mbed ecosystem (developers, partners, customers) about which platforms are currently tested in the Mbed OS CI. It would also be good to have a list of Mbed 5 Enabled boards that aren't tested in the Mbed OS CI and why. As part of Mbed Enabled, we receive 10 boards from partners, so board availability should not be an issue.

[OPpuolitaival]

@maclobdell Test results are easiest place to see what we really run in CI. For example: http://mbed-os-ci.s3-website-eu-west-1.amazonaws.com/?prefix=jenkins-ci/artifacts/9250/1/

There are activities ongoing to add more boards on nightly tests. We are also integrating Austin and Oulu labs working together. If you can provide 4 pieces of those in Oulu or Austin labs we can start trying how well those works in CI.

[maclobdell]

Looks like KW24D is in the Mbed OS CI. http://mbed-os-ci.s3-eu-west-1.amazonaws.com/jenkins-ci/artifacts/9250/1/greentea-test/696/PASS/KW24D/IAR/report_test_greentea_KW24D_IAR.html. I'm assuming the Mbed TLS benchmark tests are not run in regular Mbed OS CI jobs. Is there a separate job that can run the benchmark tests on the same boards that are in Mbed OS CI? I'm happy to run it manually, but it will take some time and it would be nice to have it automated.

[cmonr]

@maclobdell The most up to date list of greentea CI tests can be found here: https://github.com/ARMmbed/mbed-os-ci/blob/master/mbed-os-jenkins/mbed-os-ci-greentea-test/Jenkinsfile.groovy#L44 (private repo).

Afaik, the CI team has tickets for all of the tests that aren't enabled and/or the stability issues are being worked out. @OPpuolitaival can confirm.