From 6be8b09c214b8427c48cbf74f3914d562eb51e07 Mon Sep 17 00:00:00 2001 From: jarsi Date: Fri, 10 Dec 2021 14:27:47 +0100 Subject: [PATCH 1/5] Update readme --- README.md | 45 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 296521b..102549f 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,46 @@ # benchmark-models -A collection of models adapted to interface with the [nest benchmarking framework](https://github.com/INM-6/nest_benchmarking_framework). Here, documentation on how to interface a new model is provided (README, section _Developing the framework_). \ No newline at end of file +A collection of models adapted to interface with the [nest benchmarking framework](https://github.com/INM-6/nest_benchmarking_framework). Here, documentation on how to interface a new model is provided (README, section _Developing the framework_). + + + +### Input + +In order to include a model to the JUBE workflow it needs to be able to receive input from the JUBE benchmark file. The easiest way is to use placeholders in the simulation script which are substituted with values defined in the corresponding JUBE file. For example the hpc\_benchmark.yaml contains this substituteset: + +``` +substituteset: + name: model_substitutions + iofile: {in: hpc_benchmark.py, out: hpc_benchmark.py} + sub: + - {source: "{num_vps}", dest: $num_vps} + - {source: "{record_spikes}", dest: $record_spikes} + - {source: "{model_time_sim}", dest: $model_time_sim} + - {source: "{model_time_presim}", dest: $model_time_presim} + - {source: "{N_SCALING}", dest: $scale} + - {source: "{rng_seed}", dest: $rng_seed} +``` + +This substitueset is catched like this in the hpc\_benchmark.py simulation file: + +``` +params = { + 'nvp': {num_vps}, # total number of virtual processes + 'scale': {N_SCALING}, # scaling factor of the network size + # total network size = scale*11250 neurons + 'simtime': {model_time_sim}, # total simulation time in ms + 'presimtime': {model_time_presim}, # simulation time until reaching equilibrium + 'dt': 0.1, # simulation step + 'record_spikes': {record_spikes}, # switch to record spikes of excitatory + # neurons to file + 'rng_seed': {rng_seed}, # random number generator seed + 'path_name': '.', # path where all files will have to be written + 'log_file': 'logfile', # naming scheme for the log files +} +``` + +Depending on the usecase and model specifications this substituteset can be extended, as done for example in the microcircuit. A model might require a different way of providing the input, see for example the multi-area model benchmark where the input is passed via command line arguments. + +### Placement of Python level timers + +### Output From ae4ac4e98b2af5a773c5d78da8814fa30c831f75 Mon Sep 17 00:00:00 2001 From: jarsi Date: Fri, 10 Dec 2021 16:13:06 +0100 Subject: [PATCH 2/5] Update README --- README.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index d83cd3a..1f9a105 100644 --- a/README.md +++ b/README.md @@ -53,11 +53,16 @@ On the python level several blocks of code are measured. These blocks are timed * This is the actual simulation phase. This timer is a wrapper around `nest.Simulate(t_sim)` Furthermore it is useful to write out some information of memory consumption. -* network_memory + +* base\_memory + * Used memory before any NEST related operation has been done. +* node\_memory + * Used memory after Creation of all nodes. +* network\_memory * Used memory after Connection of all nodes. -* init_memory +* init\_memory * Used memory after Presimulation. -* total_memory +* total\_memory * Used memory after Simulation has finished. ### Output From af351ba7d6116ccc9c7b46e90350a31bd0b35de4 Mon Sep 17 00:00:00 2001 From: jarsi Date: Fri, 10 Dec 2021 16:16:03 +0100 Subject: [PATCH 3/5] Change wording --- README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 1f9a105..9e320ed 100644 --- a/README.md +++ b/README.md @@ -55,15 +55,15 @@ On the python level several blocks of code are measured. These blocks are timed Furthermore it is useful to write out some information of memory consumption. * base\_memory - * Used memory before any NEST related operation has been done. + * Memory consumption before any NEST related operation has been done. * node\_memory - * Used memory after Creation of all nodes. + * Memory consumption after Creation of all nodes. * network\_memory - * Used memory after Connection of all nodes. + * Memory consumption after Connection of all nodes. * init\_memory - * Used memory after Presimulation. + * Memory consumption after Presimulation. * total\_memory - * Used memory after Simulation has finished. + * Memory consumption after Simulation has finished. ### Output From c13b7e93e9e76f52c6b0b2ccdcf602669d13cfb2 Mon Sep 17 00:00:00 2001 From: jasperalbers Date: Fri, 10 Dec 2021 22:00:37 +0100 Subject: [PATCH 4/5] review suggestions --- README.md | 68 ++++++++++++++++++++++++++++--------------------------- 1 file changed, 35 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index 9e320ed..0b9aa1b 100644 --- a/README.md +++ b/README.md @@ -1,27 +1,12 @@ -# benchmark-models +# beNNch-models -A collection of models adapted to interface with the [nest benchmarking framework](https://github.com/INM-6/nest_benchmarking_framework). An overview on how to interface a new model is provided [here](https://github.com/INM-6/beNNch#developer-guide). A more detailed description is provided below. +A collection of models adapted to interface with [beNNch](https://github.com/INM-6/beNNch). An overview on how to interface a new model can be found [here](https://github.com/INM-6/beNNch#developer-guide). A more detailed description is provided below. ### Input -In order to include a model to the JUBE workflow it needs to be able to receive input from the JUBE benchmark file. The easiest way is to use placeholders in the simulation script which are substituted with values defined in the corresponding JUBE file. For example the hpc\_benchmark.yaml contains this substituteset: +In order to include a model to the JUBE workflow it needs to be able to receive input from its JUBE benchmark file. We recommend using placeholders in the simulation script which can be substituted by JUBE with values defined in the corresponding JUBE config file. For example, the [simulation file of the HPC-benchmark](./hpc_benchmark/hpc_benchmark.py) contains the following placeholders: -``` -substituteset: - name: model_substitutions - iofile: {in: hpc_benchmark.py, out: hpc_benchmark.py} - sub: - - {source: "{num_vps}", dest: $num_vps} - - {source: "{record_spikes}", dest: $record_spikes} - - {source: "{model_time_sim}", dest: $model_time_sim} - - {source: "{model_time_presim}", dest: $model_time_presim} - - {source: "{N_SCALING}", dest: $scale} - - {source: "{rng_seed}", dest: $rng_seed} -``` - -This substitueset is catched like this in the hpc\_benchmark.py simulation file: - -``` +```python params = { 'nvp': {num_vps}, # total number of virtual processes 'scale': {N_SCALING}, # scaling factor of the network size @@ -37,34 +22,51 @@ params = { } ``` -Depending on the usecase and model specifications this substituteset can be extended, as done for example in the microcircuit. A model might require a different way of providing the input, see for example the multi-area model benchmark where the input is passed via command line arguments. +The corresponding `subsituteset` of the JUBE benchmarking file then looks as follows: + +```yaml +substituteset: + name: model_substitutions + iofile: {in: hpc_benchmark.py, out: hpc_benchmark.py} + sub: + - {source: "{num_vps}", dest: $num_vps} + - {source: "{record_spikes}", dest: $record_spikes} + - {source: "{model_time_sim}", dest: $model_time_sim} + - {source: "{model_time_presim}", dest: $model_time_presim} + - {source: "{N_SCALING}", dest: $scale} + - {source: "{rng_seed}", dest: $rng_seed} +``` +where `$` are parameters defined in either the model-specific config file or in the JUBE benchmarking script. + +Depending on the use case and model specifications this `substituteset` can be extended, as done for example in the microcircuit. A model might require a different way of providing the input, see for example the multi-area model benchmark where the input is passed via command line arguments. -### Python level measurement devices +### Python--level measurements -On the python level several blocks of code are measured. These blocks are timed via wrapping `time.time()` from the time library around these blocks of code. These timers are common: +In the provided models, several blocks of code are timed on the python level via wrappings of `time.time()` from the `time` library. These timers are common: * py\_time\_create - * This is the time that the creation of nodes (neuronal populations, devices) and the setting of the initial membrane potentials takes. + * Time for creation of nodes (neuronal populations, devices) and setting of initial membrane potentials. * py\_time\_connect - * This is the time that connecting nodes with each other, devices with nodes and preparing the connection infrastructure (i.e. call to `nest.Prepare()`) take. + * Time for connecting nodes with each other, devices with nodes and preparing the connection infrastructure (i.e., calls to `nest.Prepare()`). * py\_time\_presimulate - * Time that the presimulation phase takes. This is the first call to `nest.Simulate(t_presim)` which simulates the network for a short amount of time until it reaches its stable state. + * Time for the pre-simulation phase. This is defined as the first call to `nest.Simulate(t_presim)` which simulates the network for a short amount of time until it reaches its stable state. * py\_time\_simulate - * This is the actual simulation phase. This timer is a wrapper around `nest.Simulate(t_sim)` + * Time for the actual simulation phase. This timer is a wrapper around `nest.Simulate(t_sim)` -Furthermore it is useful to write out some information of memory consumption. +Furthermore, it is useful to record some information on memory consumption. * base\_memory - * Memory consumption before any NEST related operation has been done. + * Memory consumption before any NEST related operations have been executed. * node\_memory - * Memory consumption after Creation of all nodes. + * Memory consumption after creation of all nodes. * network\_memory - * Memory consumption after Connection of all nodes. + * Memory consumption after connection of all nodes. * init\_memory - * Memory consumption after Presimulation. + * Memory consumption after pre-simulation. * total\_memory - * Memory consumption after Simulation has finished. + * Memory consumption after the simulation has finished. ### Output -The timing and memory data mentioned above should be written out to logging files by every task to task private files. Furthermore a complete dump of the nest Kernel status should be written out to this file. This data can be accessed via `nest.GetKernelStatus()`. +The timing and memory data mentioned above should be written out to process-private logging files by every process. `beNNch` takes care of combining those to single result files by averaging (or summing) the recorded quantities. +Furthermore, a complete dump of the NEST KernelStatus should be written out to file to keep track of simulation metadata. This data can be accessed via `nest.GetKernelStatus()`. From 3e67151ce536a93a61990ca4a6fc2165d062a60a Mon Sep 17 00:00:00 2001 From: jasperalbers Date: Fri, 10 Dec 2021 22:12:24 +0100 Subject: [PATCH 5/5] add link to hpc-benchmark JUBE file --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 0b9aa1b..cbcdb4a 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,7 @@ params = { } ``` -The corresponding `subsituteset` of the JUBE benchmarking file then looks as follows: +The corresponding `subsituteset` of the [JUBE benchmarking file](https://github.com/INM-6/beNNch/blob/main/benchmarks/hpc_benchmark_31.yaml) then looks as follows: ```yaml substituteset: