Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] run_regularly operations on the CPU #97

Merged
merged 7 commits into from
Nov 7, 2019
Merged

Conversation

mstimberg
Copy link
Member

This PR still needs a bit more thorough testing, but it seems to work in principle. run_regularly operations are now always executed on the CPU, which comes with a number of advantages over the previous solution:

  • Functions that are only available on the CPU can be used (most importantly, TimedArray Timed arrays #13).
  • Synapses can have run_regularly operations as well (Fixes Support run-regularly for synapses #58)
  • More than one run_regulary operation per object allowed (not sure how relevant this is in practice, though).

If the user has a run_regularly operation that gets executed every time step, then the new approach will probably slow the simulation down a bit. For the more standard use case of a rarely executed operation, I don't think we'll see any difference.

Fixes #96.

Copy link
Contributor

@tnowotny tnowotny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Marcel, in principle very nice and tidy. However, unless I misunderstand (which is a definitive possibility), there is a lot of unnecessary pushing and pulling to and from the GPU going on? Can you clarify?

Comment on lines 201 to 204
{% for var in run_reg['write'] %}
{% if not run_reg['owner'].variables[var].owner.name in states_pushed %}
push{{run_reg['owner'].variables[var].owner.name}}StateToDevice();
{% if states_pushed.append(run_reg['owner'].variables[var].owner.name) %}{% endif %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this mean that you are pushing target variables of run_regularly operations every timestep, even if they have not changed because i % {{obj['step']}} == 0 was false?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hah, indeed! I guess I was so focussed on not pushing the same state twice per time step that I forgot the obvious :)

Comment on lines 240 to 249
{% for run_reg in run_regularly_operations %}
{% for var in run_reg['read'] %}
{% if not var in ['t', 'dt'] %}
{% if not run_reg['owner'].variables[var].owner.name in states_pulled %}
pull{{run_reg['owner'].variables[var].owner.name}}StateFromDevice();
{% if states_pulled.append(run_reg['owner'].variables[var].owner.name) %}{% endif %}
{% endif %}
{% endif %}
{% endfor %}
{% endfor %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't this also mean that you are pulling variables every timestep, regardless whether the run_regularly operations will take place in the next iteration? I wonder whether one should not test here whether the variable will be needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, actually it would make much more sense to put the pullStateFromDevice together with the actual execution of the operation. I think I got confused by all the state pulling that is going on here for the monitors, which is preparing things for the next time step as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the code I realized that while it is not intuitive to have the state pulling at the end of the time step, it allows us to not do it twice if the state has already been pulled for a state monitor. I'll leave it this way then and just add a check that the next time step will execute it.

@mstimberg
Copy link
Member Author

I think my last commit fixes the issues you identified, would be grateful if you could have another look.

{% if nm.run_regularly_object != None %}
if (i % {{nm.run_regularly_step}} == 0)
{% else %}
if (i % {{obj['step']}} == 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be sure - is the intended behaviour that run_regulary timing is based on the local iteration variable i? Rather than on the global integer time step iT? I can't remember how we determine riT and whether the loop would ever be executed multiple times?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We currently do not allow for multiple runs, so i and iT should always be identical. I think I prefer to use i here, since iT is a bit less straightforward (it gets updated as part of stepTimeGPU/stepTimeCPU, etc.).

Copy link
Contributor

@tnowotny tnowotny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think this fixes the issue very nicely. Now the question is - do we have any tests that cover some of the likely use cases? Do you have run-regularly tests in the Brian tests?

@mstimberg
Copy link
Member Author

Now the question is - do we have any tests that cover some of the likely use cases? Do you have run-regularly tests in the Brian tests?

We do have a couple of tests, e.g. a basic test for NeuronGroup:

    G = NeuronGroup(1, 'v : 1')
    G.run_regularly('v += 1', dt=2*defaultclock.dt)
    M = StateMonitor(G, 'v', record=0, when='end')
    run(10 * defaultclock.dt)
    assert_allclose(G.v[:], 5)
    assert_allclose(np.diff(M.v[0]), np.tile([0, 1], 5)[:-1])

and a similar one for Synapses:

    G = NeuronGroup(10, 'v : 1', threshold='False')
    tau = 10*ms
    S = Synapses(G, G, 'w : 1', on_pre='v += w')
    S.connect(j='i')
    S.run_regularly('w = i')
    run(defaultclock.dt)
    assert_allclose(S.w[:], np.arange(10))

I also specifically added a test that changes the order argument of StateMonitor and run_regularly which is mostly relevant for brian2genn (in standard Brian, you'd rather use scheduling via named slots which only has very basic support in brian2genn). They all pass (ignore all the failing tests for the ..._stable configurations, current brian2genn from master needs brian2 from master...).

@mstimberg mstimberg merged commit 0c8944b into master Nov 7, 2019
@mstimberg mstimberg deleted the run_regularly_CPU branch November 7, 2019 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support TimedArray and rethink approach to run_regularly Support run-regularly for synapses
2 participants