Improve basic n4 app

[gonzaponte]

This addresses the comments raised in #7. In this PR, we move the basic app to a subfolder and provide documentation on how to compile and run the app.

[jacg]

Apologies: I won't be able to look at this immediately.

[jacg]

Please read https://diataxis.fr/, especially https://diataxis.fr/tutorials-how-to/.

We don't necessarily have follow it religiously, but I agree with much of what it says.

Are we trying to write a tutorial or a how-to?

[jacg]

Hmm, don't want to encourage use of existing nain4 installations, but I guess that until we have understood what to do about #33 and made some progress on its future children, this is probably a reasonable choice at this point.

[gonzaponte]

Are we trying to write a tutorial or a how-to?

Definitely a tutorial. I had in mind to move it to the tutorials section and then forgot. Can we do it just before merging?

[jacg]

Definitely a tutorial.

In that case, there are not enough things for the learner to do and there is too much explanation.

Even for a how-to, there aren't enough active steps. Perhaps this is an explanation.

Perhaps we haven't thought enough about what it is that we are writing, before having started writing it.

The one thing we can be pretty sure this is not, is reference. So we should read and ruminate on

• https://diataxis.fr/tutorials/
• https://diataxis.fr/how-to-guides/
• https://diataxis.fr/explanation/
• https://diataxis.fr/tutorials-how-to/

Yes, it's frustrating to slow down and think: I also want to be creating material at a quicker rate, and to see the void being filled more rapidly, but I suspect that we'll do a lot better in the mid- and long-term, if we take it a little more slowly at the beginning, here.

I suspect that we are conflating a number of things, and that, instead, we should have:

1. A default nain4 application, which, out of the box, compiles, runs, and
   shows a pretty picture.

2. A how-to (or tutorial? or both? some people will be familiar with installing
   stuff, and need a how-to; others will require more guidance and might need a
   tutorial instead), which explains how to install and run the default app,
   along with a how-to about troubleshooting the installation.

3. Some tutorials, which take you through the motions of adapting different
   aspects (geometry, generator, actions, physics list, sensitive detector) of
   the default app.

4. Explanation of why the default app and nain4 are structured the way they
   are.

5. Some how-tos describing the details of specific and varied ways of doing the
   more general things explored in the tutorials in no 3 above. For example a
   bunch of different things you might want to do when creating your own
   geometry.

And plenty more, but with less direct links to this initial effort.

But I think it all has to start with the app, which should just be an app,
without documentation. Then we grow the documentation around it.

[gonzaponte]

I keep going back and forth about this, and I need to think about it a bit more. For now, I will just lay down a few points that I find relevant:

• If you want to start with the basic app as something that is given, the tutorial should take you from this app to a slightly more complex one. So far we have only written the basic app, so this discussion is probably more relevant for the upcoming stages than for the current one.

• I think the current documentation (an explanation) on how the basic app is built is still useful. From your comment, I deduce that you would postpone it to a later stage of the tutorial and start with "take the basic app, don't look at any of this, just build a geometry". I am still on the fence about this. I think a brief description of what you are looking at when you open the file is very helpful.

• The needs of our most immediate users will fall more on the work side than the study side (right?). Therefore we might want to prioritize how-to guides in the beginning.

Many of my doubts come also from whether we are thinking in the same terms or not. I also need to absorb diataxis more in-depth to have a clearer view. Perhaps we should discuss more interactively about this at some point.

We should also consider the pros and cons of offering documentation quickly to satisfy the current needs. We can always take a step back and redo it, better. It's more work, but it can also give us experience and perspective (which I find very important). On the other hand, it might bias our approach and push us to repeat the patterns that we have tried already.

[jacg]

Originally I had a couple of different things in mind

1. A default app, which takes almost no code to write, because it has default implementations of physics list, geometry, generator, actions, UI configuration, and maybe even sensitive detector, IO and other stuff, hidden behind its interface. All these should be replaceable by overriding the defaults. But, crucially, the word default implies that there are things there which are both not visible and overridable.

2. A template, which hides nothing, and therefore throws much more code in your face.

Then you opened your PR and I was won over by its brevity, because my template idea would throw too much in your face, and I had no clear vision of how to make the default app idea work well.

But now I'm less convinced.

I think that many of the tutorials and how-tos can be thought of as edges in a graph. Here's some lovely ASCII art about it:

[0] WTF is Geant4?
     |
     |
     V
[D] I have a running G4 app, but it doesn't do what I need
      |    |    |
      |    |    V
      |    |   [G] G4 app with geometry that's starting to look like mine
      |    V
      |   [P] G4 app with primary generators that are starting to look like mine
      V
     [W] G4 app with <whatever> that's starting to look like mine

and much more. Now,

If you want to start with the basic app as something that is given, the tutorial should take you from this app to a slightly more complex one.

I think we need multiple tutorials and how-tos describing the transitions: D -> G, D -> P, D -> W*.

So far we have only written the basic app, so this discussion is probably more relevant for the upcoming stages than for the current one.

Except that the upcoming stages to which you refer all depend on [D], so we need to have a [D] that is a good starting point for all of them. I think it's important that there be a single [D] because

• There should be a starting point for all these tutorials and how-tos that the learner can get running reliably. Maintaining, testing and documenting multiple [D]s would be a lot of work.
• A common starting point for all of them reduces the cognitive burden on the learner.

I think the current documentation (an explanation) on how the basic app is built is still useful.

Of course. That's 0 -> D. This is where having [D] be an explicit template rather than a default-hiding piece of nain4 magic, would be better. But for the all the D -> * edges, the default app would be better, because it hides away details which are irrelevant to whatever you are doing right now, but still leaves you with a whole program that works.

From your comment, I deduce that you would postpone it to a later stage of the tutorial

Your use of the word 'later' and the singular 'the tutorial' worry me. There will be many tutorials and how-tos, and they are not ordered in time. The reader should pick whichever one is most relevant to current needs and mood.

I imagine that a good order for many people would be to do a couple of D -> *s, before going to 0 -> D to put it into context, which, without having done some concrete D -> steps, would have been too abstract.

and start with "take the basic app, don't look at any of this, just build a geometry". I am still on the fence about this. I think a brief description of what you are looking at when you open the file is very helpful.

That depends entirely on who you are, and were you are in your Geant4 learning journey. The thing that has always hugely bothered me when writing the notes for my courses (which set out the order in which things will be presented to the people I have in front of me in class, but also act as a reference document for them, not only in class, but also later on¹), is that there is a pressure to cater for all possible different individuals' needs in all possible situations is a single, linear document. This is frustrating and counterproductive for exactly the reasons that Diátaxis points out. Diátaxis (and the fact that we're not aiming this at a bunch of people who will be sitting in a room and following the things we say, in the order in which we say them) is hugely liberating. Embrace that liberty!

So, in summary

a brief description of what you are looking at when you open the file is very helpful

That depends on the reader's current needs. Sometimes it will be helpful, sometimes it will be noise. Diátaxis proposes a strategy:

• Explanations of what you're looking at (or anything else) belong in explanations.
• If it's a tutorial, it should be a bunch of linear steps to follow, to give the learner an enjoyable, foolproof experience of going through the motions to get from A to B. But avoid explanations. It's all about learning by doing.
• If it's a how-to, it should assume background knowledge and competence, and point out alternatives. Still no explanations.

Crucially, the last two can refer to explanations, but shouldn't really explain anything themselves.

I'd say that 0 -> D should be explanation, while the D -> * should be tutorials and how-tos.

The needs of our most immediate users will fall more on the work side than the study side (right?).

I completely disagree. For pretty much the same reasons that I think the HEP and C++ situation is such a disaster. It's a huge fallacy to believe that you can write software fit for purpose, by getting complete C++ newbie students (aka cheap/slave labour) to start writing C++ for WORK, and that a decade down the line, after having copy-pasted their way to 'success', without ever having bothered to STUDY and understand C++, a tool which is both hugely complex and fundamental to their work, they can be considered even remotely-competent C++ programmers, let alone experts.

Yes, our readers will use n4/G4 for work, but if they never take the time to study them, they will only ever produce shit.

Not taking the time to understand what you are doing, is as bad as not taking the time to automatically test your code. Both seem very expedient in the short term. Both are a disaster in the long term.

Our immediate users, and their bosses, will be tempted to skip the study and the tests. We must try hard to help them avoid these grave errors.

Therefore we might want to prioritize how-to guides in the beginning.

Agreed. But only at the beginning.

Perhaps we should discuss more interactively about this at some point.

At some stage, probably, yes.

We should also consider the pros and cons of offering documentation quickly to satisfy the current needs.

Indeed. AFAICT, we have 2 active users. One of them got bootstrapping tuition con peras from me, and then instructed the other. The most short-term needs were covered by this. But if we are going to be users' needs-driven, then we should ask them what they need.

Footnotes

1. It always surprises me when people tell me that they keep coming back to my notes, because they're such a good reference: I always feel that my notes are shit, precisely because (and thanks to Diátaxis I have a way of articulating something that has been bothering me for years) by trying to do all 4 things at once, they can't possibly do any one of them well. ↩

[jacg]

Maybe the default app and template (as described at the top of my last comment) aren't such a great idea.

If we do #8 (and #30, though that one is minor), then we can get a picture or the screen with no more than

#include "nain4.hh"
#include "g4-mandatory.hh"

#include <G4SystemOfUnits.hh> // physical units such as `m` for metre
#include <G4Box.hh>           // for creating shapes in the geometry
#include <FTFP_BERT.hh>       // our choice of physics list

auto my_geometry() {
  auto hl = 1 * m; // HALF-length of world volume
  auto air = n4::material("G4_AIR");
  auto world = n4::volume<G4Box>("world", air, hl, hl, hl);
  return n4::place(world).now();
}

int main(int argc, char* argv[]) {
  auto run_manager = n4::run_manager();
  run_manager -> SetUserInitialization(new FTFP_BERT{0}); // version 0
  run_manager -> SetUserInitialization(n4::geometry(my_geometry));
  run_manager -> SetUserInitialization(n4::actions(n4::generator([](auto){})));
  run_manager -> Initialize();

  n4::ui(argc, argv).run();
}

So we can probably even afford to spend an extra line on

auto do_not_generate_anything = [] (auto) {};

And this could serve the purpose of default app and template in one go, and act as the starting point of all the D -> how-tos and tutorials.

[jacg]

Also, with #31 I'd like to reduce the run manager stuff to something more like

  auto run_manager = n4::run_manager()
    -> physics<FTPF_BERT>(0)
    -> geometry(my_geometry)
    -> actions((n4::generator([](auto){})))
    -> init();

We'd have to be careful to preserve the ability to inject a wider range of things than just these highly convenient/concise examples.

---

BTW, the FTPF_BERT argument means verbosity, not version: https://gitlab.cern.ch/geant4/geant4/-/blob/master/source/physics_lists/lists/src/FTFP_BERT.cc#L62-L69

Moral of the story: don't use abbreviations to name your variables.

[gonzaponte]

I think we need multiple tutorials and how-tos describing the transitions: D -> G, D -> P, D -> W*.

Agreed.

Except that the upcoming stages to which you refer all depend on [D]

Not only on [D]. I was thinking of a linear tutorial that guides you from the bare basics to a full-on monster app; i.e. 0 -> D -> P -> W. But

Your use of the word 'later' and the singular 'the tutorial' worry me. There will be many tutorials and how-tos, and they are not ordered in time. The reader should pick whichever one is most relevant to current needs and mood.

I see now that you were not thinking about a linear tutorial but rather a pick-your-choice scheme (perhaps with a suggested route through the different tutorials). I am definitely the kind of person that would take a linear tutorial, but that's my personal preference.
Moreover, I'm a terrible teacher and vastly inexperienced, so regardless of what diataxis says, I will trust your criterium.

I'd say that 0 -> D should be explanation, while the D -> * should be tutorials and how-tos.

Agreed.

Yes, our readers will use n4/G4 for work, but if they never take the time to study them, they will only ever produce shit.

I fully agree. But if I understood the situation correctly, we needed something digestible for last week. I was assuming we had to provide a first step to get the users jogging, even if they fall because they don't know how to walk. In order words, give up on creating a good documentation for the first users (who need something now) and focus that effort on the future users. But seeing this:

Our immediate users, and their bosses, will be tempted to skip the study and the tests. We must try hard to help them avoid these grave errors.

it looks like you don't want to go down that route.

I'd like to reduce the run manager stuff to [...]

This looks very good to me. We can even avoid pointers in the interface.

[jacg]

I suggest we proceed roughly like this

1. Provide an abstraction for initialization of the user interface. #8
2. Consider a run manager abstraction #31
3. minimal app with GUI: [D]
4. Tutorial D -> G that makes the geometry a bit more interesting. (How-to(s) about various tools & tricks for making geometries should come later.)
5. Tutorial G -> P that makes a primary generator. (How-to(s) about generators to be added later on.)
6. Tutorial P -> S that adds a sensitive detector
7. Tutorial S -> I that adds some IO
8. Tutorial I -> A that adds some actions. Possible actions:
   • Use stepping action and volume identity to as alternative to sensitive detector
   • Use stacking action to suppress secondaries
   • Anything you can think of?

I'm sure that we will have changed our mind by the time we get this far.

Getting the user to write tests, needs to be part of each of the above, from D onwards.

What do you think?

[gonzaponte]

What do you think?

Sounds more than reasonable. Is there anything specific that you want me to do or should I start tackling them one by one in that order?

Anything you can think of?

• Use an event action to decide on IO (e.g. require a minimum energy deposition in an SD to store the event)
• Use a tracking action to bias events toward one direction
• Use a run action to summarize the simulation

I've only done the first one, but I am trying to bring every action into the game.

[jacg]

Is there anything specific that you want me to do or should I start tackling them one by one in that order?

I think that one of us should do 1 and the other do 2. When we've reviewed and merged them, we'll be ready to polish off 3 (which is this PR, essentially). Then we can decide how to proceed further.

Or, if you prefer, maybe we can do 1&2 con peras. I should be available most of tomorrow.

[gonzaponte]

I have a few things to get done in the morning, I'll let you know.

[jacg]

Faffing around with the macros is a PITA. I now think that the first port of call, should be what you have right now, without the G4UI, but with the improved n4 run manager interface. So we should do (at least part of) #31 before #8.

[jacg]

I propose a slight change of strategy. I get the impression that the documentation is not that urgent right now. So how about concentrating on building up an example app, approximately as laid out in #34 (comment), without writing any documentation yet? But we should keep in mind that the evolution of the app will be used as the core of the documentation.

We shouldn't write too much undocumented stuff: writing docs, like tests, is never much fun and produces poor quality material, if they lag too far behind the implementation. OTOH, writing docs takes a lot of time, and having to rewrite them because the implementation changes, is also a huge burden. Developing a big example app should give us a more stable implementation to document.

[gonzaponte]

Sounds good to me. What about the interface for setting/building actions? Which one takes priority?

[jacg]

The actions interface requires a stroke of inspired insight, to get us anything significatly better that what we already have with n4::actions. So we can spend a lot of time thinking about this without making any progress. With the app we can make lots of progress without the need for inspiration: there's plenty of stuff to do that we just need to get on with and do.

If and when we come up with a better actions interface, it shouldn't be too much work to adapt the app.

So I'd say the app has priority. Probably starting with the UI abstraction, which I suspect will be the least fun.

[jacg]

The last two client-side tests are failing, because of missing OpenGL stuff. I'm struggling to understand what is going on. You are much more familiar with this whole setup, so I guess you'll see it immediately.

[gonzaponte]

Hmm. The other two tests pass because it's using v0.0.2. Looks like including the ui in nain4 forces the client to add the same boilerplate to the cmakelists that we added here.

[gonzaponte]

Perhaps, if there is not a huge downside we can include the ui always to reduce the noise?

[gonzaponte]

With this commit the tests pass

[gonzaponte]

This has become partially obsolete, but it contains an interesting discussion about how to organize the tutorials. We should summarize it and put in an issue so we can close this.