Don't allow instantaneous fields and averaged fields to be on the same history file, time is different for each

[ekluzek]

This is linked to the discussion on the issue in CAM where we should implement the same solution.

ESCOMP/CAM#159

For averaged (or min, max, or sum) fields time should be the middle of the averaging interval. And time_bounds should have the start and end of the time interval.
For instantaneous fields time should be the time of the output. time_bounds should be the start and end of the time-step.

Because the time axis will be different for each, this means you can't have ":I" fields on the same history output stream as other fields.

[billsacks]

@ekluzek thanks for opening this issue. Ideally we'll coordinate with CAM developers (@gold2718, @brian-eaton and others) on the user interface so we have consistency between CAM and CTSM. My initial thinking was that we'd get rid of the ability to have a ":I" designation and instead have a new hist_* variable (like hist_instantaneous) with one value per stream (similar to hist_dov2xy, etc.). But I'm not tied to that idea.

[ekluzek]

@ekluzek thanks for opening this issue. Ideally we'll coordinate with CAM developers (@gold2718, @brian-eaton and others) on the user interface so we have consistency between CAM and CTSM. My initial thinking was that we'd get rid of the ability to have a ":I" designation and instead have a new hist_* variable (like hist_instantaneous) with one value per stream (similar to hist_dov2xy, etc.). But I'm not tied to that idea.

There, actually already is a variable that sets the averaging type per stream "hist_avgflag_pertape". And the only thing that you can't have is ":I" fields with the other averaging types (:A, :X, :M). So it's actually ok to have A, X, and M mixed up on one stream. So it seems like the current user interface is fine, we just need additional checking to make sure I type averaging isn't mixed up with any of the other types.

[billsacks]

Thanks for pointing out hist_avgflag_pertape – I wasn't aware of that. That gets us most of the way there. I'd still argue for removing the :I designator, because it feels like it adds conceptual and code complexity to allow that designator but then make sure it's only used in very limited circumstances – circumstances that could also be achieved via hist_avgflag_pertape.

[ekluzek]

So allow :A, :M, and :X, but disallow :I (it that tape isn't instantaneous). That does make sense. That would still keep the current interface and just add additional checking to make sure it was correct.

[billsacks]

Copying some notes from https://docs.google.com/document/d/1r1WUXONjEKjKqECDkML72gtQnpj-EjGQE1_a6EryhY8 . These notes were from 2020-06-19, so a few days before this issue was opened:

Gokhan & Mariana feel this should actually get attention at fairly high priority.

Since it doesn’t seem feasible to have multiple time dimensions, at least in the short-term (because it sounds like this would require NetCDF4): One proposed implementation would be to require components to separate time-averaged and instantaneous fields into separate files.

We’d like input from CAM SEs on how hard it would be to do this (or some other solution to this problem).

CTSM would also have to do some development work for this. I looked quickly (git grep -n 'avgflag=' | grep -v "avgflag='A'") and it looks like the only default-active instantaneous field is SNOW_PERSISTENCE. Assuming we can think of a way around that one, and that we can also get rid of / change the handful of history variables whose default is instantaneous: I think the main things this would require would be (1) changing some infrastructure and documentation so that users specify instantaneous on a per-file basis rather than a per-field basis; and (2) allowing for a larger number of history files, because in practice we seem to often want instantaneous as well as average fields at a few different time frequencies and/or spatial averages.

I [Bill Sacks] wonder about another option, which would be: instantaneous fields are saved in the time step that is closest to the midpoint of the averaging period, rather than at the end of the averaging period. i.e., those instantaneous fields would still apply at the time given in the history file – but that time would now be in the middle of the averaging period (e.g., month) rather than at the end of the period. I think this would be easier to implement in CTSM, but I’m not sure if it would be acceptable for analyses.

[billsacks]

One thing we'll need to consider is how to deal with the cmip6 user mods. @olyson can help with this.

[wwieder]

At today's SE meeting we mentioned that there may need to be some coordination with FATES too, although @rgknox suggested this may not be too difficult and can be added to code here

[billsacks]

As per ESCOMP/CESM#194 (comment) - time_bounds can / should be left off of files that just have instantaneous values.

[billsacks]

See also discussion here ESCOMP/CESM#194 (comment) about possible naming of the different streams.

[slevis-lmwg]

From a first look at the code:
Subr. htapes_fieldlist determines total number of history tapes.
First task is to separate instantaneous variables into hi files, so I will modify this subroutine to add hi files for instantaneous variables.

[slevis-lmwg]

Wrapping my head around what the code does now and how I can best use the existing infrastructure.

This is my current plan of first steps in this PR, which may differ from what we agreed above, though I'm pretty sure it's close. Comments always welcome:

• Only SNOW_PERSISTENCE is set to avgflag = 'I'. Seems most sensible to first change this field to avgflag = 'A' rather than keeping it uniquely different from all other fields.
• Users can then use the existing fincl and hist_avgflag_pertape infrastructure to manually add history tape(s) with instantaneous-only fields.
• I will eliminate the instantaneous option from fincl and from hist_addfld1d and 2d, to prevent inadvertent mixing of instantaneous fields with other fields in a single tape.

Update: Plan hasn't changed. I just wanted to correct the above statement about SNOW_PERSISTENCE. Turns out there are also five inactive fields set to 'I'.

[ekluzek]

CAM just completed this work. So now we are free to copy Courtney's work in

ESCOMP/CAM#903

CAM divides it into

hXi and hXa files.