New CAM-SE grids in CTSM

[adamrher]

Five (5) new CAM-SE grids should be in CTSM as part of the CESM2.2 release. The related CAM and CIME issues (which are both closed) are here: ESCOMP/CAM#62 and here: ESMCI/cime#3385.

The new grids are two (2) new CSLAM grids ne30pg3 & ne120pg3, and three (3) variable-resolution grids ARCTIC, ARCTICGRIS and CONUS. Based on discussions with @ekluzek, the minimum level of support for each grid should include:

yr 1850 fsurdat for crop (to be used for both crop and no-crop)
yr 2000 fsurdat for crop (to be used for both crop and no-crop)

Since the flanduse_timeseries files can be extremely large for global hi-res (i.e. ne120), only require these transient datasets for low resolution and variable-resolution grids. For ne30pg3, ARCTIC, ARCTICGRIS & CONUS, include transients:

yrs 1850-2000 flanduse_timeseries for crop (to be used for both crop and no-crop)
yrs 1850-2100 flanduse_timeseries for SSP5-RCP8.5 scenario, for crop (to be used for both crop and no-crop)

The surface datasets required by these five (5) grids need to have entries in namelist_defaults_ctsm.xml. Additionally, these new grids need to be added as valid values to the "res" namelist via namelist_defaults_ctsm.xml. Most of these source code mods and datasets are in @fischer-ncar 's branch (ctsm1.0.dev079.se_grids.n02). I will make sure to get the rest of the datasets to him in the next few days.

While the ne30np4 grid is already in CTSM, it does not have the correct 1850-2100 flanduse_timeseries, for crop. I will make sure to get these datasets to Chris, and am including this in this issue.

[fischer-ncar]

My latest branch is ctsm1.0.dev086.n01.

[ekluzek]

We should actually only bring in the datasets for crop at this point. The code has been changed so that you can use crop surface and landuse-timeseries files for both crop and no-crop. I've worked with John Truesdale to make sure it works for the FV3 grids and it does. See #891

[ekluzek]

I closed this by accident.

[ekluzek]

We also should have PE layouts for I compsets for these new grids. And for the ones that are considered supported we should have at least one test for them.

[adamrher]

I would like to also make sure that all the surface datasets for these new grids do not contain this bug: #697. Some of the datasets in @fischer-ncar 's repo are from me, which I generated using ctsm1.0.dev049. If the surface datasets in Chris's repo have not been updated to account for #697, then I can re-generate all the surface datasets for all CAM-SE grids.

Is this bug fix on CTSM master or is it only in the release version?

[ekluzek]

You should regenerate all the files with the release version where this is fixed. I'd suggest using release-clm5.0.30, since that's the latest cesm2.2.1.3 release version. We are also putting the files into a subdirectory with the version that created them. This is what we've done with the FV3 grids, and there already is a release-clm5.0.30 subdirectory you can use for this purpose.

I thought we had talked about this. But, yes this would've been one of the things I'd ask about in the PR review.

[adamrher]

ok. I generated and put all the required surface datasets for the new grids here:

/glade/scratch/aherring/ctsm_release-clm5.0.30/tools/mksurfdata_map/cam-se/

The -y 1850-2100 -ssp_rcp SSP5-8.5 option spits out another 1850 fsurdat that Im fairly certain is just a duplicate of the file generated with -y 1850 (and no -ssp flag) ... e.g.,

surfdata_ne30np4.pg3_SSP5-8.5_78pfts_CMIP6_simyr1850_c200426.nc
surfdata_ne30np4.pg3_hist_78pfts_CMIP6_simyr1850_c200426.nc

So I've removed the SSP duplicate.

I also (re)generated and put the ne30np4 and ne120np4 surface datasets there as well, i.e. the cam-se grids already in ctsm.

[fischer-ncar]

Thanks for regenerated all of those files.

[adamrher]

After speaking w/ Peter L., we would like to include the CSLAM grids w/ lower resolution physics in CTSM if possible, namely ne30pg2 and ne120pg2. This addition would add to the total number of new grids, which would then be four (4) CSLAM grids, and three (3) variable-resolution grids. Since these two pg2 grids are already in Chris's CTSM repo, CIME master and CAM master, it should be pretty straight forward to add these two additional grids to CTSM master.

The surface datasets for ne30pg2 and ne120pg2 are in the directory with all the other cam-se surface datasets, which I pointed to in my last post.

[ekluzek]

Adding lower resolution grids for official support without landuse-timeseries files is no problem. I also fully support adding the changes and mapping files for ANY grid. The concern I have is for higher resolution grids WITH landuse-timeseries. In the above directory you only have landuse.timeseries files for the lower resolution grids -- so that's fine. The total size is also only a half TByte, so I think that's OK as well. And it looks like we can fit it on cheyenne inputdata.

So bottom line this all looks good to me.

[adamrher]

Great. Yes, It's very much baked into my mind to not generate flanduse_timeseries for high resolution grids. And so the directory w/ all the (re)generated surface datasets is complete.

[fischer-ncar]

@ekluzek, which I compset should I use when adding tests?
@adamrher, do you have any processor layouts for the new grids?

[adamrher]

I don't have any processor layouts for these grids. I actually have never worked with pe layouts before. If I need to do this, I can reach out to the appropriate people to point me in the right direction.

[fischer-ncar]

I can come up with some pe layouts to get the I compset tests to work. Since these grids will be
supported in the release, it might be a good idea to get pe layouts into a cam tag.

[ekluzek]

Yeah, we need both PE layouts for I compsets (that go into the CTSM tag) and ones for F compsets (that go into the cam tag). And I suppose we also need fully coupled ones that go into the CESM tag right?

[ekluzek]

Chris on compsets to test. Which of the new grids are going to be considered "fully supported"? And are 1850, 2000 and Historical going to be fully supported?

To test everything we'd need to do 1850, 2000, and historical for both Sp and BgcCrop. But, I don't think we should do that many tests.

[ekluzek]

OK, we got confirmation from Adam, but I also reread the whole top of this issue and it lays it out there. So sorry for needlessly adding people to repeat.

The following are should be fully supported: ne30pg2, ne30pg3, ne120pg2, ne120pg3, CONUS, ARCTIC and ARCTICGRIS for 1850 and 2000. ne30pg2, ne30pg3, CONUS, ARCTIC and ARCTICGRIS should also support transient.

So we will by default create all of these grids with surface dataset updates. To test all of the grids we'd then need 14 constant year, and 5 transient and then do both for Sp and Bgc-Crop, so 28 total. Rather than do that, we should make sure they are all tested in the namelist testing, and then do a selection of them. So I'm thinking a spread like this...

ne30pg2 at I2000Clm50Sp, ne30pg3 at I2000Clm50BgcCrop, CONUS at I1850Clm50BgcCrop, ARCTICGRIS at I1850Clm50Sp and ARCTICGRIS at ISSP585Clm50BgcCrop.

[billsacks]

@ekluzek thank you for giving some thought to how to get decent coverage of these grids without adding too many tests. I'd also add:

• These can be very short SMS non-debug tests, since the main purpose is to ensure that the model gets off the ground with these grids

• It would be good to use the comment field in the testlist xml file to note something like, "Want at least one test of ne30pg2 [or whatever] because this grid is needed by CAM", so that when we're trimming the test list later we can remember why we have added these tests.

[fischer-ncar]

Quick question, since the surface files are used for crop true and false, should the use_crop=".true." in namelist_defaults_ctms.xml file be removed?

For example

If I try testing with I2000CLM50Sp it fails to find the fsurdat file because of the crop.

[ekluzek]

Yes, exactly. Remove the use_crop=".true." attribute. Use the FV3 datasets like C96 as examples.