Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New fsufdat and landuse time series files for ne grids #1038

Merged
merged 43 commits into from
Jul 23, 2020

Conversation

fischer-ncar
Copy link
Contributor

@fischer-ncar fischer-ncar commented Jun 11, 2020

Updated and add new ne fsurdat and landuse_timeseries input files. Updates for grids
ne30np4, ne30np4.pg2, ne30pg3, ne120np4, ne120np4.pg2, ne0npp4CONUS.ne30x8, ne0np4.ARCTIC.ne30x4 and ne0np4.ARCTICGRIS.ne30x8

ne0np4.ARCTIC.ne30x4 and ne0np4.ARCTICGRIS.ne30x8 grids will fail, with unkown grid alias, until cime is updated to cime5.8.28.

Add pes layouts for CONUS and the ARCTIC grids.

Add new tests to aux_clm cheyenne_intel test list.
SMS_Ld1.ne30pg2_ne30pg2_mg17.I2000Clm50Sp.cheyenne_intel.clm-basic
SMS_Ld1.ne30pg3_ne30pg3_mg17.I2000Clm50BgcCrop.cheyenne_intel.clm-basic SMS_Ld1.ne0CONUSne30x8_ne0CONUSne30x8_mt12.I1850Clm50BgcCrop.cheyenne_intel.clm-basic SMS_Ld1.ne0ARCTICGRISne30x8_ne0ARCTICGRISne30x8_mt12.ISSP585Clm50BgcCrop.cheyenne_intel.clm-ciso_dec2050Start SMS_Ld1.ne0ARCTICGRISne30x8_ne0ARCTICGRISne30x8_mt12.I1850Clm50Sp.cheyenne_intel.clm-

Contributors other than yourself, if any: @adamrher

CTSM Issues Fixed (include github issue #): #992 #994 #888
Fixes #992
Fixes #994
Fixes #888

Are answers expected to change (and if so in what way)? Answer will change for the above mentioned grids.

Any User Interface Changes (namelist or namelist defaults changes)?

Testing performed, if any: Tested using cime5.8.25
SMS_Ld1.ne30pg2_ne30pg2_mg17.I2000Clm50Sp.cheyenne_intel.clm-basic
SMS_Ld1.ne30pg3_ne30pg3_mg17.I2000Clm50BgcCrop.cheyenne_intel.clm-basic SMS_Ld1.ne0CONUSne30x8_ne0CONUSne30x8_mt12.I1850Clm50BgcCrop.cheyenne_intel.clm-basic SMS_Ld1.ne0ARCTICGRISne30x8_ne0ARCTICGRISne30x8_mt12.ISSP585Clm50BgcCrop.cheyenne_intel.clm-ciso_dec2050Start SMS_Ld1.ne0ARCTICGRISne30x8_ne0ARCTICGRISne30x8_mt12.I1850Clm50Sp.cheyenne_intel.clm-

Several other tests by hand for the different grids and the different years.

@ekluzek ekluzek self-assigned this Jun 11, 2020
@ekluzek ekluzek added PR status: ready PR: this is ready to merge in, with all tests satisfactory and reviews complete priority: high High priority to fix/merge soon, e.g., because it is a problem in important configurations enhancement new capability or improved behavior of existing capability labels Jun 11, 2020
@ekluzek ekluzek added this to the cesm2.2.0 milestone Jun 11, 2020
@ekluzek
Copy link
Collaborator

ekluzek commented Jun 12, 2020

@fischer-ncar we need the mapping files in place for the new grids: ne120np4.p
g2,ne120np4.pg3,ne0np4CONUS.ne30x8,ne0np4.ARCTIC.ne30x4,ne0np4.ARCTICGRIS.ne30x8,ne240np4, so that we can easily create them each time new surface datasets are created.

@ekluzek
Copy link
Collaborator

ekluzek commented Jun 12, 2020

See the little script createMapEntry.pl in bld/namelist_files to help with that.

Copy link
Collaborator

@ekluzek ekluzek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to get the mapping files in place for each new grid. And it looks like there are some PE layouts missing for new grids. I doubt the default layouts really work for these new grids.

@adamrher
Copy link
Contributor

you need all the mapping files from mkmapdata? If so, I do have these here:

/glade/scratch/aherring/ctsm_release-clm5.0.30/tools/mkmapdata/

@ekluzek
Copy link
Collaborator

ekluzek commented Jun 12, 2020

Awesome @adamrher I'm glad you have those. We need those copied to subdriectories under $DIN_LOC_ROOT/lnd/clm2/mappingdata/maps/ and added to svn inputdata. And then added as xml entries in: namelist_defaults_ctsm.xml.

@fischer-ncar
Copy link
Contributor Author

I'll take care of the mappingdata files and update the PR.

I tested all of the grids with the config_pes.xml in this PR, they all passed. All of the ne30 grids (ne30, ne30pg2, ne30pg3) matched ne30, the same for the ne120 grids. So, I didn't feel I needed to explicitly list each ne30pg* and ne120pg* grid.

@ekluzek
Copy link
Collaborator

ekluzek commented Jun 12, 2020

OK, perfect, now looking at the config_pe file I see that it does a more generic match to ne30 and not to just ne30np4. That and since you tested to see they are good is great to me.

@fischer-ncar
Copy link
Contributor Author

I've added mapping files for ARCTIC, ARCTICGRIS, ne30np4.pg2, ne30pg3, ne120np4.pg2, ne120np4.pg3, and imported then to svn.

@ekluzek
Copy link
Collaborator

ekluzek commented Jun 18, 2020

@fischer-ncar can you make a cime tag that includes the renaming of the ne grids? I'll include that in the externals in this PR. I didn't see a tag that included it, but might have missed it.

@fischer-ncar
Copy link
Contributor Author

cime5.8.25 has the new grid aliases. The newest tag is cime5.8.26, but there might be issues with it.

@ekluzek
Copy link
Collaborator

ekluzek commented Jun 18, 2020

Ahh, perfect. I must have missed it in looking at the list. Thanks for letting me know the tag. We have a branch for cime, right now so I'll probably have to keep it on our branch.

ekluzek added 2 commits June 18, 2020 11:44
Change some hardcoded parameters to go on the parameter files. This is needed in preparation
of running the Perturbed Parameter Ensemble.
Update the standard list of resolutions to make when surface
datasets are created. Add a new list of SE and variable mesh
grids. Remove the f05 which we don't think we need.
@adamrher
Copy link
Contributor

Would it be easy to to use consistent names for the pg2 and pg3 grids? Or is it too late in the game?

now:
ne30np4.pg2, ne30pg3, ne120np4.pg2 and ne120np4.pg3

proposed:
ne30pg2, ne30pg3, ne120pg2 and ne120pg3

the np4 does not add any additional info as all cam-se grids use np4 (4x4 GLL points per element).

@ekluzek
Copy link
Collaborator

ekluzek commented Jun 18, 2020

@adamrher I was thinking the same thing. It will require coordinating several tags though. It's in cime this way as well as here. And presumably it's in CAM as well (and maybe cice)?

I think it's worth getting consistent as otherwise users aren't going to get it right. @fischer-ncar what do you think about changing this? I do need an updated cime tag anyway (because of the PIO1 issue that @billsacks just pushed). If you can make sure this is right in any other tags, I'll make sure it's right here.

@fischer-ncar
Copy link
Contributor Author

After doing some digging. It looks like I'll just need to do a cime tag. The CAM configure script will take the shorter grid name, and add the np4 automatically, so all of the configuration should be the same. Weather the grid name is ne30np4.pg2 or ne30pg2. So I'll go ahead and make a cime PR and tag.

@fischer-ncar
Copy link
Contributor Author

I'm assuming that the filenames and directories wont be changing? Is that true?
Otherwise, we'll need to rimport more data to svn.

@adamrher
Copy link
Contributor

adamrher commented Jun 18, 2020

I just want to point out that there are a ton (20ish?) of cam-se grids in config_grids.xml that are using the long form naming convention (neXXnp4.pgX). And I am pretty sure that ne30pg3 is the only cam-se grid that departs from the long form. So while I would prefer to slim down the naming convention like I proposed, it is clear that easiest way to make the naming conventions consistent in both ctsm and cime is by changing ne30pg3 to the long form ne30np4.pg3.

the cime domain/mapping filenames adhere to the long form, except the mapping files for ne30pg3 ... whereas the ne30pg3 domain files use the long-form (i.e., ne30np4.pg3).

I don't think there are directories named for individual grids

@ekluzek
Copy link
Collaborator

ekluzek commented Jun 18, 2020

@fischer-ncar I'd like to leave the filenames as they are. Most importantly so we don't needlessly increase our data usage. But, also the filenames aren't something that a user normally has to directly work with -- but the resolution names are. So to make it easy for users I think we should make sure they are consistent. We will eventually remake the filenames and I can make them consistent at that time.

@fischer-ncar
Copy link
Contributor Author

@adamrher the 20ish changes to cime wasn't that bad to do. I already have a sandbox with the shorter grid names in it, /glade/u/home/fischer/sandbox/cime.

@adamrher
Copy link
Contributor

great! Are you planning to do it for all neXXnp4.pgX? There are still plenty of the more obscure grids in your sandbox that still use this long form, e.g., ne120np4.pg4.

@mvertens
Copy link

mvertens commented Jul 13, 2020 via email

@ekluzek
Copy link
Collaborator

ekluzek commented Jul 13, 2020

@fischer-ncar and @adamrher I noticed we are missing the SCRIP grid files for the: ne0np4CONUS.ne30x8,ne0np4.ARCTIC.ne30x4,ne0np4.ARCTICGRIS.ne30x8 grids. Could you point me to those?

@ekluzek
Copy link
Collaborator

ekluzek commented Jul 13, 2020

I also noticed that the old name conus_30_x8 is used in some places where the new name ne0np4CONUS.ne30x8 should be used. So I'll change the name used to make it consistent.

@adamrher
Copy link
Contributor

The SCRIP files are in the cam repo here:

/glade/p/cesmdata/cseg/inputdata/atm/cam/coords/ne0ARCTICne30x4_scrip_c191212.nc
/glade/p/cesmdata/cseg/inputdata/atm/cam/coords/ne0ARCTICGRISne30x8_scrip_c191209.nc
/glade/p/cesmdata/cseg/inputdata/atm/cam/coords/ne0CONUSne30x8_scrip_c200107.nc

Thanks for making sure the conus names are consistent!

@ekluzek
Copy link
Collaborator

ekluzek commented Jul 13, 2020

@adamrher the new SCRIP grid file for CONUS is off by roundoff from the one that I was using and it's mapping files. Do we care about that? Since it's roundoff different it's within machine precision, but will technically be off from the one that CAM is using. As such my inclination is to ignore this, as it would take some work to put in place the technically correct files, both to generate them and get them into the xml. Let me know if you disagree. It's possible we'll notice this discrepancy in the future, so hopefully we'll remember this.

@adamrher
Copy link
Contributor

adamrher commented Jul 13, 2020

I'm not entirely clear here. I remade all the mapping files with the ctsm_release-clm5.0.30 using this same SCRIP file:

567 /glade/u/home/aherring> ncdump -h map_5x5min_nomask_to_ne0np4.CONUS.ne30x8_nomask_aave_da_c200426.nc | tail -n 11
:domain_a = "/glade/p/cesm/cseg/inputdata/lnd/clm2/mappingdata/grids/SCRIPgrid_5x5min_nomask_c110530.nc" ;
:domain_b = "/glade/p/cesmdata/cseg/inputdata/atm/cam/coords/ne0CONUSne30x8_scrip_c200107.nc" ;
:grid_file_src = "/glade/p/cesm/cseg/inputdata/lnd/clm2/mappingdata/grids/SCRIPgrid_5x5min_nomask_c110530.nc" ;
:grid_file_dst = "/glade/p/cesmdata/cseg/inputdata/atm/cam/coords/ne0CONUSne30x8_scrip_c200107.nc" ;

Is this the one you are comparing against? /glade/p/cesmdata/cseg/inputdata/lnd/clm2/mappingdata/grids/SCRIPgrid_conus_30_x8_nomask_c170111.nc

@ekluzek
Copy link
Collaborator

ekluzek commented Jul 13, 2020

Ahh, OK. That actually answers my question. I had updated the maps for the conus_30_x8 grid, but since you recreated all the mapping files I can point to the files you created rather than the conus_30_x8 ones. Then the grids will all be consistent, since you've already made the mapping files.

@adamrher
Copy link
Contributor

Glad that's cleared up. That old conus_30_x8 SCRIP file in the clm2/mapping/grids/ was made from the old way, an ncl script called HOMME2SCRIP.ncl. We now use fortran code to generate var-res SCRIP files.

ekluzek added 15 commits July 14, 2020 15:37
…rop version of fsurdat now), and remove the old conus_30x8 mapping files (will replace them with the new ones)
…s settings had to be set more carefully, and there is a dependence between two namelist_defaults files that I ran into
…n, add notes about answer changes to change files, update time for one test that ran over
@ekluzek ekluzek merged commit 65b6725 into ESCOMP:master Jul 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement new capability or improved behavior of existing capability priority: high High priority to fix/merge soon, e.g., because it is a problem in important configurations
Projects
No open projects
4 participants