-
Notifications
You must be signed in to change notification settings - Fork 321
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dynamic urban #1546
Dynamic urban #1546
Conversation
Created a new branch as the basis of our new development
Created a new branch
Mod added to read dynamic PCT_URBAN from landuse.timeseries.
Update control flag management for dynamic urban landuse.
Call subroutines in dynurbanFileMod to initialize and change dynamic urban weights.
Add logical do_transient_urban and set default to .false.
Add sub setup_logic_do_transient_urban: Set default value and perform error checking on do_transient_urban. Function based on crops / lakes. Co-Authored-By: Keith Oleson <[email protected]>
Remove call UrbanInput with mode='finalize' which deallocates memory for urbinp datatype, as the arrays are needed for dynamic urban landunits.
Resolve CH4 conservation error when PCT_URBAN changes. Current approach is to use dzsoi_decomp instead of dz in ch4_totcolch4 over urban columns. See ESCOMP#1445 (comment). Co-Authored-By: Keith Oleson <[email protected]>
fix syntax error
fix error. test case completed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks really great. Sorry that it has taken me a couple of weeks to finish reviewing it. Thank you for your detailed PR comment, and especially for the initial testing you have done. This is really helpful!
I have a few minor inline comments. In addition, I have a few bigger picture comments. But overall, this looks really close to being ready for final testing & merge. Please see https://github.com/ESCOMP/CTSM/wiki/Pull-request-review-workflow for a description of the checkbox-based workflow I use for reviewing PRs to help us all stay on top of what still needs to be done.
-
For running urban everywhere: You say in Implementing dynamic urban land cover in CESM #1445 (comment) that you have been setting
run_zero_weight_urban
to.true.
. That seems reasonable for development, but I think what we want long-term is something similar to lake; this is less of a performance and memory hit than the currentrun_zero_weight_urban
mechanism (I'm not sure if you want to keeprun_zero_weight_urban
in addition to this mechanism). I don't feel strongly about whether this is done as part of issue Implementing dynamic urban land cover in CESM #1445 or deferred to its own issue, since @olyson showed in Implementing dynamic urban land cover in CESM #1445 (comment) that the performance hit isn't too great for the current mechanism. You can let me know if (a) you'd like to do it as part of this PR, or if (b) I should open a separate issue for this, or if (c) I should edit the initial comment in Implementing dynamic urban land cover in CESM #1445 with this change request. In any case, here's what I think should be done:- Add a hasurban variable, similar to haslake (do we want a separate one for each urban landunit, or just one overall for urban – so if there is ever any urban in a given grid cell in a transient run, we'll allocate memory for all urban landunits in that grid cell?)
- Add a subroutine urban_landunit_exists, similar to the lake_landunit_exists function added in ctsm5.1.dev003
- I think we can (and should) change the urban logic in is_active_l to be similar to lakes: if memory is allocated for an urban column, then make it active, period. I think what we want is to have some logic that determines whether a given urban landunit should exist in memory, and then as long as the landunit exists in memory, it should be active so that it is spunup. (I think that's what we want, but I'm open to discussion here.)
- Add code like surfrd_lakemask for urban in surfrdMod
-
We should probably add a test that covers this. For now it can use a single-point dataset that is created manually (see the comment in smallville_dynlakes_monthly for an example). I'm happy to talk more with you about how to set this up.
-
If the biogeophys conservation has been checked (which it sounds like it has been, according to Implementing dynamic urban land cover in CESM #1445 (comment) ), then remove the comment in dynConsBiogeophys, " ! No thought has been given to c2l_scale_type here. This may need to be fixed if we ever start caring about urban averages in this routine."
src/dyn_subgrid/dynurbanFileMod.F90
Outdated
do_check_sums_equal_1=.false., data_shape=wturb_shape) | ||
|
||
! Check that urban data is valid | ||
!KO call CheckUrban(bounds%begg, bounds%endg, wturban(bounds%begg:bounds%endg,:), subname) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- I'm thinking this commented-out line should simply be removed (this check is done in the interp method, and isn't needed here in the init method, right?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is removed. Thanks!
src/dyn_subgrid/dynurbanFileMod.F90
Outdated
! dyn_var_time_uninterp_type. | ||
! | ||
! !USES: | ||
use landunit_varcon , only : isturb_tbd, isturb_hd, isturb_md, isturb_MIN, isturb_MAX |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- isturb_MIN and isturb_MAX aren't used here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed!
src/biogeochem/ch4Mod.F90
Outdated
! Rather than trying to keep the BGC variables physically meaningful in urban landunits, | ||
! we will just pack these variables in a way that should conserve these variables, even if | ||
! the values in each of the urban columns is somewhat nonsensical. Specifically: we'll take | ||
! col%wtgcell at face value in urban columns in dynColumnStateUpdaterMod \u2013 i.e., for the sake |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- I think "\u2013" is meant to be a dash
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment will be further moved according to the comment below -- the dash is fixed for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll move this to the design document and fix the "\u2013".
src/biogeochem/ch4Mod.F90
Outdated
(finundated(c)*conc_ch4_sat(c,j) + (1._r8-finundated(c))*conc_ch4_unsat(c,j)) * & | ||
dz(c,j)*catomw | ||
l = col%landunit(c) | ||
! Rather than trying to keep the BGC variables physically meaningful in urban landunits, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- [OPTIONAL] I appreciate that you added some explanation here. However, a lot of this comment feels out of place here, since it is really referring to bigger-picture stuff than just the following line of code. I think the best thing to do would be to introduce a file in
doc/design
(maybe calleddynamic_urban.rst
) and put this comment along with some of the other context from the issue in there. Then, here in the code, we could simply say something like, "See doc/design/dynamic_urban.rst for an explanation of why we use dzsoi_decomp for urban landunits." @olyson what do you think about this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds like a good idea. I'll do this and push it to the branch.
@@ -354,7 +379,8 @@ logical function run_has_transient_landcover() | |||
|
|||
run_has_transient_landcover = & | |||
(get_do_transient_pfts() .or. & | |||
get_do_transient_crops()) | |||
get_do_transient_crops() .or. & | |||
get_do_transient_urban()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was initially concerned that you added urban to this conditional, but lake is not included. However, after some analysis of how this function is used, I think this is correct: this is just used by the fire code, which really wants to know whether there are any potential deforestation fires. Forests being replaced by urban could be linked with deforestation fires; forests being replaced by lakes / reservoirs seem very unlikely to be linked with deforestation fires.
Probably what we should do is to move this function into the fire code, and make it fire-specific, since otherwise it isn't obvious why this should include some transitions but not others. I'll open a separate issue to do that: for the sake of this PR, this can be left as is.
Update: see #1560
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, thanks, a separate issue is fine.
Thanks for the review. We've discussed and with regards to the bigger picture questions:
|
3d19d28
to
a7defab
Compare
I created a test for dynamic urban based on the dynamic lake test suggested: ERS_Lm25.1x1_smallvilleIA.IHistClm50BgcCropQianRs.cheyenne_gnu.clm-smallville_dynurban_monthly I also added a dynamic urban design document as suggested: doc/design/dynamic_urban.rst These features have been pushed to the branch. |
@olyson it looks like this file is one that you created by hand by modifying an existing one and changing the data you needed to, rather than running through mksurfdata_map. Is that correct? If so we might want to add a little more metadata to it to say as much and what the parent file was to compare to. If we do that I could copy it into inside $DIN_LOC_ROOT/lnd/clm2/surfdata_map. Does that sound like the thing to do here? |
Yes, it was created by hand. Following the dynamic lake example (cime_config/testdefs/testmods_dirs/clm/smallville_dynlakes_monthly/user_nl_clm), I put some notes in: cime_config/testdefs/testmods_dirs/clm/smallville_dynurban_monthly/user_nl_clm to describe how this was created. I.e., "! This file was created with the following NCL script: |
Following the dynamic lake example, the namelist should be: flanduse_timeseries = '$DIN_LOC_ROOT/lnd/clm2/surfdata_map/landuse.timeseries_1x1_smallvilleIA_hist_78pfts_simyr1850-1855_dynUrban_c211206.nc |
I went ahead and added that information to the global history attribute of that file as well. |
OK, I added it to DIN_LOC_ROOT on cheyenne, and checked it into svn as well. lnd/clm2/surfdata_map/landuse.timeseries_1x1_smallvilleIA_hist_78pfts_simyr1850-1855_dynUrban_c211206.nc |
Done - see #1572 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks great now! I think this is essentially ready to come to master. I just have one small final request below.
run_zero_weight_urban = .true. | ||
|
||
! This file was created with the following NCL script: | ||
! /glade/p/cgd/tss/people/oleson/modify_surfdata/modify_smallville_with_dynurban.ncl |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Thank you for your detailed comments on how the dyn urban test dataset was made. Can you please copy the ncl script into the testmods directory so that we can easily recreate it if needed (in case your directories get reorganized, etc.)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Added modify_smallville_with_dynurban.ncl to cime_config/testdefs/testmods_dirs/clm/smallville_dynurban_monthly
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome, thanks!
@olyson I think this is ready to go. Let's discuss its tag ordering at tomorrow's ctsm-software meeting, and who will do the final steps (final testing, writing ChangeLog). |
This set of changes synchronizes CTSM with API 20 of FATES. API 20 of FATES requires a modification to timing boundary conditions so that FATES can maintain its own internal running means. FATES also no-longer asks CTSM for the 24hour vegetation temperature, since this is a patch(pft) level variable, and FATES has more suitable averaging mechanisms. These changes are synchronized with FATES Pull Request: NGEET/fates#724, which subsequently created tag: https://github.com/NGEET/fates/releases/tag/sci.1.52.0_api.20.0.0
Redo options list to remove positional arguments that were difficult to input correctly. Transient runs now use run_type startup and get finidat from s3 server unless --run-from-postad option is used (or finidat is not available). Use mpi instead of mpi-serial, this mod was recommended for container use. Add a new script neon_finidat_upload which allows authorized users to upload finidat files to the s3 server. This includes the following changes to the script for updating the surface dataset at neon sites using available neon data (i.e. modify_singlept_site_neon.py) to address ctsm/issues ESCOMP#1353, ESCOMP#1429, and neon/issue ESCOMP#44: Update Organic calculation to use the following equation based on discussions in https://github.com/ESCOMP/CTSM/pull/1375/files#r669590971 : ORGANIC = estimatedOC * bulkDensity / 0.58 Because estimatedOC is rounded to the nearest integer in neon data, it is sometimes bigger than carbonTot. Therefore, in cases where estimatedOC > carbonTot, we use carbonTot instead of estimatedOC in the above equation. Previously, we had missing data on neon files for some neon soil layers (see Modified NEON surface datasets have errors ESCOMP#1429 (comment)). Therefore, it caused some missing values in the updated dataset. Here, we interpolate to fill in the missing data using different interpolation techniques. Finally, we chose to use linear interpolation to fill in missing data. This includes the scripts for modification of the surface dataset for neon sites to address ESCOMP#1429. Specifically, the following has been addressed in this PR: Update the calculation of ORGANIC to use the new field (CaCO3) from NEON data. For this calculation if CaCO3 data is available, we first calculate inorganic carbon by: inorganic carbon = (caco3 /100.0869)*12.0107 Next, we calculate organic carbon by subtracting inorganic carbon from the total carbon: [organic carbon = carbon_tot - inorganic carbon] If the CaCO3 is not available then the code uses carbonTot and estimatedOC by NEON. Discussed here (Modified NEON surface datasets have errors ESCOMP#1429 (comment)) For the Ag sites (KONA and STER), it changes the PCT_NATVEG, PCT_CROP, and PCT_NAT_PFT to avoid the error that we previously had in spin-up: surfrd_veg_all ERROR: sum of wt_nat_patch not 1.00000000000000 at nl= 1 sum is: 0.000000000000000E+000 Discussed here (Modified NEON surface datasets have errors ESCOMP#1429 (comment)) There was a typo previously in the NEON data for ABBY sites caused by mix of sample measurements. Please note that this was updated by hand once data was downloaded from NEON site. With recent versions of CIME, the LILAC build with a user-defined machine was broken for a couple of reasons. This fixes it. Fix mksurfdata_map for 1x1_brazil. Get tools testing working again. Increase skip_steps by 1, which is needed for a change in CAM where balance checks need to occur after the radiation update now rather than before. glob changed for bsd_glob in perl MkDepends for mksurfdata_map.
Adding fsurdat_modifier tool fsurdat_modifier is a tool that modifies fsurdat files. It reads a surface dataset (fsurdat file) and outputs a modified copy of the same file. Current applications are limited to the simplest CTSM(SP) mode, so bgc, fire, urban, vic, lake, transient, and crop-related variables in the fsurdat file remain unchanged. It differs from modify_singlept_site_neon.py in that the latter specifically modifies soil properties of single-point surface datasets. Some functions could/should become shared for use by fsurdat_modifier. It also differs from the subset_data tool in that the latter subsets fsurdat files to regional or single-point domains. Though subset_data includes some "modify" functionality when subsetting to single-point, we intend for similar functionality to be more extensive and more object oriented in fsurdat_modifier. Again, some functions could/should become shared among these tools.
It looks like cheyenne testing has passed. There are expected namelist failures for historical compsets due to the presence of do_transient_urban=.false. in the namelist. |
Thanks @olyson ! I ran the build-namelist tests and they all pass (with namelist differences as expected). I think you can check the izumi tests by logging on to hobart. |
Thanks @billsacks . |
Great! I have two comments on the ChangeLog:
|
Once you make those changes to the ChangeLog, I'll merge this. |
Yes, very important, I've added the contributors. Not quite sure about the format. I went with github name (if available), followed by full name. |
Description of changes
This update includes code changes that enable the model to to read in a dynamic landuse file (
landuse.timeseries
) that includes urban. It updates urban weights accordingly, and resolves potential errors.Specific notes
The transient urban function (namelist variable
do_transient_urban
) is off by default in this pull request, but we have successfully conducted a simulation with transient urban active and verified proper operation as well as energy and water conservation.Contributors other than yourself, if any: @olyson @Face2sea @keerzhang1
CTSM Issues Fixed (include github issue #): #1445 Implementing dynamic urban land cover in CESM
Are answers expected to change (and if so in what way)?
No if transient urban is off. Otherwise urban land unit weights are updated annually.
Any User Interface Changes (namelist or namelist defaults changes)?
Add namelist variable
do_transient_urban = .false.
by defaultTesting performed, if any:
We ran the clm_short test suite (using this command: ./run_sys_tests -c ctsm5.1.dev061 --skip-generate -s clm_short --baseline-root /glade/p/cgd/tss/ctsm_baselines/ --account P93300641).
We also ran a CLM4.5 test (ERS_D_Ld6.f10_f10_mg37.I1850Clm45BgcCrop.cheyenne_intel.clm-clm50CMIP6frc) and a CLM5.1 historical test (ERS_Ly5_P144x1.f10_f10_mg37.IHistClm51BgcCrop.cheyenne_intel.clm-cropMonthOutput).
All tests passed (BFB with baseline).
NOTE: Be sure to check your coding style against the standard
(https://github.com/ESCOMP/ctsm/wiki/CTSM-coding-guidelines) and review
the list of common problems to watch out for
(https://github.com/ESCOMP/CTSM/wiki/List-of-common-problems).