Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optional metadata field for dynamic constituent routine #13

Merged
merged 9 commits into from
Mar 1, 2024

Conversation

peverwhee
Copy link
Owner

Overview

This PR brings in the optional metadata header field dynamic_constituent_routine, which specifies the name of a routine (contained within the module in question) that returns a run-time configurable array of constituent properties.

Code Changes

  • Modifications to constituents.py to loop over dynamic constituent routines and call them, allocate and pack an overall array of dynamic constituents, add the dynamic constituents to the host constituent object, and deallocate individual dynamic constituent arrays.
  • Add dynamic constituent dictionary (dyn_const_dict) to API object. key = scheme; value = dynamic constituent routine
  • Modify datatable xml file to include the dynamic constituent routines
  • Modify metadata parser to handle new field and add dynamic constituent routines to scheme metadata table
  • Change constituents.py to use fstrings (almost) exclusively

Testing

Test added: Added dynamic constituents routines to both advection test schemes; added checks to confirm properties are accessible by host model; confirmed that the routines are written to the datatable xml file; confirmed that the values at the end of the run are still the default values (no operations done to the dynamic constituents in the test...)

User interface changes

User can now (optionally) specify a dynamic constituent routine in the header for a module's metadata file like:

[ccpp-table-properties]
name = cld_liq
type = scheme
dynamic_constituent_routine = cld_liq_dynamic_constituents

User must also provide the (public) dynamic_constituent_routine within the .F90 file associated with the metadata file (in the above example: cld_liq.F90). The routine must include the following calling list (variable names may vary, of course):

subroutine cld_liq_dynamic_constituents(dyn_const, errcode, errmsg)
  use ccpp_constituent_prop_mod, only: ccpp_constituent_properties_t
  type(ccpp_constituent_properties_t), allocatable, intent(out) :: dyn_const(:)
  integer,                             intent(out) :: errcode
  character(len=512),                  intent(out) :: errmsg

And the routine must allocate dyn_const to the number of dynamic constituents for that scheme and then instantiate them as such:

errmsg = ''
errcode = 0
allocate(dyn_const(1), stat=errcode)
if (errcode /= 0) then
  errmsg = 'Error allocating dyn_const in cld_liq_dynamic_constituents'
end if
call dyn_const(1)%instantiate(std_name="dyn_const3", long_name='dyn const3', &
     units='kg kg-1', default_value=0._kind_phys,                            &
     vertical_dim='vertical_layer_dimension', advected=.true.,               &
     errcode=errcode, errmsg=errmsg)

Generated Code Snippet

The folllowing code is generated when a user specifies dynamic constituent routines for two schemes (bolded code is new with this PR)

 subroutine test_host_ccpp_register_constituents(suite_list, host_constituents,                 &
         dynamic_constituents , errflg, errmsg)
       ! Create constituent object for suites in < suite_list >

       use ccpp_constituent_prop_mod, only: ccpp_constituent_properties_t
       ! Suite constituent interfaces
       use ccpp_cld_suite_cap,        only: cld_suite_constituents_num_consts
       use ccpp_cld_suite_cap,        only: cld_suite_constituents_const_name
       use ccpp_cld_suite_cap,        only: cld_suite_constituents_copy_const
       ! Dynamic constituent routines
       use cld_liq, only: cld_liq_dynamic_constituents
       use cld_ice, only: cld_ice_dynamic_constituents
 
       ! Dummy arguments
       character(len=*), intent(in)  :: suite_list(:)
       type(ccpp_constituent_properties_t), target, intent(in)  :: host_constituents(:)
       type(ccpp_constituent_properties_t), allocatable, target, intent(inout) ::                  &
            dynamic_constituents(:)
       character(len=512), intent(out)   :: errmsg ! ccpp_error_message
       integer,            intent(out)   :: errflg ! ccpp_error_code
       ! Local variables
       integer                                      :: num_suite_consts
       integer                                      :: num_consts
       integer                                      :: num_dyn_consts
       integer                                      :: index, index_start
       integer                                      :: field_ind
       type(ccpp_constituent_properties_t), pointer :: const_prop
       ! dynamic constituent props variable for cld_ice
       type(ccpp_constituent_properties_t), allocatable, target :: dyn_const_prop_0(:)
       ! dynamic constituent props variable for cld_liq
       type(ccpp_constituent_properties_t), allocatable, target :: dyn_const_prop_1(:)
 
       errflg = 0
       num_consts = size(host_constituents, 1)
       num_dyn_consts = 0
       ! Number of suite constants for cld_suite
       num_suite_consts = cld_suite_constituents_num_consts(errflg=errflg, errmsg=errmsg)
       num_consts = num_consts + num_suite_consts
       ! Add in dynamic constituents
       if (errflg == 0) then
          call cld_ice_dynamic_constituents(dyn_const_prop_0, errcode=errflg, errmsg=errmsg)
          num_dyn_consts = num_dyn_consts + size(dyn_const_prop_0)
       end if
       if (errflg == 0) then
          call cld_liq_dynamic_constituents(dyn_const_prop_1, errcode=errflg, errmsg=errmsg)
          num_dyn_consts = num_dyn_consts + size(dyn_const_prop_1)
       end if
       num_consts = num_consts + num_dyn_consts
       ! Pack dynamic_constituents array
       allocate(dynamic_constituents(num_dyn_consts), stat=errflg)
       if (errflg /= 0) then
          errmsg = 'failed to allocate dynamic_constituents'
       else
          index_start = 1
          do index = 1, size(dyn_const_prop_0, 1)
             dynamic_constituents(index + index_start - 1) = dyn_const_prop_0(index)
          end do
          index_start = size(dyn_const_prop_0, 1) + 1
          deallocate(dyn_const_prop_0)
          do index = 1, size(dyn_const_prop_1, 1)
             dynamic_constituents(index + index_start - 1) = dyn_const_prop_1(index)
          end do
          index_start = size(dyn_const_prop_1, 1) + 1
          deallocate(dyn_const_prop_1)
       end if
       if (errflg == 0) then
          ! Initialize constituent data and field object
          call test_host_constituents_obj%initialize_table(num_consts)
          ! Add host model constituent metadata
          do index = 1, size(host_constituents, 1)
            if (errflg == 0) then
               const_prop => host_constituents(index)
               call test_host_constituents_obj%new_field(const_prop, errcode=errflg,              &
                    errmsg=errmsg)
            end if
            nullify(const_prop)
            if (errflg /= 0) then
               exit
            end if
         end do
      end if

      ! Add dynamic constituent properties
      if (errflg == 0) then
         do index = 1, size(dynamic_constituents, 1)
            if (errflg == 0) then
               const_prop => dynamic_constituents(index)
               call test_host_constituents_obj%new_field(const_prop, errcode=errflg,              &
                    errmsg=errmsg)
            end if
            nullify(const_prop)
            if (errflg /= 0) then
               exit
            end if
         end do
      end if
      if (errflg == 0) then
         ! Add cld_suite constituent metadata
         num_suite_consts = cld_suite_constituents_num_consts(errflg=errflg, errmsg=errmsg)
      end if
      if (errflg == 0) then
         do index = 1, num_suite_consts
            if (errflg == 0) then
               allocate(const_prop, stat=errflg)
            end if
            if (errflg /= 0) then
               errmsg = "ERROR allocating const_prop"
               exit
            end if
            if (errflg == 0) then
               call cld_suite_constituents_copy_const(index, const_prop, errflg=errflg,           &
                    errmsg=errmsg)
            end if
            if (errflg == 0) then
               call test_host_constituents_obj%new_field(const_prop, errcode=errflg,              &
                    errmsg=errmsg)
            end if
            nullify(const_prop)
            if (errflg /= 0) then
               exit
            end if
         end do
      end if
      if (errflg == 0) then
         call test_host_constituents_obj%lock_table(errcode=errflg, errmsg=errmsg)
      end if
      if (errflg == 0) then
         ! Set the index for each active constituent
         do index = 1, SIZE(test_host_model_const_indices)
            call test_host_constituents_obj%const_index(field_ind,                                &
                 test_host_model_const_stdnames(index), errcode=errflg, errmsg=errmsg)
            if (field_ind > 0) then
               test_host_model_const_indices(index) = field_ind
            else
               errflg = 1
               errmsg = 'No field index for '//trim(test_host_model_const_stdnames(index))
            end if
            if (errflg /= 0) then
               exit
            end if
         end do
      end if
   end subroutine test_host_ccpp_register_constituents

@peverwhee peverwhee requested a review from nusbaume February 13, 2024 00:59
@peverwhee peverwhee self-assigned this Feb 13, 2024
@peverwhee
Copy link
Owner Author

@nusbaume currently, the only way to determine if a constituent is moist/dry and what type of mixing ratio it is is by parsing the standard name. Is this something we'll need to add setters for for MICM?

@nusbaume
Copy link
Collaborator

@peverwhee Thanks for doing this! In terms of your moist/dry question I think that is worth a discussion, so I've added it to our Wednesday meeting (and after we come up with our preferred method we'll probably need to at least run it by NOAA in the Monday meeting as well).

Copy link
Collaborator

@nusbaume nusbaume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! However, I do have some questions and some possible change requests.

scripts/ccpp_datafile.py Show resolved Hide resolved
scripts/ccpp_datafile.py Outdated Show resolved Hide resolved
scripts/ccpp_datafile.py Show resolved Hide resolved
scripts/constituents.py Outdated Show resolved Hide resolved
scripts/constituents.py Outdated Show resolved Hide resolved
scripts/constituents.py Outdated Show resolved Hide resolved
scripts/constituents.py Outdated Show resolved Hide resolved
test/advection_test/cld_ice.F90 Show resolved Hide resolved
test/advection_test/cld_liq.F90 Show resolved Hide resolved
test/advection_test/test_host.F90 Outdated Show resolved Hide resolved
@peverwhee peverwhee requested a review from nusbaume February 28, 2024 20:34
@peverwhee
Copy link
Owner Author

Updated generated code with new error handling logic in place:

   subroutine test_host_ccpp_register_constituents(suite_list, host_constituents,                 &
        dynamic_constituents , errflg, errmsg)
      ! Create constituent object for suites in 

      use ccpp_constituent_prop_mod, only: ccpp_constituent_properties_t
      ! Suite constituent interfaces
      use ccpp_cld_suite_cap,        only: cld_suite_constituents_num_consts
      use ccpp_cld_suite_cap,        only: cld_suite_constituents_const_name
      use ccpp_cld_suite_cap,        only: cld_suite_constituents_copy_const
      ! Dynamic constituent routines
      use cld_liq, only: cld_liq_dynamic_constituents
      use cld_ice, only: cld_ice_dynamic_constituents

      ! Dummy arguments
      character(len=*), intent(in)  :: suite_list(:)
      type(ccpp_constituent_properties_t), target, intent(in)  :: host_constituents(:)
      type(ccpp_constituent_properties_t), allocatable, target, intent(inout) ::                  &
           dynamic_constituents(:)
      character(len=512), intent(out)   :: errmsg ! ccpp_error_message
      integer,            intent(out)   :: errflg ! ccpp_error_code
      ! Local variables
      integer                                      :: num_suite_consts
      integer                                      :: num_consts
      integer                                      :: num_dyn_consts
      integer                                      :: index, index_start
      integer                                      :: field_ind
      type(ccpp_constituent_properties_t), pointer :: const_prop
      ! dynamic constituent props variable for cld_ice
      type(ccpp_constituent_properties_t), allocatable, target :: dyn_const_prop_0(:)
      ! dynamic constituent props variable for cld_liq
      type(ccpp_constituent_properties_t), allocatable, target :: dyn_const_prop_1(:)

      errflg = 0
      num_consts = size(host_constituents, 1)
      num_dyn_consts = 0
      ! Number of suite constants for cld_suite
      num_suite_consts = cld_suite_constituents_num_consts(errflg=errflg, errmsg=errmsg)
      if (errflg /= 0) then
         return
      end if
      num_consts = num_consts + num_suite_consts
      ! Add in dynamic constituents
      call cld_ice_dynamic_constituents(dyn_const_prop_0, errcode=errflg, errmsg=errmsg)
      if (errflg /= 0) then
         return
      end if
      num_dyn_consts = num_dyn_consts + size(dyn_const_prop_0)
      call cld_liq_dynamic_constituents(dyn_const_prop_1, errcode=errflg, errmsg=errmsg)
      if (errflg /= 0) then
         return
      end if
      num_dyn_consts = num_dyn_consts + size(dyn_const_prop_1)
      num_consts = num_consts + num_dyn_consts
      ! Pack dynamic_constituents array
      allocate(dynamic_constituents(num_dyn_consts), stat=errflg)
      if (errflg /= 0) then
         errmsg = 'failed to allocate dynamic_constituents'
         return
      end if
      index_start = 0
      do index = 1, size(dyn_const_prop_0, 1)
         dynamic_constituents(index + index_start) = dyn_const_prop_0(index)
      end do
      index_start = size(dyn_const_prop_0, 1)
      deallocate(dyn_const_prop_0)
      do index = 1, size(dyn_const_prop_1, 1)
         dynamic_constituents(index + index_start) = dyn_const_prop_1(index)
      end do
      index_start = size(dyn_const_prop_1, 1)
      deallocate(dyn_const_prop_1)
      ! Initialize constituent data and field object
      call test_host_constituents_obj%initialize_table(num_consts)
      ! Add host model constituent metadata
      do index = 1, size(host_constituents, 1)
         const_prop => host_constituents(index)
         call test_host_constituents_obj%new_field(const_prop, errcode=errflg, errmsg=errmsg)
         nullify(const_prop)
         if (errflg /= 0) then
            return
         end if
      end do

      ! Add dynamic constituent properties
      do index = 1, size(dynamic_constituents, 1)
         const_prop => dynamic_constituents(index)
         call test_host_constituents_obj%new_field(const_prop, errcode=errflg, errmsg=errmsg)
         nullify(const_prop)
         if (errflg /= 0) then
            return
         end if
      end do
      ! Add cld_suite constituent metadata
      num_suite_consts = cld_suite_constituents_num_consts(errflg=errflg, errmsg=errmsg)
      if (errflg /= 0) then
         return
      end if
      do index = 1, num_suite_consts
         allocate(const_prop, stat=errflg)
         if (errflg /= 0) then
            errmsg = "ERROR allocating const_prop"
            return
         end if
         call cld_suite_constituents_copy_const(index, const_prop, errflg=errflg, errmsg=errmsg)
         if (errflg /= 0) then
            return
         end if
         call test_host_constituents_obj%new_field(const_prop, errcode=errflg, errmsg=errmsg)
         nullify(const_prop)
         if (errflg /= 0) then
            return
         end if
      end do

      call test_host_constituents_obj%lock_table(errcode=errflg, errmsg=errmsg)
      if (errflg /= 0) then
         return
      end if
! Set the index for each active constituent
      do index = 1, SIZE(test_host_model_const_indices)
         call test_host_constituents_obj%const_index(field_ind,                                   &
              test_host_model_const_stdnames(index), errcode=errflg, errmsg=errmsg)
         if (errflg /= 0) then
            return
         end if
         if (field_ind > 0) then
            test_host_model_const_indices(index) = field_ind
         else
            errflg = 1
            errmsg = 'No field index for '//trim(test_host_model_const_stdnames(index))
            return
         end if
      end do

Copy link
Collaborator

@nusbaume nusbaume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good to me now, thanks @peverwhee! I just had one last (completely optional) suggestion.

'banana'

# Test no table found (expect None)
>>> _find_var_dictionary(table, dict_name='apple')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this test still work if you explicitly write None as the answer? If so then I might do so just to make it clear that is what will be returned by the function.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ooh i learned another thing today - doctests ignore Nones. Added an is None check (& expected result = True)

Copy link
Collaborator

@mwaxmonsky mwaxmonsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @peverwhee! Just had one or two things that might be of interest but by no means any reason to hold this up. Thanks!

routine_name = scheme_tdict[table].dyn_const_routine
if routine_name is not None:
if routine_name not in dyn_const_dict.values():
dyn_const_dict[table] = scheme_tdict[table].dyn_const_routine
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dyn_const_dict[table] = scheme_tdict[table].dyn_const_routine
dyn_const_dict[table] = routine_name

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch! fixed.

Comment on lines 434 to 440
for routine in routines:
routine_name = routine.text
if routine_name is not None:
result.add(routine_name)
# end if
# end for
return sorted(result)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for routine in routines:
routine_name = routine.text
if routine_name is not None:
result.add(routine_name)
# end if
# end for
return sorted(result)
routine_names = [routine.text for routine in routines if routine.text]
return sorted(routine_names)

Typically, list comprehensions are more optimized for generating lists but if the current code is more readable for everyone, please disregard!

Also, if we want to add the routine_name even if it is an empty string, this is perfectly fine. But if we want to exclude empty strings, we might want to change this to the check in the list comprehension if routine.text as empty strings are considered False.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed!

@peverwhee peverwhee merged commit 8d2bd61 into add_const_interface Mar 1, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants