ERROR: Could not find all inputdata on any server when running FATES

[FiND-Tao]

I received the error below, when I tried to run FATES following the tutorial CTSM-Tutorial/notebooks/SinglePoint/ProjectExamples/FATES_NEON_fromScratch.ipynb at main · NCAR/CTSM-Tutorial. ./case.submit command tried to download files from https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/atm/datm7/UNSET/. However, I checked https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/atm/datm7/, there is no folder named UNSET.

**Model datm missing file file48 = '/home/user/inputdata/atm/datm7/UNSET/CLM1PT_data/2021-12.nc'
Trying to download file: 'atm/datm7/UNSET/CLM1PT_data/2021-12.nc' to path '/home/user/inputdata/atm/datm7/UNSET/CLM1PT_data/2021-12.nc' using SVN protocol.
svn export failed with output: and errput svn: E170000: URL 'https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/atm/datm7/UNSET/CLM1PT_data/2021-12.nc' doesn't exist

Checking server None with protocol None
Setting resource.RLIMIT_STACK to -1 from (-1, -1)
Client protocol None not enabled
ERROR: Could not find all inputdata on any server**

[wwieder]

Thanks for your inquiry here. This tutorial was set up to run on a cloud instance that was just active for the tutorial. These files are not available on svn. What system are you trying to run on?

[FiND-Tao]

Hi @wwieder , thanks for your reply. I am running it on my local server using NCAR docker image escomp/cesm-lab-neon. I also have access to NCAR HPC derecho, can I run this FATES-neon tutorial on derecho machine?

[briandobbins]

I don't think it should actually be looking for data called 'UNSET' -- what site are you running, and can you share any changes you've made? I'll try to duplicate this today.

[FiND-Tao]

Hi @briandobbins , thank you for promptly offering assistance. The site, "STEI," was used in my code and it mirrors exactly what's demonstrated in the tutorial Jupyter notebook.

[wwieder]

OK, if you have access to derecho it's easier for us to troubleshoot there.
Out of the box, I'm not able to replicate this issue.

Can you also let us know the version of CTSM you're using? If you set up your environment as we did in the tutorial you can do this by:

cd ~/CTSM
git describe

In my own sandbox I'm using ctsm5.2.001

With this version of code I needed to update the compset as follows:

export neon_site="STEI"
./create_newcase --case ~/scratch/NEON_cases/${neon_site}_FATESsp_test \
    --res CLM_USRDAT \
    --compset I1PtClm60Fates \
    --output-root ~/scratch/NEON_cases/ \
    --run-unsupported \
    --user-mods-dir NEON/FATES/${neon_site} \
    --handle-preexisting-dirs r

After doing this, my case ran as expected:
/glade/derecho/scratch/wwieder/NEON_cases/STEI_FATESsp_test

If you're OK with the easiest thing may be to update your source code to the latest CTSM tag?

• note I did all of this on the command line, just pasting in the cells into a terminal window, but I don't think this should be an issue

[FiND-Tao]

Hi @wwieder , I just tried the code on derecho and I was able to create the case, build the case and submit the case. Right now, the case in the queue of derecho waiting for receiving the calculation node. I will let you know if I can get the correct results.

[wwieder]

Great news. Derecho is down at the moment, but hopefully the case runs as you expected. I should note that CLM and FATES are not calibrated for particular NEON sites, but hopefully you're able to get results that serve as a starting ground for further work and development. Thanks for reaching out.

[FiND-Tao]

@wwieder , I am able to see the simulation results on Derecho. Thanks again for your help. To help other users who want to run FATES on Derecho, I am posting my full code below. Users need to replace YOURPROJECTID with their own project ID. Users do not need to specify the location of "fsurdat".

cd ~
git clone [email protected]:ESCOMP/CTSM.git
cd CTSM
git branch
git fetch origin
git checkout ctsm5.2.001
./manage_externals/checkout_externals
cd ~/CTSM/cime/scripts
export neon_site="STEI"
./create_newcase --case ~/scratch/NEON_cases/${neon_site}_FATESsp_test2 --res CLM_USRDAT --compset I1PtClm60Fates --output-root ~/scratch/NEON_cases/ --run-unsupported --user-mods-dir NEON/FATES/${neon_site} --handle-preexisting-dirs r --project YOURPROJECTID 

cd ~/scratch/NEON_cases/STEI_FATESsp_test2

./xmlchange RUN_STARTDATE=2014-01-01 
./xmlchange STOP_OPTION=nyears
./xmlchange STOP_N=8  
./xmlchange DATM_YR_END=2021  #this is the last complete year of input data
./xmlchange CLM_FORCE_COLDSTART=on

#user_nl_clm changes
echo "use_fates_sp = .true." >> user_nl_clm
echo "soil_decomp_method = 'None'" >> user_nl_clm
echo "use_lch4 = .false." >> user_nl_clm
echo "fates_spitfire_mode = 0" >> user_nl_clm
echo "use_fates_fixed_biogeog = .true." >> user_nl_clm
echo "use_fates_nocomp = .true." >> user_nl_clm
echo "hist_fincl2 = 'FCEV','FCTR','FGEV','FIRA','FSA','FSH',
 'FATES_GPP','FATES_GPP_PF','H2OSOI','SNOW_DEPTH','TBOT','TSOI'" >> user_nl_clm

./case.setup
./case.build
./case.submit

#Check job status
qstat -u $USER

#Check output
# find the output folder of the case
./xmlquery -p DOUT
cd ~/scratch/NEON_cases/archive/STEI_FATESsp_test2