Visualization: Python scripts for NEON analyses

[wwieder]

To be included in ESM-lab materials (and elsewhere?). These can build on the example here

• https://github.com/wwieder/ctsm_py/blob/master/notebooks/ptCLM_NEON.ipynb and here
• https://github.com/wwieder/ctsm_py/blob/master/notebooks/ptCLM_multi.ipynb.

@negin has some ideas from the ESDS discussions about how this can be done
These plots should include:

• Fluxes over time (line plots w/ model & obs, mean +- SD)
  • Diel cycle
  • Seasonal or annual cycle
• Scatter plot of model state or fluxes (model vs. obs)
  • broken down by season (e.g. DJF, JJA, etc)
• depth vs. time soil temp, moisture & C stocks
• synthesis diagnostics, evaluation statistics (e.g. RMSE) across multiple sites grouped by pft.
• scatter plots of variable vs. variable to show relationships (e.g. GPP vs. incoming shortwave for LUE)
• create table of ecosystem properties (summary statistic by sites, e.g. productivity, climate, C stocks).

These are being handled in https://github.com/NCAR/NEON-visualization
see also https://github.com/negin513/neon_scripts

[wwieder]

assigned @briandobbins so we can work with him on integration into ESM-lab as this effort matures

[wwieder]

This script will have to read in separate eval_files with the NEON observations where they are provided
https://neon-ncar.s3.data.neonscience.org/NEON/eval_files/v1/BART/2018-02.nc

Gordon recommended now worrying about where these files are stored, but focusing on scripts to do monthly, seasonal, or annual plots are most valuable. This will tie in with the CICoE visualization WG

[wwieder]

This is an excellent place to leverage tools with ESDS and CESM-lab?

[ekluzek]

If this is added to CTSM it will be under contrib and as such can come to CTSM main dev without running standard testing. But as already said this may make more sense to be in CESM-lab or somewhere else.

[wwieder]

Notes from @negin513's work so far (LINK TO REPO HERE)
Point to transient runs that Jim's done

All line and scatter plots
START HIGHER PRIORITY NEEDS

• Include other derived fluxes
  • Net radiation (FSA - FIRA)
  • Latent heat (FCTR+FGEV+FCEV)
  • NEE (GPP - ER or GPP - AR - HR)
• allow plotting of other (CLM only) variables,
  • including all 1d CLM data that are not already being plotted
  • friction velocity (ustar) eventually needs to be provided from NEON eval files
• Plot gap filled NEON data with another color (using flags, e.g. GPP_fqc)
• Check unit conversion accuracy for comparison plots

END HIGHER PRIORITY NEEDS

• Clean up statistics on linear regression (Valerio), this is now the title of the plot.
• Check that linear regression line is updating.
• Add y axis labels
• add y axis units (lower priority)

Annual cycle plots

• Print integral under the curve, annual flux
• Include option to average over day of year (true annual cycle, lower priority?).

Diel fluxes

• Subset / filter by season
• Shift time so CLM-NEON align
• Shift time so local noon is midday & centered on the plot (maybe NEON can provide this metadata easily?)
• Include x-y scatter plot of CLM-NEON on plot too (this is not critical)

Vertically resolved contour plots over time

• This is working for matplotlib, but may need help for bokeh
• soil C plots should just be a line plot (depth vs. C stocks, converted to gC/m2). Time doesn't matter here.
• eventually soil temperature, soil C, soil N, and soil 14C could be compared with NEON observations, but let's not worry about this now.

Even lower priority ideas for down the road

• Use long names for variables to plot, and on axis labels
• Additional plot types:
  • components of latent heat flux (as a proportion of the total)
  • components of NEE plotted on a single plot (e.g., GPP, AR, HR)

Do we need additional output by default?

[wwieder]

As with #1385, this seems like something we want for the paper but that's not critical for the release to beta testers.

[wwieder]

We seem to have enough post processing tools around to mark this as done for now.