Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bring recent updates from develop and configure reforecast #3015

Merged
Show file tree
Hide file tree
Changes from 68 commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
85c7659
Support coupling on AWS (#2859)
weihuang-jedi Aug 28, 2024
5ccdd8b
Correct ocean `MOM.res_#` stage copy (#2868)
KateFriedman-NOAA Aug 29, 2024
7f160f6
Add diffusion/diag B for aerosol DA and some other needed changes (#2…
CoryMartin-NOAA Aug 29, 2024
66fc89c
Add 3 and 9 hr increment files to IC staging (#2876)
AndrewEichmann-NOAA Aug 30, 2024
5c304b5
Support global-workflow GEFS C48 on Google Cloud (#2861)
weihuang-jedi Sep 4, 2024
2e4f4b7
Add ability to run CI test C96_atm3DVar.yaml to Gaea-C5 (#2885)
DavidBurrows-NCO Sep 4, 2024
d3ea8e2
Update aerosol climatology to 2013-2024 mean (#2888)
AnningCheng-NOAA Sep 6, 2024
6519211
Eliminate race conditions and remove DATAROOT last in cleanup (#2893)
DavidHuber-NOAA Sep 6, 2024
ac93a9b
Reenable Orion Cycling Support (#2877)
DavidHuber-NOAA Sep 7, 2024
2816c3b
Add an archive task to GEFS system to archive files locally (#2816)
AntonMFernando-NOAA Sep 7, 2024
b8080cd
Restructure the bufr sounding job (#2853)
BoCui-NOAA Sep 7, 2024
49f697a
Create JEDI class (#2805)
DavidNew-NOAA Sep 7, 2024
0953c0f
Add FAQ page and a caution with using reserved variables (#2898)
aerorahul Sep 9, 2024
b443915
Consolidate python configuration dictionaries (#2839)
DavidHuber-NOAA Sep 10, 2024
6ea681b
Replace APRUN with APRUN_default (#2871)
DavidHuber-NOAA Sep 11, 2024
108db4d
Add new UPP links to .gitignore (#2904)
DavidHuber-NOAA Sep 12, 2024
4ad9695
Disable native grid writes for non-JEDI experiments; update C384 comp…
aerorahul Sep 13, 2024
d866982
Support gefs C48 on Azure (#2881)
weihuang-jedi Sep 13, 2024
19455ba
add 1 deg ocean/ice info to parm/config/gfs/config.resources (#2922)
jswhit2 Sep 14, 2024
a13bbef
Build GDASApp and unset memory in Gaea-C5 xml files (#2912)
DavidBurrows-NCO Sep 14, 2024
b7f71e0
Cleanup job for GEFS (#2919)
AntonMFernando-NOAA Sep 14, 2024
03ee9f8
Update config.resources for bufr sounding job postsnd (#2917)
BoCui-NOAA Sep 16, 2024
2602eac
Update global atmos upp job to use COMIN/COMOUT (#2867)
mingshichen-noaa Sep 16, 2024
7588d2b
Update to obsproc/v1.2.0 and prepobs/v1.1.0 (#2903)
KateFriedman-NOAA Sep 18, 2024
c6e3262
Remove the GTS BUFR2IODA part of the snow obs prep job (#2900)
jiaruidong2017 Sep 19, 2024
3c86873
Adjust C768 resources for Hera (#2819)
DavidHuber-NOAA Sep 20, 2024
7bdb362
Update global atmos vminmon job with COMIN/COMOUT for COM prefix (#2939)
mingshichen-noaa Sep 21, 2024
fe57bb4
Remove need for a ice_prod dependency check script (#2809)
EricSinsky-NOAA Sep 23, 2024
ec63492
Refine issue and PR templates (#2947)
WalterKolczynski-NOAA Sep 23, 2024
7088a91
Add an archive task to GEFS system to archive files in HPSS (#2895)
AntonMFernando-NOAA Sep 25, 2024
a694cf1
Move the variational scripts to this repository (#2920)
guillaumevernieres Sep 26, 2024
a443fd1
Enable parallel metp jobs and fix race condition with the gfscleanup …
DavidHuber-NOAA Sep 27, 2024
386aa6d
Add GDA request template (#2948)
WalterKolczynski-NOAA Sep 27, 2024
e2f56f5
Replace separate JEDI radiance bias correction files with tarball (#2…
RussTreadon-NOAA Sep 30, 2024
8f0541c
Move to contrib installation of spack-stack on Jet (#2878)
InnocentSouopgui-NOAA Sep 30, 2024
330e74b
Add corrections to the variable names for GEFS (#2969)
EricSinsky-NOAA Oct 7, 2024
a42c833
Adding a CI for using Replay ICs with an offset hour of 3 (#2788)
NeilBarton-NOAA Oct 8, 2024
a0771f8
Update rocoto task names (#2981)
WalterKolczynski-NOAA Oct 10, 2024
ac3cde5
Update memory settings in config.resources for job postsnd (#3003)
BoCui-NOAA Oct 11, 2024
44c9279
Merge remote-tracking branch 'EMC/develop' into feature/update_refore…
EricSinsky-NOAA Oct 15, 2024
9db297a
Fix indentation for repair_replay task
EricSinsky-NOAA Oct 15, 2024
9d770fd
Update task name for repair_replay
EricSinsky-NOAA Oct 18, 2024
fb40a90
Define reanalysis in config.base
EricSinsky-NOAA Oct 18, 2024
7989a72
Bring back grouped post jobs for GEFS
EricSinsky-NOAA Oct 22, 2024
c08d761
Fix pynorms
EricSinsky-NOAA Oct 22, 2024
aa3be1c
Fix grouped ensstat dependency
EricSinsky-NOAA Oct 22, 2024
c50b90a
Fix extractvars dependency
EricSinsky-NOAA Oct 22, 2024
3391b87
Update PE configuration for reforecast
EricSinsky-NOAA Oct 23, 2024
6bb1c2c
Add ensemble information to products
EricSinsky-NOAA Oct 23, 2024
d74b18d
Move PEVPR from b to a group
EricSinsky-NOAA Oct 8, 2024
a736269
Add tob to ocean products
EricSinsky-NOAA Oct 8, 2024
0bf197c
Update gfs_utils hash
EricSinsky-NOAA Oct 10, 2024
fadd224
Fix shellnorm errors
EricSinsky-NOAA Oct 23, 2024
42abed1
Set restart_interval to FHMAX_GFS
EricSinsky-NOAA Oct 24, 2024
e9aedd1
Fix shellnorm errors
EricSinsky-NOAA Oct 24, 2024
1138de2
Replace gfs_cyc with an interval (#2928)
WalterKolczynski-NOAA Oct 22, 2024
59d20f8
Allow DO_CA condition to be used for all members
EricSinsky-NOAA Oct 24, 2024
312147d
Define base_seed for control member
EricSinsky-NOAA Oct 24, 2024
0d7a3dc
Include YYMMDDHH in the job scheduler job name
EricSinsky-NOAA Oct 24, 2024
3197b6b
Update defaults.yaml for reforecast
EricSinsky-NOAA Oct 24, 2024
142c253
Remove duplicate OCNSPPT and EPBL settings
EricSinsky-NOAA Oct 24, 2024
4f13fca
Add option to download ICs from HPSS for GEFS
EricSinsky-NOAA Oct 25, 2024
18ea885
Fix pynorm errors
EricSinsky-NOAA Oct 25, 2024
4087d3a
Add aws module to workflow
EricSinsky-NOAA Oct 26, 2024
8eec685
Add ability to download and stage replay analysis
EricSinsky-NOAA Oct 26, 2024
e28d2dc
Define location of analysis in repair_replay task
EricSinsky-NOAA Oct 26, 2024
53a7210
Fix pynorms error
EricSinsky-NOAA Oct 26, 2024
00a7bb1
Move where subprocess is imported
EricSinsky-NOAA Oct 26, 2024
3e43d62
Add option to copy analysis data locally
EricSinsky-NOAA Oct 28, 2024
059b09d
Use which from wxflow instead of subprocess
EricSinsky-NOAA Oct 28, 2024
052080e
Remove unused Python packages
EricSinsky-NOAA Oct 28, 2024
f5d5d85
Remove unused tar_cmd and hsi variables
EricSinsky-NOAA Oct 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/NCO_bug_report.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: NCO Bug report
name: NCO Bug Report
description: Report something that is incorrect or broken
labels: ["nco-bug", "triage"]
assignees:
Expand Down
14 changes: 12 additions & 2 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Bug report
name: Bug Report
description: Report something that is incorrect or broken
labels: ["bug", "triage"]

Expand All @@ -9,6 +9,8 @@ body:
Your bug may already be reported!
Please search on the [Issue tracker](https://github.com/NOAA-EMC/global-workflow/issues) before creating one.

Is this actually a workflow bug? If not, please open an issue in the appropriate repository first.

- type: textarea
id: current_behavior
attributes:
Expand Down Expand Up @@ -46,6 +48,13 @@ body:
validations:
required: true

- type: input
id: hash
attributes:
label: What global-workflow hash are you using?
validations:
required: true

- type: textarea
id: reproduction_steps
attributes:
Expand All @@ -63,8 +72,9 @@ body:
attributes:
label: Additional information
description: Provide context or any additional information about the bug.
placeholder: Optional
validations:
required: true
required: false

- type: textarea
id: propsed_implementation
Expand Down
37 changes: 37 additions & 0 deletions .github/ISSUE_TEMPLATE/dump_request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Global Observation Dump Request
description: Request additional dates be added to a machine's global dump archive (GDA) or introduce experimental dump data to the GDA
labels: ["Static Data Mgmt"]
assignees:
- KateFriedman-NOAA
- WalterKolczynski-NOAA

body:
- type: dropdown
attributes:
label: Machine
options:
- WCOSS2
- Hera/Ursa
- Orion/Hercules
- Jet
multiple: true
validations:
required: true

- type: input
attributes:
label: Start date
validations:
required: true

- type: input
attributes:
label: End date
validations:
required: true

- type: textarea
attributes:
label: Additional information
placeholder: |
Any additional information needed (experimental obs, etc.)
17 changes: 13 additions & 4 deletions .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Feature request
name: Feature Request
description: Request new capability
labels: ["feature", "triage"]

Expand All @@ -10,8 +10,7 @@ body:
- Something is wrong or broken
- It is a request from NCO
- It is initiating a production update
- Files need to be updated or added to fix

- Files need to be updated in a directory managed by workflow CMs under "glopara"

Please search on the [Issue tracker](https://github.com/NOAA-EMC/global-workflow/issues) to make sure the feature has not already been requested to avoid duplicates.

Expand All @@ -34,7 +33,17 @@ body:
id: criteria
attributes:
label: Acceptance Criteria
placeholder: What does it mean for this issue to be complete?
description: |
A checklist of criteria to be satisfied before this feature is considered complete.

Examples:
- Forecast with option X completes successfully
- File X produced
- Output unchanged
placeholder: |
- [ ] Criterion #1
- [ ] Criterion #2
- [ ] Criterion #3
validations:
required: true

Expand Down
97 changes: 0 additions & 97 deletions .github/ISSUE_TEMPLATE/fix_file.yml

This file was deleted.

29 changes: 15 additions & 14 deletions .github/ISSUE_TEMPLATE/production_update.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,18 +33,19 @@ body:
validations:
required: true

- type: markdown
- type: checkboxes
attributes:
value: |
** Tasks **
- [ ] Create release branch
- [ ] Make workflow changes for upgrade in release branch (add additional checklist items as needed)
- [ ] Create release notes
- [ ] Cut hand-off tag for CDF
- [ ] Submit CDF to NCO
- [ ] Implementation into operations complete
- [ ] Merge release branch into operational branch
- [ ] Cut version tag from operational branch
- [ ] Release new version tag
- [ ] Announce to users
- [ ] Update Read-The-Docs operations status version in develop
label: Tasks
description: List of tasks to complete update. For use after issue is created.
options:
- label: Create release branch
- label: Make workflow changes for upgrade in release branch (add additional checklist items as needed)
- label: Create release notes
- label: Cut hand-off tag for CDF
- label: Submit CDF to NCO
- label: Implementation into operations complete
- label: Merge release branch into operational branch
- label: Cut version tag from operational branch
- label: Release new version tag
- label: Announce to users
- label: Update Read-The-Docs operations status version in develop
134 changes: 134 additions & 0 deletions .github/ISSUE_TEMPLATE/static_data.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
name: Static Data Update
description: Request static data be added or updated
labels: ["Static Data Mgmt"]
assignees:
- KateFriedman-NOAA
- WalterKolczynski-NOAA

body:
- type: dropdown
attributes:
label: Type of static data
description: |
- Fix: large static files needed to run global-workflow
- Initial conditions: ICs needed to run one of global-workflow's included tests
- Experimental data: Temporary datasets needed for development that will be replaced by other sources for operations (e.g. data that will be in DCOM)
options:
- Fix
- Initial conditions
- Experimental data
validations:
required: true

- type: dropdown
attributes:
label: Target directory
options:
- "Fix: aer"
- "Fix: am"
- "Fix: chem"
- "Fix: cice"
- "Fix: cpl"
- "Fix: datm"
- "Fix: gdas/crtm"
- "Fix: gdas/fv3jedi"
- "Fix: gdas/gsibec"
- "Fix: gldas"
- "Fix: glwu"
- "Fix: gsi"
- "Fix: lut"
- "Fix: mom6"
- "Fix: orog"
- "Fix: raw"
- "Fix: reg2grb2"
- "Fix: sfc_climo"
- "Fix: ugwd"
- "Fix: verif"
- "Fix: wave"
- "Fix: *NEW*"
- "ICs: C48/C48 mx500"
- "ICs: C96/C48"
- "ICs: C96 mx100"
- "ICs: C384/C192"
- "ICs: C384 mx025"
- "ICs: C768 mx025"
- "ICs: C1152 mx025"
- "ICs: *NEW RESOLUTION*"
- "Exp Data: GOCART emissions"
- "Exp Data: JEDI obs"
- "Exp Data: *NEW*"
multiple: true
validations:
required: true

- type: checkboxes
attributes:
label: Type of change
options:
- label: Update existing files (Version update needed)
- label: Remove files (Version update needed)
- label: Add new files to existing directory
- label: Add new directory (Version update needed if fix)
validations:
required: true

- type: dropdown
attributes:
label: Any workflow changes needed?
description: |
ANY fix or IC change other than adding files will require at least a new version.
options:
- No change needed
- Fix/IC version update only
- Fix/IC version update + additional changes
- Non-version changes only
validations:
required: true

- type: textarea
attributes:
label: |
Please list all related issues. If this request requires a workflow update (including a fix version update), please open a *separate* global-workflow issue to track the change.
description: Please link any related issues
placeholder: NOAA-EMC/repo#1234
validations:
required: false

- type: input
attributes:
label: Pickup location
description: |
Please provide a path to the data on either Hera or WCOSS.

If this requires a version update, please provide a CLEAN copy of the *entire* directory, including the structure and unchanged files from current directory.

If this is just adding additional files, you may provide just the new files. The directory structure should still match the target.
placeholder: '/path/to/new/data'
validations:
required: true

- type: input
attributes:
label: Size
description: How much data needs to be copied (size of pickup location in MB)?
placeholder: '0'
validations:
required: true

- type: textarea
attributes:
label: Additional information
description: |
Please provide any additional information needed for this request. If this is a new directory, please provide a short description and a point of contact.
validations:
required: false

- type: markdown
attributes:
value: |
# Tasks
- [ ] Discuss needs with global-workflow developer assigned to request
- [ ] Stage a CLEAN copy of data on Hera or WCOSS in the location provided above
- [ ] [global-workflow CM] Update data in "glopara"-managed space on supported platforms
- [ ] [Fix only] Update "Fix File Management" [spreadsheet](https://docs.google.com/spreadsheets/d/1BeIvcz6TO3If4YCqkUK-oz_kGS9q2wTjwLS-BBemSEY/edit?usp=sharing)
- [ ] Make related workflow/component updates (if any)
Loading
Loading