ECMWF CDS migration guide

[yuankailiu]

The old instance of the CDS will be decommissioned on 26 September 2024 and will no longer be accessible from this date onwards. The new CDS-beta will become the official new CDS for ERA5 data distribution. We have updated PyAPS to adapt to this change in version 0.3.5. Here is how to migrate to this new version:

UPDATE (Feb 8, 2025): The new CDS (renamed from CDS-beta) has become the new official instance. We have updated PyAPS to this change in version 0.3.6 and the migration guide below.

1. Update your pyaps3 to version 0.3.6+

via conda

conda update pyaps3

or via mamba

mamba update pyaps3

or via pip

pip install pyaps3 --upgrade

2. Register a new CDS account

The old CDS account won't work, so you need to register a new CDS account and accept the Term of Use of ERA5 data if you haven't, check README.

3. Update your local account credential config file

1) the ~/.cdsapirc file [recommended]:

from (for the old CDS)

url: https://cds.climate.copernicus.eu/api/v2
key = UID:THIS-IS-YOUR-API-KEY

to (for the new CDS)

url: https://cds.climate.copernicus.eu/api
key = YOUR-CDS-PERSONAL-ACCESS-TOKEN

2) or the model.cfg file:

from (for the old CDS)

[CDS]
key = UID:THIS-IS-YOUR-API-KEY

to (for the new CDS)

[CDS]
key = YOUR-CDS-PERSONAL-ACCESS-TOKEN

4. Test the new setup

cd ~/tools/
python $YOUR_PYAPS_HOME/tests/test_dload.py

The output should be like below:

------------------------------------------------
import pyaps3 from /home/ykliu/apps/mambaforge/envs/insar/lib/python3.12/site-packages/pyaps3/__init__.py
------------------------------------------------
test ERA5 data download
NOTE: Account setup is required on the Copernicus Climate Data Store (CDS).
      More detailed info can be found on: https://cds.climate.copernicus.eu/how-to-api
      Add your account info to ~/.cdsapirc file.
INFO: You are using the latest ECMWF platform for downloading datasets:  https://cds.climate.copernicus.eu/api
2025-02-08 14:23:45,598 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-02-08 14:23:45,599 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
Downloading 1 of 2: /home/ykliu/apps/PyAPS/tests/data/ERA5/ERA5_N30_N40_E120_E140_20200601_14.grb 
{'product_type': ['reanalysis'], 'variable': ['geopotential', 'temperature', 'specific_humidity'], 'year': ['2020'], 'month': ['06'], 'day': ['01'], 'time': ['14:00'], 'pressure_level': ['1', '2', '3', '5', '7', '10', '20', '30', '50', '70', '100', '125', '150', '175', '200', '225', '250', '300', '350', '400', '450', '500', '550', '600', '650', '700', '750', '775', '800', '825', '850', '875', '900', '925', '950', '975', '1000'], 'data_format': 'grib', 'area': [40, 120, 30, 140]}
2025-02-08 14:23:46,619 INFO Request ID is ab0c6db0-f759-46c0-8087-596528061253
2025-02-08 14:23:47,133 INFO status has been updated to accepted
2025-02-08 14:23:53,411 INFO status has been updated to running
2025-02-08 14:23:57,216 INFO status has been updated to successful
2025-02-08 14:24:04,951 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-02-08 14:24:04,951 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
Downloading 2 of 2: /home/ykliu/apps/PyAPS/tests/data/ERA5/ERA5_N30_N40_E120_E140_20200901_14.grb 
{'product_type': ['reanalysis'], 'variable': ['geopotential', 'temperature', 'specific_humidity'], 'year': ['2020'], 'month': ['09'], 'day': ['01'], 'time': ['14:00'], 'pressure_level': ['1', '2', '3', '5', '7', '10', '20', '30', '50', '70', '100', '125', '150', '175', '200', '225', '250', '300', '350', '400', '450', '500', '550', '600', '650', '700', '750', '775', '800', '825', '850', '875', '900', '925', '950', '975', '1000'], 'data_format': 'grib', 'area': [40, 120, 30, 140]}
2025-02-08 14:24:06,075 INFO Request ID is 9be3e9f8-0d8a-48e2-826f-b802b448ec60
2025-02-08 14:24:07,101 INFO status has been updated to accepted
2025-02-08 14:24:13,757 INFO status has been updated to running
2025-02-08 14:24:17,647 INFO status has been updated to successful
------------------------------------------------
Downloads OK
------------------------------------------------