Skip to content

Commit

Permalink
Merge branch 'main' into netcdf
Browse files Browse the repository at this point in the history
  • Loading branch information
LinhHo authored Dec 13, 2024
2 parents 7cbf498 + 13b23e5 commit 958a569
Show file tree
Hide file tree
Showing 30 changed files with 1,316 additions and 1,050 deletions.
57 changes: 0 additions & 57 deletions .github/workflows/build-docs.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ jobs:
#------------------------------------------------
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --with dev,docs,optional_io_formats,optional_plotting,tutorials,wbdata --no-root
run: poetry install --no-interaction --with calamine,dev,docs,optional_io_formats,optional_plotting,tutorials,wbdata --no-root

#------------------------
# install root project
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pytest-legacy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ jobs:
#------------------------------------------------
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --with dev,optional_io_formats,optional_plotting,tutorials --no-root
run: poetry install --no-interaction --with calamine,dev,optional_io_formats,optional_plotting,tutorials --no-root

#------------------------
# install root project
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
# install your project
#------------------------
- name: Install library
run: poetry install --no-interaction --with dev,optional_io_formats,optional_plotting,tutorials,wbdata
run: poetry install --no-interaction --with calamine,dev,optional_io_formats,optional_plotting,tutorials,wbdata

# run tests without Matplotlib & CodeCode tests on earlier Python versions
- name: Test with pytest
Expand Down
39 changes: 26 additions & 13 deletions RELEASE_NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,9 +120,11 @@ method. This feature is now implemented via the **nomenclature.RegionProcessor**
- [#773](https://github.com/IAMconsortium/pyam/pull/773) Remove `map_regions()` and default mappings csv
- [#772](https://github.com/IAMconsortium/pyam/pull/772) Show all missing rows for `require_data()`
- [#771](https://github.com/IAMconsortium/pyam/pull/771) Refactor to start a separate validation module
- [#766](https://github.com/IAMconsortium/pyam/pull/766) Use **ixmp4** for credentials to access a Scenario Explorer database
- [#766](https://github.com/IAMconsortium/pyam/pull/766) Use **ixmp4** for credentials to access a Scenario Explorer
database
- [#764](https://github.com/IAMconsortium/pyam/pull/764) Clean-up exposing internal methods and attributes
- [#763](https://github.com/IAMconsortium/pyam/pull/763) Implement a fix against carrying over unused levels when initializing from an indexed pandas object
- [#763](https://github.com/IAMconsortium/pyam/pull/763) Implement a fix against carrying over unused levels when
initializing from an indexed pandas object
- [#759](https://github.com/IAMconsortium/pyam/pull/759) Excise "exclude" column from meta and add a own attribute
- [#747](https://github.com/IAMconsortium/pyam/pull/747) Drop support for Python 3.7

Expand Down Expand Up @@ -183,10 +185,12 @@ Bump minimum version of **pandas** to v1.2.0 to support automatic engine selecti
## Individual updates

- [#715](https://github.com/IAMconsortium/pyam/pull/715) Add a `require_data()` method
- [#713](https://github.com/IAMconsortium/pyam/pull/713) Informative error when using lists for filter by level, `level` now a forbidden column.
- [#713](https://github.com/IAMconsortium/pyam/pull/713) Informative error when using lists for filter by level, `level`
now a forbidden column.
- [#709](https://github.com/IAMconsortium/pyam/pull/709) Hotfix ops to support `fillna=0`
- [#708](https://github.com/IAMconsortium/pyam/pull/708) Remove 'xls' as by-default-supported file format
- [#686](https://github.com/IAMconsortium/pyam/pull/686) Add support for (weighted) quantile timeseries as `df.compute.quantiles()` with a [tutorial](https://pyam-iamc.readthedocs.io/en/stable/tutorials/quantiles.html)
- [#686](https://github.com/IAMconsortium/pyam/pull/686) Add support for (weighted) quantile timeseries as
`df.compute.quantiles()` with a [tutorial](https://pyam-iamc.readthedocs.io/en/stable/tutorials/quantiles.html)

# Release v1.6.0

Expand All @@ -204,15 +208,18 @@ dependency for better performance.
## Individual updates

- [#702](https://github.com/IAMconsortium/pyam/pull/702) Migrate `compute_bias()` to `compute` module
- [#701](https://github.com/IAMconsortium/pyam/pull/701) Add **xlsxwriter** as dependency to improve `to_excel()` performance
- [#699](https://github.com/IAMconsortium/pyam/pull/699) Add filter options to IIASA API `index()`, `meta()` and `properties()` methods
- [#701](https://github.com/IAMconsortium/pyam/pull/701) Add **xlsxwriter** as dependency to improve `to_excel()`
performance
- [#699](https://github.com/IAMconsortium/pyam/pull/699) Add filter options to IIASA API `index()`, `meta()` and
`properties()` methods
- [#697](https://github.com/IAMconsortium/pyam/pull/697) Add warning if IIASA API returns empty result
- [#696](https://github.com/IAMconsortium/pyam/pull/696) Added ability to load preferentially from a local cache
- [#695](https://github.com/IAMconsortium/pyam/pull/695) Remove unused meta levels during initialization
- [#688](https://github.com/IAMconsortium/pyam/pull/688) Remove ixmp as optional dependency
- [#684](https://github.com/IAMconsortium/pyam/pull/684) Use new IIASA-manager API with token refresh
- [#679](https://github.com/IAMconsortium/pyam/pull/679) `set_meta()` now supports pandas.DataFrame as an argument
- [#674](https://github.com/IAMconsortium/pyam/pull/674) Support filtering data by model-scenario pairs with the `index` argument to `filter()` and `slice()`
- [#674](https://github.com/IAMconsortium/pyam/pull/674) Support filtering data by model-scenario pairs with the `index`
argument to `filter()` and `slice()`

# Release v1.5.0

Expand All @@ -224,7 +231,8 @@ class that allows faster filtering and inspection of an **IamDataFrame**.
## Individual updates

- [#668](https://github.com/IAMconsortium/pyam/pull/668) Allow renaming of empty IamDataFrame objects
- [#665](https://github.com/IAMconsortium/pyam/pull/665) Provide better support for IamDataFrame objects with non-standard index dimensions
- [#665](https://github.com/IAMconsortium/pyam/pull/665) Provide better support for IamDataFrame objects with
non-standard index dimensions
- [#659](https://github.com/IAMconsortium/pyam/pull/659) Add an `offset` method
- [#657](https://github.com/IAMconsortium/pyam/pull/657) Add an `IamSlice` class

Expand Down Expand Up @@ -253,7 +261,8 @@ an empty **IamDataFrame**. Previously, this raised an error.

## Individual updates

- [#651](https://github.com/IAMconsortium/pyam/pull/651) Pin `pint<=0.18` as a quickfix for a regression in the latest release
- [#651](https://github.com/IAMconsortium/pyam/pull/651) Pin `pint<=0.18` as a quickfix for a regression in the latest
release
- [#650](https://github.com/IAMconsortium/pyam/pull/650) Add IPCC AR6 WGIII colors to PYAM_COLORS
- [#647](https://github.com/IAMconsortium/pyam/pull/647) Pin `unfccc-di-api` to latest release
- [#634](https://github.com/IAMconsortium/pyam/pull/634) Better error message when initializing with invisible columns
Expand All @@ -274,13 +283,15 @@ pandas [v1.4.0](https://pandas.pydata.org/docs/whatsnew/v1.4.0.html).

## Individual updates

- [#608](https://github.com/IAMconsortium/pyam/pull/608) The method `assert_iamframe_equals()` passes if an all-nan-col is present
- [#608](https://github.com/IAMconsortium/pyam/pull/608) The method `assert_iamframe_equals()` passes if an all-nan-col
is present
- [#604](https://github.com/IAMconsortium/pyam/pull/604) Add an annualized-growth-rate method
- [#602](https://github.com/IAMconsortium/pyam/pull/602) Add a `compute` module/accessor and a learning-rate method
- [#600](https://github.com/IAMconsortium/pyam/pull/600) Add a `diff()` method
- [#592](https://github.com/IAMconsortium/pyam/pull/592) Fix for running in jupyter-lab notebooks
- [#590](https://github.com/IAMconsortium/pyam/pull/590) Update expected figures of plotting tests to use matplotlib 3.5
- [#586](https://github.com/IAMconsortium/pyam/pull/586) Improve error reporting for non-numeric data in any value column
- [#586](https://github.com/IAMconsortium/pyam/pull/586) Improve error reporting for non-numeric data in any value
column

# Release v1.2.0

Expand All @@ -298,10 +309,12 @@ was added as a dependency.

## Individual updates

- [#585](https://github.com/IAMconsortium/pyam/pull/585) Include revisions to the ORE manuscript source code following acceptance/publication
- [#585](https://github.com/IAMconsortium/pyam/pull/585) Include revisions to the ORE manuscript source code following
acceptance/publication
- [#583](https://github.com/IAMconsortium/pyam/pull/583) Add profiler module for performance benchmarking
- [#579](https://github.com/IAMconsortium/pyam/pull/579) Increase performance of IamDataFrame initialization
- [#572](https://github.com/IAMconsortium/pyam/pull/572) Unpinned the requirements for xlrd and added openpyxl as a requirement to ensure ongoing support of both `.xlsx` and `.xls` files out of the box
- [#572](https://github.com/IAMconsortium/pyam/pull/572) Unpinned the requirements for xlrd and added openpyxl as a
requirement to ensure ongoing support of both `.xlsx` and `.xls` files out of the box

# Release v1.1.0

Expand Down
1 change: 0 additions & 1 deletion docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ and methods.
api/general
api/iamdataframe
api/database
api/slice
api/filtering
api/compute
api/plotting
Expand Down
2 changes: 1 addition & 1 deletion docs/api/database.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Reading from an |ixmp4| platform
--------------------------------

The |pyam| package provides a simple interface to read timeseries data and meta
indicators from local or remote |ixmp4| platform instancs.
indicators from local or remote |ixmp4| platform instances.

.. autofunction:: read_ixmp4

Expand Down
77 changes: 69 additions & 8 deletions docs/api/filtering.rst
Original file line number Diff line number Diff line change
@@ -1,12 +1,73 @@
.. currentmodule:: pyam

Advanced filtering
==================
Filtering and slicing
=====================

|pyam| includes a function to directly downselect a :class:`pandas.DataFrame`
with appropriate columns or index dimensions
(i.e., :code:`['model', 'scenario']`)
using a :class:`IamDataFrame` and keyword arguments similar
to the :meth:`IamDataFrame.filter` function.
Arguments for filtering an :class:`IamDataFrame`
------------------------------------------------

.. autofunction:: filter_by_meta
The |pyam| package provides several methods to filter an :class:`IamDataFrame` by its
(timeseries) **data** or **meta** values. Read more about the `Data Model <data.html>`_
that is implemented by an :class:`IamDataFrame`.

The following arguments are available for filtering and can be combined as needed:

Index
^^^^^
- A *column* of the :attr:`IamDataFrame.index`
(usually '**model**' and '**scenario**'): string or list of strings
- '**index**': list of model/scenario-tuples or a :class:`pandas.MultiIndex`

Timeseries data coordinates
^^^^^^^^^^^^^^^^^^^^^^^^^^^
- Any *column* of the :attr:`IamDataFrame.coordinates <pyam.IamDataFrame.coordinates>`
('**region**', '**variable**', '**unit**'): string or list of strings
- '**measurand**': a tuple (or list of tuples) of '*variable*' and '*unit*'
- '**depth**': the "depth" of entries in the '*variable*' column (number of '|')
- '**level**': the "depth" of entries in the '*variable*' column (number of '|'),
excluding the strings in the '*variable*' argument (if given)
- '**year**': takes an integer (int/:class:`numpy.int64`), a list of integers or
a range. Note that the last year of a range is not included,
so ``range(2010, 2015)`` is interpreted as ``[2010, ..., 2014]``
- '**time_domain**': can be 'year' or 'datetime'
- Arguments for filtering by :class:`datetime.datetime` or :class:`numpy.datetime64`
('**month**', '**hour**', '**time**')

Meta indicators and other attributes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- Any *column* of the :attr:`IamDataFrame.meta <pyam.IamDataFrame.meta>` dataframe:
string, integer, float, or list of these
- '**exclude**' (see :attr:`IamDataFrame.exclude <pyam.IamDataFrame.exclude>`): boolean

.. note::

In any string filters, '*' is interpreted as wildcard, unless the keyword argument
*regexp=True* is used; in this case, strings are treated as
`regular expressions <https://docs.python.org/3/library/re.html>`_.

Methods for filtering and slicing an :class:`IamDataFrame`
----------------------------------------------------------

.. automethod:: pyam.IamDataFrame.filter
:noindex:

.. automethod:: pyam.IamDataFrame.slice
:noindex:

The **IamSlice** class
----------------------

This class is an auxiliary feature to streamline the implementation of the
:meth:`IamDataFrame.filter` method.

.. autoclass:: pyam.slice.IamSlice
:members: dimensions, time, info

Filtering using a proxy :class:`IamDataFrame`
---------------------------------------------

|pyam| includes a function to directly filter a :class:`pandas.DataFrame`
with appropriate columns or index dimensions (i.e.,'*model*' and '*scenario*') using
an :class:`IamDataFrame` and keyword arguments similar to :meth:`IamDataFrame.filter`.

.. autofunction:: pyam.filter_by_meta
7 changes: 0 additions & 7 deletions docs/api/slice.rst

This file was deleted.

Loading

0 comments on commit 958a569

Please sign in to comment.