Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc/v3 migration guide #2102

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Zarr-Python
:hidden:

getting_started
migration
tutorial
api/index
spec
Expand Down
2 changes: 1 addition & 1 deletion docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ latest GitHub main::

$ pip install git+https://github.com/zarr-developers/zarr-python.git

To work with Zarr source code in development, see `Contributing <contributing.html>`_.
To work with Zarr source code in development, see `Contributing <contributing.html>`_.
89 changes: 89 additions & 0 deletions docs/migration.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
Zarr-Python 3 Migration Guide
=============================

Zarr-Python 3 introduces a number of changes to the API, including a number
of significant breaking changes and pending deprecations.

This page provides a guide highlighting the most notable changes to help you
migrate your code from Zarr-Python 2.x to Zarr-Python 3.x.
jhamman marked this conversation as resolved.
Show resolved Hide resolved

Compatibility target
--------------------

Zarr-Python 3 represents a major refactor of the Zarr-Python codebase. Some of the goals motivating this refactor included:

- adding support for the V3 specification (alongside the V3 specification)
jhamman marked this conversation as resolved.
Show resolved Hide resolved
- cleaning up internal and user facing APIs
- improving performance (particularly in high latency storage environments like cloud object store)

Though these goals necessitated some breaking changes to the API (hence the major version update), we have tried to maintain
jhamman marked this conversation as resolved.
Show resolved Hide resolved
backwards compatibility in the most widely used parts of the API including the `Array` and `Group` classes and the top-level
API (e.g. `zarr.open_array` and `zarr.open_group`). It is worth noting that we significantly evolved the internal data model,
moving away from a model that was tightly coupled to the v2 specification, and to a more generic representation of Zarr objects.
jhamman marked this conversation as resolved.
Show resolved Hide resolved

Getting ready for 3.0
---------------------

Ahead of the 3.0 release, we suggest projects that depend on Zarr-Python take the following actions:

1. Pin the supported Zarr-Python version to ``zarr>=2,<3``. This is a best practice and will protect your users from any incompatibilities that may arise during the release of Zarr-Python 3.0.
2. Limit your imports from the Zarr-Python package. Most of the primary API ``zarr.*`` will be compatible in 3.0. However, the following breaking API changes are planned:
jhamman marked this conversation as resolved.
Show resolved Hide resolved

- ``numcodecs.*`` will no longer be available in ``zarr.*``. (Suggested action: transition to importing codecs from ``numcodecs`` directly.)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- ``numcodecs.*`` will no longer be available in ``zarr.*``. (Suggested action: transition to importing codecs from ``numcodecs`` directly.)
- ``zarr.numcodecs.*`` will no longer be available. These imports can be replaced by importing ``numcodecs`` directly.

We should be doing more than "suggesting", we should be providing concrete fixes/code updates!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish it were that easy! Unfortunately, the top level namespace is littered with * imports

from zarr.codecs import *

from numcodecs import *

If there is a reasonable way for us to deprecate folks using these imports, I'm all ears.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if there isn't a reasonable way to deprecate the imports, as a minimum this migration guide should provide a complete list of all imports that are disappearing and how to replace them.

- The ``zarr.v3_api_available`` feature flag is being removed. (Suggested action: this experimental feature was deprecated in v2.18.)
jhamman marked this conversation as resolved.
Show resolved Hide resolved
- The following internal modules are being removed or significant changed:
jhamman marked this conversation as resolved.
Show resolved Hide resolved

- ``zarr.attrs``
- ``zarr.codecs``
- ``zarr.context``
- ``zarr.core``
- ``zarr.hierarchy``
- ``zarr.indexing``
- ``zarr.meta``
- ``zarr.meta_v1``
- ``zarr.storage``
- ``zarr.sync``
- ``zarr.types``
- ``zarr.util``
- ``zarr.n5``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a clear list of how to update code to adapt to these removals or changes.


jhamman marked this conversation as resolved.
Show resolved Hide resolved
Continue using Zarr-Python 2
jhamman marked this conversation as resolved.
Show resolved Hide resolved
----------------------------

Zarr-Python 2.x is still available, though we recommend migrating to Zarr-Python 3 for its improvements and new features.
Security and bug fixes will be made to the 2.x series for at least 6 months following the first Zarr-Python 3 release.

If you need to use the latest Zarr-Python 2 release, you can install it with:

$ pip install "zarr==2.*"
jhamman marked this conversation as resolved.
Show resolved Hide resolved


Migration Guide
---------------

The following sections provide details on the most important changes in Zarr-Python 3.

Changes to the Array class
~~~~~~~~~~~~~~~~~~~~~~~~~~

1. Disallow direct construction

Changes to the Group class
~~~~~~~~~~~~~~~~~~~~~~~~~~

1. Disallow direct construction

Changes to the Store class
~~~~~~~~~~~~~~~~~~~~~~~~~~

Some of the biggest changes in Zarr-Python 3 are found in the `Store` class. The most notable changes to the Store API are:

1. Dropped the ``MutableMapping`` base class in favor of a custom abstract base class (``zarr.abc.store.Store``).
2. Switched to a primarily Async interface.

TODO

Miscellaneous changes
~~~~~~~~~~~~~~~~~~~~~

TODO