Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up export/import action mixin #91

Merged
merged 1 commit into from
Jan 21, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,9 @@ repos:
pass_filenames: false
types: [python]
stages: [pre-push]
- id: doc_build_verify
name: verify that docs could be build
entry: inv docs.build
language: system
pass_filenames: false
stages: [pre-push]
11 changes: 8 additions & 3 deletions HISTORY.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,12 @@ UNRELEASED
------------------

* Add explicit `created_by` argument to `CeleryResourceMixin` and pass it in
`ExportJobSerializer` validation
`ExportJobSerializer` validation
* Add export/import action mixins `api.mixins.ExportStartActionMixin`
and `api.mixins.ImportStartActionMixin`
* Add `api.views.BaseExportJobViewSet`, `BaseExportJobForUsersViewSet`,
`api.views.BaseImportJobViewSet` and `BaseImportJobForUsersViewSet` for
job management

1.3.1 (2025-01-13)
------------------
Expand All @@ -21,8 +26,8 @@ UNRELEASED
* Small actions definition refactor in `ExportJobViewSet/ExportJobViewSet` to allow easier overriding.
* Add support for ordering in `export`
* Add settings for DjangoFilterBackend and OrderingFilter in export api.
`DRF_EXPORT_DJANGO_FILTERS_BACKEND` with default `django_filters.rest_framework.DjangoFilterBackend` and
`DRF_EXPORT_ORDERING_BACKEND` with default `rest_framework.filters.OrderingFilter`.
`DRF_EXPORT_DJANGO_FILTERS_BACKEND` with default `django_filters.rest_framework.DjangoFilterBackend` and
`DRF_EXPORT_ORDERING_BACKEND` with default `rest_framework.filters.OrderingFilter`.

1.2.0 (2024-12-26)
------------------
Expand Down
Binary file added docs/_static/images/action-bands-openapi.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 18 additions & 0 deletions docs/api_drf.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,27 @@ API (Rest Framework)
.. autoclass:: import_export_extensions.api.ExportJobForUserViewSet
:members:

.. autoclass:: import_export_extensions.api.BaseImportJobViewSet
:members:

.. autoclass:: import_export_extensions.api.BaseExportJobViewSet
:members:

.. autoclass:: import_export_extensions.api.BaseImportJobForUserViewSet
:members:

.. autoclass:: import_export_extensions.api.BaseExportJobForUserViewSet
:members:

.. autoclass:: import_export_extensions.api.LimitQuerySetToCurrentUserMixin
:members:

.. autoclass:: import_export_extensions.api.ImportStartActionMixin
:members:

.. autoclass:: import_export_extensions.api.ExportStartActionMixin
:members:

.. autoclass:: import_export_extensions.api.CreateExportJob
:members: create, validate

Expand Down
20 changes: 19 additions & 1 deletion docs/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ a ``status page``, where you can monitor the progress of the import/export proce
A screenshot of Django Admin export status page

Import/Export API
-----------------
----------------------------------------------------------------

The ``api.views.ExportJobViewSet`` and ``api.views.ImportJobViewSet`` are provided to create
the corresponding viewsets for the resource.
Expand Down Expand Up @@ -169,3 +169,21 @@ the OpenAPI specification will be available.
.. figure:: _static/images/bands-openapi.png

A screenshot of the generated OpenAPI specification


Import/Export API actions mixins
TheSuperiorStanislav marked this conversation as resolved.
Show resolved Hide resolved
----------------------------------------------------------------

Alternatively you can use ``api.mixins.ExportStartActionMixin`` and ``api.mixins.ImportStartActionMixin``
to add to your current viewsets ability to create import/export jobs.
You would also need to use ``api.views.BaseExportJobViewSet`` or ``BaseExportJobForUsersViewSet``
and ``api.views.BaseImportJobViewSet`` or ``BaseImportJobForUsersViewSet`` to setup endpoints to be able to:

* ``list`` - Returns a list of jobs for the ``resource_class`` set in ViewSet.
* ``retrieve`` - Returns details of a job based on the provided ID.
* ``cancel`` - Stops the import/export process and sets the job's status to ``CANCELLED``.
* ``confirm`` - Confirms the import after the parse stage. This action is available only in import jobs.

.. figure:: _static/images/action-bands-openapi.png

A screenshot of the generated OpenAPI specification
14 changes: 13 additions & 1 deletion docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,25 @@ By default, it uses the `mimetypes.types_map <https://docs.python.org/3/library/
from Python's mimetypes module.

``STATUS_UPDATE_ROW_COUNT``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Defines the number of rows after import/export of which the task status is
updated. This helps to increase the speed of import/export. The default value
is 100. This parameter can be specified separately for each resource by adding
``status_update_row_count`` to its ``Meta``.

``DRF_EXPORT_DJANGO_FILTERS_BACKEND``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Specifies filter backend class for ``django-filters`` in export action.
Default: ``django_filters.rest_framework.DjangoFilterBackend``

``DRF_EXPORT_ORDERING_BACKEND``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Specifies filter backend class for ``ordering`` in export action.
Default: ``rest_framework.filters.OrderingFilter``

Settings from django-import-export
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Additionally, the package supports settings from the original django-import-export package.
Expand Down
2 changes: 1 addition & 1 deletion docs/migrate_from_original_import_export.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ install the package by following the :ref:`the installation guide<Installation a
Then, all you need to do is update the base classes for your resource and admin models.

Migrate resources
-----------------
----------------------------------------------------------------

To enable import/export via Celery, simply replace the base resource classes from the original package
with ``CeleryResource`` or ``CeleryModelResource`` from ``django-import-export-extensions``:
Expand Down
10 changes: 9 additions & 1 deletion import_export_extensions/api/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
from .mixins import LimitQuerySetToCurrentUserMixin
from .mixins import (
ExportStartActionMixin,
ImportStartActionMixin,
LimitQuerySetToCurrentUserMixin,
)
from .serializers import (
CreateExportJob,
CreateImportJob,
Expand All @@ -8,6 +12,10 @@
ProgressSerializer,
)
from .views import (
BaseExportJobForUserViewSet,
BaseExportJobViewSet,
BaseImportJobForUserViewSet,
BaseImportJobViewSet,
ExportJobForUserViewSet,
ExportJobViewSet,
ImportJobForUserViewSet,
Expand Down
3 changes: 3 additions & 0 deletions import_export_extensions/api/mixins/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from .common import LimitQuerySetToCurrentUserMixin
from .export_mixins import ExportStartActionMixin
from .import_mixins import ImportStartActionMixin
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@ class LimitQuerySetToCurrentUserMixin:

def get_queryset(self):
"""Return user's jobs."""
if self.action == "start":
if self.action in (
getattr(self, "import_action", ""),
getattr(self, "export_action", ""),
):
# To make it consistent and for better support of drf-spectacular
return super().get_queryset() # pragma: no cover
return (
Expand Down
141 changes: 141 additions & 0 deletions import_export_extensions/api/mixins/export_mixins.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
import collections.abc
import contextlib
import typing

from django.conf import settings
from django.utils import module_loading

from rest_framework import (
decorators,
request,
response,
status,
)

from ... import resources
from .. import serializers


class ExportStartActionMixin:
"""Mixin which adds start export action."""

resource_class: type[resources.CeleryModelResource]
export_action = "start_export_action"
export_action_name = "export"
export_action_url = "export"
export_detail_serializer_class = serializers.ExportJobSerializer
export_ordering: collections.abc.Sequence[str] = ()
export_ordering_fields: collections.abc.Sequence[str] = ()
export_open_api_description = (
"This endpoint creates export job and starts it. "
"To monitor progress use detail endpoint for jobs to fetch state of "
"job. Once it's status is `EXPORTED`, you can download file."
)

def __init_subclass__(cls) -> None:
super().__init_subclass__()
# Skip if it is has no resource_class specified
if not hasattr(cls, "resource_class"):
return
filter_backends = [
module_loading.import_string(
settings.DRF_EXPORT_DJANGO_FILTERS_BACKEND,
),
]
if cls.export_ordering_fields:
filter_backends.append(
module_loading.import_string(
settings.DRF_EXPORT_ORDERING_BACKEND,
),
)

def start_export_action(
self: "ExportStartActionMixin",
request: request.Request,
) -> response.Response:
return self.start_export(request)

setattr(cls, cls.export_action, start_export_action)
decorators.action(
methods=["POST"],
url_name=cls.export_action_name,
url_path=cls.export_action_url,
detail=False,
queryset=cls.resource_class.get_model_queryset(),
serializer_class=cls().get_export_create_serializer_class(),
filterset_class=getattr(
cls.resource_class,
"filterset_class",
None,
),
filter_backends=filter_backends,
ordering=cls.export_ordering,
ordering_fields=cls.export_ordering_fields,
)(getattr(cls, cls.export_action))
# Correct specs of drf-spectacular if it is installed
with contextlib.suppress(ImportError):
from drf_spectacular import utils

utils.extend_schema_view(
**{
cls.export_action: utils.extend_schema(
description=cls.export_open_api_description,
filters=True,
responses={
status.HTTP_201_CREATED: cls().get_export_detail_serializer_class(), # noqa: E501
},
),
},
)(cls)

def get_queryset(self):
"""Return export model queryset on export action.

For better openapi support and consistency.

"""
if self.action == self.export_action:
return self.resource_class.get_model_queryset() # pragma: no cover
OttoAndrey marked this conversation as resolved.
Show resolved Hide resolved
return super().get_queryset()

def get_export_detail_serializer_class(self):
"""Get serializer which will be used show details of export job."""
return self.export_detail_serializer_class

def get_export_create_serializer_class(self):
"""Get serializer which will be used to start export job."""
return serializers.get_create_export_job_serializer(
self.resource_class,
)

def get_export_resource_kwargs(self) -> dict[str, typing.Any]:
"""Provide extra arguments to resource class."""
return {}

def get_serializer(self, *args, **kwargs):
"""Provide resource kwargs to serializer class."""
if self.action == self.export_action:
kwargs.setdefault(
"resource_kwargs",
self.get_export_resource_kwargs(),
)
return super().get_serializer(*args, **kwargs)

def start_export(self, request: request.Request) -> response.Response:
"""Validate request data and start ExportJob."""
ordering = request.query_params.get("ordering", "")
if ordering:
ordering = ordering.split(",")
serializer = self.get_serializer(
data=request.data,
ordering=ordering,
filter_kwargs=request.query_params,
)
serializer.is_valid(raise_exception=True)
export_job = serializer.save()
return response.Response(
data=self.get_export_detail_serializer_class()(
instance=export_job,
).data,
status=status.HTTP_201_CREATED,
)
Loading
Loading