Skip to content

Commit

Permalink
Update README.rst (#263)
Browse files Browse the repository at this point in the history
  • Loading branch information
SandraGH5 authored Jun 4, 2021
1 parent ad5348e commit 1c73954
Showing 1 changed file with 9 additions and 10 deletions.
19 changes: 9 additions & 10 deletions cookbook/integrations/flytekit_plugins/papermilltasks/README.rst
Original file line number Diff line number Diff line change
@@ -1,20 +1,19 @@
Executing Jupyter Notebooks as tasks
=========================================
Papermill
=========

It is possible to run a Jupyter notebook as a Flyte task using `papermill <https://github.com/nteract/papermill>`_.
Papermill executes the notebook as a whole, so before using this plugin, it is essential to construct your notebook as recommended by papermill.

Salient points
---------------

#. This plugin can be used for any task - type.
- It can be straight-up python code, which can be a tensorflow model, a data transformation, etc - but things that run in a
container and you would typically write in a ``@task``.
- It can be a ``dynamic task``
- It can be a any other plugin like ``spark``, ``sagemaker`` etc, **ensure that the plugin is installed as well**
#. Flytekit will execute the notebook and capture the output notebook as an *.ipynb* and an HTML rendered notebook as well
#. Flytekit will pass the inputs into the notebook as long as you have the first cell annotated as ``parameters`` and inputs are specified
#. Flytekit will read the outputs from the notebook, as long as you use annotate the notebook with ``outputs`` and outputs are specified
1. This plugin can be used for any task - type.
- It can be straight-up python code, which can be a tensorflow model, a data transformation, etc., but typically run in a container and written in a ``@task``.
- It can be a ``dynamic task``.
- It can be a any other plugin like ``spark``, ``sagemaker``, etc., **but ensure that the plugin is installed**.
2. Flytekit will execute the notebook and capture the output notebook as an *.ipynb* and an HTML rendered notebook as well.
3. Flytekit will pass the inputs into the notebook, as long as the first cell is annotated as ``parameters`` and the inputs are specified.
4. Flytekit will read the outputs from the notebook, as long as annotate the notebook is used with ``outputs`` and the outputs are specified.


Installation
Expand Down

0 comments on commit 1c73954

Please sign in to comment.