Skip to content

Commit

Permalink
Add openvino stable diffusion pipeline documentation (#204)
Browse files Browse the repository at this point in the history
* Add openvino stable diffusion documentation

* update image

* update documentation
  • Loading branch information
echarlaix authored Mar 2, 2023
1 parent 5da170f commit 56b9deb
Showing 1 changed file with 35 additions and 0 deletions.
35 changes: 35 additions & 0 deletions docs/source/inference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,40 @@ tokenizer.save_pretrained(save_directory)
[{'translation_text': "Il n'est jamais sorti sans un livre sous son bras, et il est souvent revenu avec deux."}]
```


## Run inference for stable diffusion models

Stable diffusion models can also be used when running inference with OpenVINO. When Stable diffusion models
are exported to the OpenVINO format, they are decomposed into three components that are later combined during inference:
- The text encoder
- The U-NET
- The VAE decoder

Here is an example of how you can load an OpenVINO Stable Diffusion model and run inference using OpenVINO Runtime:

```python
from optimum.intel.openvino import OVStableDiffusionPipeline

model_id = "echarlaix/stable-diffusion-2-1-openvino"
stable_diffusion = OVStableDiffusionPipeline.from_pretrained(model_id)
prompt = "sailing ship in storm by Rembrandt"
image = stable_diffusion(prompt).images[0]
```

To load your PyTorch model and convert it to the OpenVINO format on-the-fly, simply pass `export=True` to the `from_pretrained` method.
After this step, don't forget to save your model using the `save_pretrained` method, to be able to load your OpenVINO model directly.

```python
model_id = "stabilityai/stable-diffusion-2-1"
stable_diffusion = ORTStableDiffusionPipeline.from_pretrained(model_id, export=True)

# Save the exported model
stable_diffusion.save_pretrained("a_local_path")
```

![img](https://huggingface.co/datasets/optimum/documentation-images/resolve/main/intel/openvino/stable_diffusion_v2_1_sail_boat_rembrandt.png)


## Supported tasks

As shown in the table below, each task is associated with a class enabling to automatically load your model.
Expand All @@ -139,3 +173,4 @@ As shown in the table below, each task is associated with a class enabling to au
| `masked-lm` | `OVModelForMaskedLM` |
| `causal-lm` | `OVModelForCausalLM` |
| `seq2seq-lm` | `OVModelForSeq2SeqLM` |
| `text-to-image` | `OVStableDiffusionPipeline` |

0 comments on commit 56b9deb

Please sign in to comment.