Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Tensorflow demo #17

Draft
wants to merge 10 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added tf-mobilenet-image/PurpleGallinule.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
68 changes: 68 additions & 0 deletions tf-mobilenet-image/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Mobilenet Example For WASI-NN with Tensorflow Backend

This package is a high-level Rust bindings for [wasi-nn] example of Mobilenet with Tensorflow backend.

[wasi-nn]: https://github.com/second-state/wasmedge-wasi-nn

## Dependencies

This crate depends on the `wasi-nn` in the `Cargo.toml`:

```toml
[dependencies]
wasmedge-wasi-nn = "0.2.1"
```

## Build

Compile the application to WebAssembly:

```bash
cd rust/tf-mobilenet && cargo build --target=wasm32-wasi --release
```

The output WASM file will be at [`rust/tf-mobilenet/target/wasm32-wasi/release/wasmedge-wasinn-example-tf-mobilenet-image.wasm`](wasmedge-wasinn-example-tf-mobilenet-image.wasm).
To speed up the image processing, we can enable the AOT mode in WasmEdge with:

```bash
wasmedgec rust/tf-mobilenet/target/wasm32-wasi/release/wasmedge-wasinn-example-tf-mobilenet-image.wasm run.wasm
```

## Run

### Download fixture

The testing image is located at `./PurpleGallinule.jpg` downloaded from [link](https://raw.githubusercontent.com/second-state/wasm-learning/master/rust/mobilenet_birds_tfhub/PurpleGallinule.jpg).

The `tf` model is located at `./saved_model.pb` downloaded from [link](https://tfhub.dev/google/aiy/vision/classifier/birds_V1/1). Please note that, the model should be saved by tensorflow [SavedModel](https://www.tensorflow.org/guide/saved_model) format with no extra assets and variables.

### Generate Image Tensor

If you want to generate the [raw](birdx224x224x3.rgb) tensor, you can run:

```shell
cd rust/image-converter/ && cargo run ../../PurpleGallinule.jpg ../../birdX224X224X3.rgb && cd ../..
```

### Execute

Execute the WASM with the `wasmedge` with Tensorflow supporting:

```bash
wasmedge --dir .:. wasmedge-wasinn-example-tf-mobilenet-image.wasm saved_model.pb PurpleGallinule.jpg
```

You will get the output:

```console
Read graph weights, size in bytes: 3561598
Loaded graph into wasi-nn with ID: 0
Created wasi-nn execution context with ID: 0
Read input tensor, size in bytes: 150528
Executed graph inference
1.) [576](0.8951)Porphyrio martinicus
2.) [427](0.0123)Cynanthus latirostris
3.) [331](0.0064)Porphyrio poliocephalus
4.) [798](0.0039)Porphyrio hochstetteri
5.) [743](0.0033)Hemiphaga novaeseelandiae
```
Binary file added tf-mobilenet-image/birdX224X224X3.rgb
Binary file not shown.
9 changes: 9 additions & 0 deletions tf-mobilenet-image/rust/image-converter/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[package]
name = "image-converter"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
image = { version = "0.23.14", default-features = false, features = ["gif", "jpeg", "ico", "png", "pnm", "tga", "tiff", "webp", "bmp", "hdr", "dxt", "dds", "farbfeld"] }
48 changes: 48 additions & 0 deletions tf-mobilenet-image/rust/image-converter/src/main.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
use std::env;
use std::fs;
use std::fs::File;
use std::io::Read;

fn main() {
let args: Vec<String> = env::args().collect();
let image_name: &str = &args[1];
let output_name: &str = &args[2];

let tensor_data = image_to_tensor(image_name.to_string(), 224, 224);
println!(
"Read image with {} bytes to write {}",
tensor_data.len(),
output_name
);
fs::write(output_name.to_string(), tensor_data).expect("Failed to write tensor")
}

fn image_to_tensor(path: String, height: u32, width: u32) -> Vec<u8> {
let mut file_img = File::open(path).unwrap();
let mut img_buf = Vec::new();
file_img.read_to_end(&mut img_buf).unwrap();
let img = image::load_from_memory(&img_buf).unwrap().to_rgb8();
let resized =
image::imageops::resize(&img, height, width, ::image::imageops::FilterType::Triangle);
let mut flat_img: Vec<f32> = Vec::new();
for rgb in resized.pixels() {
flat_img.push(rgb[0] as f32 / 255.0);
flat_img.push(rgb[1] as f32 / 255.0);
flat_img.push(rgb[2] as f32 / 255.0);
}
let bytes_required = flat_img.len() * 4;
let mut u8_f32_arr: Vec<u8> = vec![0; bytes_required];

for c in 0..3 {
for i in 0..(flat_img.len() / 3) {
// Read the number as a f32 and break it into u8 bytes
let u8_f32: f32 = flat_img[i * 3 + c] as f32;
let u8_bytes = u8_f32.to_ne_bytes();

for j in 0..4 {
u8_f32_arr[((i * 3 + c) * 4) + j] = u8_bytes[j];
}
}
}
return u8_f32_arr;
}
16 changes: 16 additions & 0 deletions tf-mobilenet-image/rust/tf-mobilenet/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[package]
name = "wasmedge-wasinn-example-tf-mobilenet-image"
version = "0.19.0"
authors = ["Second-State"]
readme = "README.md"
edition = "2018"
publish = false

[dependencies]
image = { version = "0.23.14", default-features = false, features = ["gif", "jpeg", "ico", "png", "pnm", "tga", "tiff", "webp", "bmp", "hdr", "dxt", "dds", "farbfeld"] }
wasi-nn = { version = "0.2.1"}

# This crate is built with the wasm32-wasi target, so it's separate
# from the main Wasmtime build, so use this directive to exclude it
# from the parent directory's workspace.
[workspace]
Loading