Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DeepSeek-R1 sample that uses remote model #85

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
+48 −15
Draft

Add DeepSeek-R1 sample that uses remote model #85

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
1249868
Rename sample `DeepSeek-R1` to `DeepSeek-R1-local`
Jiawei-Shao Feb 27, 2025
88ff558
Add the first version of DeepSeek-R1 besides DeepSeek-R1-local
Jiawei-Shao Feb 27, 2025
aadc51d
Don't ask users to load model from local
Jiawei-Shao Feb 27, 2025
1dc35d9
Add warm up at the end of loading model
Jiawei-Shao Feb 27, 2025
364579b
Try downloading model from hf-mirror.com
Jiawei-Shao Feb 27, 2025
449ceda
Merge branch 'main' into add-deepseek-using-remote-model
Jiawei-Shao Feb 27, 2025
cf50bd2
Run `npm run format`
Jiawei-Shao Feb 27, 2025
a823aaa
Merge branch 'main' into add-deepseek-using-remote-model
Jiawei-Shao Feb 27, 2025
1754c23
Small fix
Jiawei-Shao Feb 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 11 additions & 10 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,13 +105,14 @@ https://_your-server-ip_:8080

Some samples for this repository are based on modifications of examples from [Transformers.js](https://github.com/xenova/transformers.js)(Apache-2.0) and [MediaPipe](https://github.com/google-ai-edge/mediapipe)(Apache-2.0).

| Sample | Source | Model | Model License |
| --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| [Background Removal](./samples/image_background_removal/) | [Transformers.js](https://github.com/xenova/transformers.js/tree/main/examples/remove-background-client) | [RMBG-1.4](https://huggingface.co/briaai/RMBG-1.4) | [bria-rmbg-1.4](https://bria.ai/bria-huggingface-model-license-agreement/) |
| [Image to text](./samples/image_to_text/) | [Transformers.js](https://github.com/xenova/transformers.js/tree/main/examples/demo-site) | [ViT-GPT2](https://huggingface.co/nlpconnect/vit-gpt2-image-captioning) | Apache-2.0 |
| [Question Answering](./samples/question_answering/) | [Transformers.js](https://github.com/xenova/transformers.js/tree/main/examples/demo-site) | [DistilBERT](https://huggingface.co/distilbert/distilbert-base-cased-distilled-squad) | Apache-2.0 |
| [Summarization](./samples/summarization/) | [Transformers.js](https://github.com/xenova/transformers.js/tree/main/examples/demo-site) | [DistilBART CNN](https://huggingface.co/sshleifer/distilbart-cnn-6-6) | Apache-2.0 |
| [Phi3 WebGPU](./samples/phi3-webgpu/) | [Transformers.js](https://github.com/xenova/transformers.js/tree/v3/examples/webgpu-chat) | [Phi-3-mini-4k](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx) | [MIT](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx/blob/main/LICENSE) |
| [LLM Gemma](./samples/llm_gemma/) | [MediaPipe](https://github.com/google-ai-edge/mediapipe) | [Gemma-2B](https://www.kaggle.com/models/google/gemma/tfLite/) | [Gemma](https://ai.google.dev/gemma/terms) |
| [SD Turbo](./samples/stable_diffusion/) | [guschmue/ort-webgpu](https://github.com/guschmue/ort-webgpu/tree/master/sd-turbo) | [SD Turbo](https://huggingface.co/schmuell/sd-turbo-ort-web/) | [STABILITY AI COMMUNITY LICENSE AGREEMENT](https://huggingface.co/stabilityai/sd-turbo/blob/main/LICENSE.md) |
| [DeepSeek R1 WebGPU](./samples/deepseek-r1-webgpu-local/) | [Transformers.js](https://github.com/huggingface/transformers.js-samples/tree/main/deepseek-r1-webgpu-local) | [DeepSeek-R1](https://huggingface.co/onnx-community/DeepSeek-R1-Distill-Qwen-1.5B-ONNX) | [MIT](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B/blob/main/LICENSE) |
| Sample | Source | Model | Model License |
| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| [Background Removal](./samples/image_background_removal/) | [Transformers.js](https://github.com/xenova/transformers.js/tree/main/examples/remove-background-client) | [RMBG-1.4](https://huggingface.co/briaai/RMBG-1.4) | [bria-rmbg-1.4](https://bria.ai/bria-huggingface-model-license-agreement/) |
| [Image to text](./samples/image_to_text/) | [Transformers.js](https://github.com/xenova/transformers.js/tree/main/examples/demo-site) | [ViT-GPT2](https://huggingface.co/nlpconnect/vit-gpt2-image-captioning) | Apache-2.0 |
| [Question Answering](./samples/question_answering/) | [Transformers.js](https://github.com/xenova/transformers.js/tree/main/examples/demo-site) | [DistilBERT](https://huggingface.co/distilbert/distilbert-base-cased-distilled-squad) | Apache-2.0 |
| [Summarization](./samples/summarization/) | [Transformers.js](https://github.com/xenova/transformers.js/tree/main/examples/demo-site) | [DistilBART CNN](https://huggingface.co/sshleifer/distilbart-cnn-6-6) | Apache-2.0 |
| [Phi3 WebGPU](./samples/phi3-webgpu/) | [Transformers.js](https://github.com/xenova/transformers.js/tree/v3/examples/webgpu-chat) | [Phi-3-mini-4k](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx) | [MIT](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx/blob/main/LICENSE) |
| [LLM Gemma](./samples/llm_gemma/) | [MediaPipe](https://github.com/google-ai-edge/mediapipe) | [Gemma-2B](https://www.kaggle.com/models/google/gemma/tfLite/) | [Gemma](https://ai.google.dev/gemma/terms) |
| [SD Turbo](./samples/stable_diffusion/) | [guschmue/ort-webgpu](https://github.com/guschmue/ort-webgpu/tree/master/sd-turbo) | [SD Turbo](https://huggingface.co/schmuell/sd-turbo-ort-web/) | [STABILITY AI COMMUNITY LICENSE AGREEMENT](https://huggingface.co/stabilityai/sd-turbo/blob/main/LICENSE.md) |
| [DeepSeek R1 WebGPU (local model)](./samples/deepseek-r1-webgpu-local/) | [Transformers.js](https://github.com/huggingface/transformers.js-samples/tree/main/deepseek-r1-webgpu-local) | [DeepSeek-R1](https://huggingface.co/onnx-community/DeepSeek-R1-Distill-Qwen-1.5B-ONNX) | [MIT](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B/blob/main/LICENSE) |
| [DeepSeek R1 WebGPU (remote model)](./samples/deepseek-r1-webgpu/) | [Transformers.js](https://github.com/huggingface/transformers.js-samples/tree/main/deepseek-r1-webgpu) | [DeepSeek-R1](https://huggingface.co/onnx-community/DeepSeek-R1-Distill-Qwen-1.5B-ONNX) | [MIT](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B/blob/main/LICENSE) |
8 changes: 7 additions & 1 deletion build_scripts.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,11 @@ const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

function buildSubProjects(args) {
const PROJECT_ARRAY = ["phi3-webgpu", "deepseek-r1-webgpu-local"];
const PROJECT_ARRAY = [
"phi3-webgpu",
"deepseek-r1-webgpu",
"deepseek-r1-webgpu-local"
];
const buildCmd =
args === "--github"
? "build:github"
Expand All @@ -47,6 +51,8 @@ function copyResourcesIntoDist(args) {
const REMOTE_DEMOS_DIST = {
"samples/phi3-webgpu/dist/assets": "/assets",
"samples/phi3-webgpu/dist/index.html": "samples/phi3-webgpu/",
"samples/deepseek-r1-webgpu/dist/assets": "/assets",
"samples/deepseek-r1-webgpu/dist/index.html": "samples/deepseek-r1-webgpu/",
"samples/deepseek-r1-webgpu-local/dist/assets": "/assets",
"samples/deepseek-r1-webgpu-local/dist/index.html":
"samples/deepseek-r1-webgpu-local/"
Expand Down
2 changes: 2 additions & 0 deletions eslint.config.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ export default [
"dist/**/*.js",
"samples/phi3-webgpu/node_modules/**/*.js",
"samples/phi3-webgpu/dist/**/*.js",
"samples/deepseek-r1-webgpu/node_modules/**/*.js",
"samples/deepseek-r1-webgpu/dist/**/*.js",
"samples/deepseek-r1-webgpu-local/node_modules/**/*.js",
"samples/deepseek-r1-webgpu-local/dist/**/*.js",
"**/*.config.js",
Expand Down
13 changes: 13 additions & 0 deletions js/main.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,18 @@ let SAMPLES = [
devices: [DEVICES.GPU],
update: "2024-06-08"
},
{
id: "webgpu_deepseek_r1",
title: "DeepSeek-R1",
desc: "Language model released in Jan 2025 by DeepSeek",
sampleUrl: "./samples/deepseek-r1-webgpu/index.html",
models: ["DeepSeek R1 Distill Qwen 1.5B"],
tasks: "Text Generation",
webApis: [BACKENDS.WEBGPU],
framework: "Transformers.js",
devices: [DEVICES.GPU],
update: "2025-02-27"
},
{
id: "webgpu_deepseek_r1_local",
title: "DeepSeek-R1-local",
Expand Down Expand Up @@ -442,6 +454,7 @@ if (import.meta.env.MODE === "development") {
SAMPLES = SAMPLES.filter(
(sample) =>
sample.id !== "webgpu_phi3_mini" ||
sample.id !== "webgpu_deepseek_r1" ||
sample.id !== "webgpu_deepseek_r1_local"
);
}
Expand Down
19 changes: 15 additions & 4 deletions samples/deepseek-r1-webgpu-local/src/worker.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,17 +142,28 @@ async function load() {
baseUrl = "/web-ai-showcase";
}

env.allowLocalModels = true;
env.allowRemoteModels = true;
env.localModelPath = `${baseUrl}/models/`;
// For debugging
//env.allowLocalModels = true;
//env.allowRemoteModels = true;
//env.localModelPath = `${baseUrl}/models/`;
//env.backends.onnx.wasm.wasmPaths = `${baseUrl}/models/frameworks/ort-web/ort-web@transformers_js_3_3_1/`;
env.remoteHost = `https://hf-mirror.com/`;

// Load the pipeline and save it for future use.
await TextGenerationPipeline.getInstance((x) => {
const [tokenizer, model] = await TextGenerationPipeline.getInstance((x) => {
// We also add a progress callback to the pipeline so that we can
// track model loading.
self.postMessage(x);
});

self.postMessage({
status: "loading",
data: "Warming up"
});

// Run model with dummy input to compile shaders
const inputs = tokenizer("a");
await model.generate({ ...inputs, max_new_tokens: 1 });
self.postMessage({ status: "ready" });
}
// Listen for messages from the main thread
Expand Down
Loading