From 2710edbff22ad78867951b0fa678eaddd8af8966 Mon Sep 17 00:00:00 2001 From: andy-neuma Date: Tue, 3 Sep 2024 13:32:29 -0400 Subject: [PATCH 1/5] update readme about archival --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 5bdff29dfd159..98f60174c9544 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,10 @@ # nm-vllm -## Overview -`nm-vllm` is our supported enterprise distribution of [vLLM](https://github.com/vllm-project/vllm). +__THIS REPO HAS BEEN ARCHIVED AS OF SEPTEMBER 2024. NEURAL MAGIC IS STILL RELEASING ENTERPRISE PACKAGES RELATED TO VLLM. OUR RELASE REPO HAS JUST GONE PRIVATE.__ + +If you are looking to contribute to `vllm`, then please visit [vLLM](https://github.com/vllm-project/vllm). Also please checkout our work there as most of our contributions are still open to the public. + ## Installation From 9fd7b3e0d17c1788592b530a2c7bf8eae6242252 Mon Sep 17 00:00:00 2001 From: andy-neuma Date: Tue, 3 Sep 2024 14:30:37 -0400 Subject: [PATCH 2/5] Sasa and Jeannie --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 98f60174c9544..71f443a1df1d5 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,9 @@ # nm-vllm -__THIS REPO HAS BEEN ARCHIVED AS OF SEPTEMBER 2024. NEURAL MAGIC IS STILL RELEASING ENTERPRISE PACKAGES RELATED TO VLLM. OUR RELASE REPO HAS JUST GONE PRIVATE.__ +__THIS REPO HAS BEEN ARCHIVED AS OF SEPTEMBER 2024. NEURAL MAGIC IS STILL RELEASING ENTERPRISE PACKAGES RELATED TO VLLM. OUR RELEASE REPO HAS JUST GONE PRIVATE.__ -If you are looking to contribute to `vllm`, then please visit [vLLM](https://github.com/vllm-project/vllm). Also please checkout our work there as most of our contributions are still open to the public. +To contribute and to see our contributions to vLLM , visit [vLLM](https://github.com/vllm-project/vllm). ## Installation From 6f30cf43d9dae06fe13aed7ed33c29b4988347b2 Mon Sep 17 00:00:00 2001 From: andy-neuma Date: Wed, 4 Sep 2024 10:36:18 -0400 Subject: [PATCH 3/5] okay --- README.md | 37 +------------------------------------ 1 file changed, 1 insertion(+), 36 deletions(-) diff --git a/README.md b/README.md index 71f443a1df1d5..3bb46359ca85f 100644 --- a/README.md +++ b/README.md @@ -1,45 +1,10 @@ # nm-vllm - __THIS REPO HAS BEEN ARCHIVED AS OF SEPTEMBER 2024. NEURAL MAGIC IS STILL RELEASING ENTERPRISE PACKAGES RELATED TO VLLM. OUR RELEASE REPO HAS JUST GONE PRIVATE.__ To contribute and to see our contributions to vLLM , visit [vLLM](https://github.com/vllm-project/vllm). - -## Installation - -### PyPI -The [nm-vllm PyPi package](https://pypi.neuralmagic.com/simple/nm-vllm/index.html) includes pre-compiled binaries for CUDA (version 12.1) kernels. For other PyTorch or CUDA versions, please compile the package from source. - -Install it using pip: -```bash -pip install nm-vllm --extra-index-url https://pypi.neuralmagic.com/simple -``` - -To utilize the weight sparsity features, include the optional `sparse` dependencies. -```bash -pip install nm-vllm[sparse] --extra-index-url https://pypi.neuralmagic.com/simple -``` - -You can also build and install `nm-vllm` from source (this will take ~10 minutes): -```bash -git clone https://github.com/neuralmagic/nm-vllm.git -cd nm-vllm -pip install -e .[sparse] --extra-index-url https://pypi.neuralmagic.com/simple -``` - -### Docker - -The [`nm-vllm` container registry](https://github.com/neuralmagic/nm-vllm/pkgs/container/nm-vllm-openai) includes premade docker images. - -Launch the OpenAI-compatible server with: - -```bash -MODEL_ID=Qwen/Qwen2-0.5B-Instruct -docker run --gpus all --shm-size 2g ghcr.io/neuralmagic/nm-vllm-openai:latest --model $MODEL_ID -``` - -## Models +To view the latest releases, benchmarking, models, and evaluations from Neural Magic, visit [nm-vllm-certs](https://github.com/neuralmagic/nm-vllm-certs). Neural Magic maintains a variety of optimized models on our Hugging Face organization profiles: - [neuralmagic](https://huggingface.co/neuralmagic) From 5e72cc8b61838240488e7aa059623a918e03fa41 Mon Sep 17 00:00:00 2001 From: andy-neuma Date: Wed, 4 Sep 2024 10:51:44 -0400 Subject: [PATCH 4/5] whitespace typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3bb46359ca85f..bdfc2a3276256 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ __THIS REPO HAS BEEN ARCHIVED AS OF SEPTEMBER 2024. NEURAL MAGIC IS STILL RELEASING ENTERPRISE PACKAGES RELATED TO VLLM. OUR RELEASE REPO HAS JUST GONE PRIVATE.__ -To contribute and to see our contributions to vLLM , visit [vLLM](https://github.com/vllm-project/vllm). +To contribute and to see our contributions to vLLM, visit [vLLM](https://github.com/vllm-project/vllm). To view the latest releases, benchmarking, models, and evaluations from Neural Magic, visit [nm-vllm-certs](https://github.com/neuralmagic/nm-vllm-certs). From 02dc2b167ddd794e62e2fb05305dc46fac12d865 Mon Sep 17 00:00:00 2001 From: andy-neuma Date: Wed, 4 Sep 2024 11:03:54 -0400 Subject: [PATCH 5/5] PR feedback --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index bdfc2a3276256..a24de13f8911f 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ __THIS REPO HAS BEEN ARCHIVED AS OF SEPTEMBER 2024. NEURAL MAGIC IS STILL RELEASING ENTERPRISE PACKAGES RELATED TO VLLM. OUR RELEASE REPO HAS JUST GONE PRIVATE.__ +To learn more about nm-vllm Enterprise, visit the [nm-vllm product page](https://neuralmagic.com/nm-vllm/). + To contribute and to see our contributions to vLLM, visit [vLLM](https://github.com/vllm-project/vllm). To view the latest releases, benchmarking, models, and evaluations from Neural Magic, visit [nm-vllm-certs](https://github.com/neuralmagic/nm-vllm-certs).