Code

10x Faster New Worker#18

Code

Merged

justinmerrell merged 53 commits intomainrunpod-workers/worker-vllm:mainfrom merge-readyrunpod-workers/worker-vllm:merge-readyCopy head branch name to clipboardDec 14, 2023

+407-984

Commits

Commits on Nov 16, 2023

• Changes to worker

  alpayariyak

  committed
  e7c8de4

  Copy full SHA for e7c8de4

• Update Dockerfile

  alpayariyak

  committed
  8f17764

  Copy full SHA for 8f17764

Commits on Nov 20, 2023

• Entrypoint for testing

  alpayariyak

  committed
  c34d79e

  Copy full SHA for c34d79e

Commits on Nov 22, 2023

• fix: clean up extra

  justinmerrell

  committed
  5ca8e56

  Copy full SHA for 5ca8e56

• worker changes

  

  Ubuntu

  committed
  c1b9ff1

  Copy full SHA for c1b9ff1

• worker changes

  

  alpayariyak

  authored and

  Ubuntu

  committed
  9661241

  Copy full SHA for 9661241

• Merge branch 'revamp2' of https://github.com/runpod-workers/worker-vllm into revamp2

  

  Ubuntu

  committed
  12a151d

  Copy full SHA for 12a151d

• Update download_model.py

  justinmerrell

  committed
  cbde2cb

  Copy full SHA for cbde2cb

• Update requirements.txt

  justinmerrell

  committed
  ff3ed55

  Copy full SHA for ff3ed55

• Update Dockerfile

  justinmerrell

  committed
  d06fb9e

  Copy full SHA for d06fb9e

• fix: added hf_transfer requirement

  justinmerrell

  committed
  98af834

  Copy full SHA for 98af834

Commits on Nov 26, 2023

• Merge remote-tracking branch 'origin/new-concurrency' into revamp2

  alpayariyak

  committed
  7f73f41

  Copy full SHA for 7f73f41

Commits on Nov 28, 2023

• handler changes

  alpayariyak

  committed
  97480ff

  Copy full SHA for 97480ff

• edge case

  alpayariyak

  committed
  7cadff2

  Copy full SHA for 7cadff2

• Update handler.py

  alpayariyak

  committed
  fdddeac

  Copy full SHA for fdddeac

Commits on Nov 29, 2023

• Download logic + other changes

  alpayariyak

  committed
  820a21f

  Copy full SHA for 820a21f

Commits on Nov 30, 2023

• cleanup

  justinmerrell

  committed
  9b9c1cf

  Copy full SHA for 9b9c1cf

• Update handler.py

  justinmerrell

  committed
  c455508

  Copy full SHA for c455508

• Update Dockerfile

  justinmerrell

  committed
  4131b7a

  Copy full SHA for 4131b7a

• Update Dockerfile

  justinmerrell

  committed
  bd3ede6

  Copy full SHA for bd3ede6

• Update handler.py

  justinmerrell

  committed
  f958014

  Copy full SHA for f958014

• Update handler.py

  alpayariyak

  committed
  86916bf

  Copy full SHA for 86916bf

• Update handler.py

  justinmerrell

  committed
  8dbf197

  Copy full SHA for 8dbf197

Commits on Dec 1, 2023

• Make handler stream aggregate_text

  alpayariyak

  committed
  daf2f4d

  Copy full SHA for daf2f4d

• Refactor handler + add non-streaming, add utils.py

  alpayariyak

  committed
  b242b0c

  Copy full SHA for b242b0c

• Logging change

  alpayariyak

  committed
  8d27261

  Copy full SHA for 8d27261

Commits on Dec 5, 2023

• Batched Tokens, cleanup

  alpayariyak

  committed
  3a9393b

  Copy full SHA for 3a9393b

Commits on Dec 6, 2023

• Packing model into worker for internal use

  alpayariyak

  committed
  b4c61e6

  Copy full SHA for b4c61e6

• Model Downloading, no GPU

  alpayariyak

  committed
  66c23d4

  Copy full SHA for 66c23d4

• Version Bump + small changes

  alpayariyak

  committed
  48c520b

  Copy full SHA for 48c520b

• model packing, cuda version selection on build

  alpayariyak

  committed
  b705544

  Copy full SHA for b705544

• Token counting, cuda-related changes

  alpayariyak

  committed
  8dbd8f1

  Copy full SHA for 8dbd8f1

• Cleanup

  alpayariyak

  committed
  5acbbf5

  Copy full SHA for 5acbbf5

Commits on Dec 8, 2023

• feat: add docker build action

  justinmerrell

  committed
  09a85ca

  Copy full SHA for 09a85ca

• Final changes

  alpayariyak

  committed
  8156b1d

  Copy full SHA for 8156b1d

• Merge branch 'merge-ready' of https://github.com/runpod-workers/worker-vllm into merge-ready

  alpayariyak

  committed
  4052a17

  Copy full SHA for 4052a17

Commits on Dec 9, 2023

• Update Dockerfile

  justinmerrell

  committed
  35a7447

  Copy full SHA for 35a7447

• Update Dockerfile

  justinmerrell

  committed
  11e6495

  Copy full SHA for 11e6495

• Update Dockerfile

  justinmerrell

  committed
  deb6f67

  Copy full SHA for deb6f67

• Update Dockerfile

  justinmerrell

  committed
  bcf37e7

  Copy full SHA for bcf37e7

• Multi-GPU Fix

  alpayariyak

  committed
  5e38a1a

  Copy full SHA for 5e38a1a

• CPU Limit

  alpayariyak

  committed
  f1afbcb

  Copy full SHA for f1afbcb

• Small changes

  alpayariyak

  committed
  8578581

  Copy full SHA for 8578581

• bug fix

  alpayariyak

  committed
  b518273

  Copy full SHA for b518273

• bug fix

  alpayariyak

  committed
  e81007d

  Copy full SHA for e81007d

Commits on Dec 12, 2023

• Update README.md

  alpayariyak

  authored
  7ca8ee7

  Copy full SHA for 7ca8ee7

• Update README.md

  alpayariyak

  authored
  50f06a5

  Copy full SHA for 50f06a5

• Update README.md

  alpayariyak

  authored
  2dbab5c

  Copy full SHA for 2dbab5c

• Merge branch 'merge-ready' of https://github.com/runpod-workers/worker-vllm into merge-ready

  alpayariyak

  committed
  91c4779

  Copy full SHA for 91c4779

• Small changes

  alpayariyak

  committed
  ef89868

  Copy full SHA for ef89868

Commits on Dec 13, 2023

• Complete refactor, improved overall functionality

  alpayariyak

  committed
  22de0b3

  Copy full SHA for 22de0b3

• Concurrency modifier

  alpayariyak

  committed
  bfa3703

  Copy full SHA for bfa3703

• Small fixes

  alpayariyak

  committed
  d39844a

  Copy full SHA for d39844a