Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

10x Faster New Worker #18

Merged
merged 53 commits into from
Dec 14, 2023
Merged

10x Faster New Worker #18

merged 53 commits into from
Dec 14, 2023

Conversation

alpayariyak
Copy link
Contributor

No description provided.

alpayariyak and others added 30 commits November 15, 2023 20:53
@alpayariyak
Changes to worker
e7c8de4
@alpayariyak
Update Dockerfile
8f17764
@alpayariyak
Entrypoint for testing
c34d79e
@justinmerrell
fix: clean up extra
5ca8e56
worker changes
c1b9ff1
@alpayariyak
worker changes
9661241
Merge branch 'revamp2' of https://github.com/runpod-workers/worker-vllm…
12a151d 
… into revamp2
@justinmerrell
Update download_model.py
cbde2cb
@justinmerrell
Update requirements.txt
ff3ed55
@justinmerrell
Update Dockerfile
d06fb9e
@justinmerrell
fix: added hf_transfer requirement
98af834
@alpayariyak
Merge remote-tracking branch 'origin/new-concurrency' into revamp2
7f73f41
@alpayariyak
handler changes
97480ff
@alpayariyak
edge case
7cadff2
@alpayariyak
Update handler.py
fdddeac
@alpayariyak
Download logic + other changes
820a21f
@justinmerrell
cleanup
9b9c1cf
@justinmerrell
Update handler.py
c455508
@justinmerrell
Update Dockerfile
4131b7a
@justinmerrell
Update Dockerfile
bd3ede6
@justinmerrell
Update handler.py
f958014
@alpayariyak
Update handler.py
86916bf
@justinmerrell
Update handler.py
8dbf197
@alpayariyak
Make handler stream aggregate_text
daf2f4d
@alpayariyak
Refactor handler + add non-streaming, add utils.py
b242b0c
@alpayariyak
Logging change
8d27261
@alpayariyak
Batched Tokens, cleanup
3a9393b
@alpayariyak
Packing model into worker for internal use
b4c61e6
@alpayariyak
Model Downloading, no GPU
66c23d4
@alpayariyak
Version Bump + small changes
48c520b
alpayariyak and others added 23 commits December 6, 2023 18:52
@alpayariyak
model packing, cuda version selection on build
b705544
@alpayariyak
Token counting, cuda-related changes
8dbd8f1
@alpayariyak
Cleanup
5acbbf5
@justinmerrell
feat: add docker build action
09a85ca
@alpayariyak
Final changes
8156b1d
@alpayariyak
Merge branch 'merge-ready' of https://github.com/runpod-workers/worke…
4052a17 
…r-vllm into merge-ready
@justinmerrell
Update Dockerfile
35a7447
@justinmerrell
Update Dockerfile
11e6495
@justinmerrell
Update Dockerfile
deb6f67
@justinmerrell
Update Dockerfile
bcf37e7
@alpayariyak
Multi-GPU Fix
5e38a1a
@alpayariyak
CPU Limit
f1afbcb
@alpayariyak
Small changes
8578581
@alpayariyak
bug fix
b518273
@alpayariyak
bug fix
e81007d
@alpayariyak
Update README.md
7ca8ee7
@alpayariyak
Update README.md
50f06a5
@alpayariyak
Update README.md
2dbab5c
@alpayariyak
Merge branch 'merge-ready' of https://github.com/runpod-workers/worke…
91c4779 
…r-vllm into merge-ready
@alpayariyak
Small changes
ef89868
@alpayariyak
Complete refactor, improved overall functionality
22de0b3
@alpayariyak
Concurrency modifier
bfa3703
@alpayariyak
Small fixes
d39844a
@justinmerrell justinmerrell merged commit f8484ec into main Dec 14, 2023
2 checks passed
@justinmerrell justinmerrell deleted the merge-ready branch December 14, 2023 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@justinmerrell justinmerrell Awaiting requested review from justinmerrell

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alpayariyak @ashleykleynhans @justinmerrell